Subject: Re: [Ietf-caldav] [Fwd: draft-reschke-http-addmember-00]

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 14:54:38 +0100

Roy T. Fielding wrote:

> POST is always an attempt to store state.  It is the response that
> tells you whether the state has been stored.  No POST can be
> interpreted by intermediaries, just as no "ADDMEMBER" can either.
> The system gains nothing from adding a new method.  It might have
> gained something by not allowing HTML forms to use POST for safe
> actions, but that debate ended years ago.  Creating new unsafe
> actions that do the same as what POST has already been defined
> to do is absurd.

Well, ADDMEMBER does not do the same thing. It's specced to do *one* 
very specific thing, while POST basically allows the server to do 
anything it likes.

>> Had HTTP a means for extending a method to declare this additional
>> expectation (as described in the draft's A.3 using RFC 2774), I agree
>> that POST + extension would be appropriate.
> 
> 
> It doesn't because it never needed it.

OK, let's step back and look at the situation that caused me to make 
this proposal.

Two separate communities (CalDAV/GroupDav and Atompub) encountered the 
same issue: what's the best way to create a new resource on an HTTP 
server when I can't use PUT as the URI space may be entirely 
server-controlled?

1) CalDav's approach is: use PUT to an arbitrary member URI of the 
container; then let the server automagically move it somewhere else, and 
report that in the Location response header 
(<http://greenbytes.de/tech/webdav/draft-reschke-http-addmember-00.html#rfc.section.A.2>).

2) Atom's approach is to use service URIs to which the client can POST, 
and to discover these service URIs by parting entity (feed) bodies.

Somehow I have the (perhaps naive) wish that identical requirements 
should lead to common protocol constructs.

1) Seems to break PUT semantics;

2) Is specific to Atom and not applicable to other protocols.

 > ...
> The media type is not limited in HTTP and it is not equivalent
> to data format.  The media type tells the recipient how to process
> the message given the method semantics.  That is why sending the
> same data as "text/html" is functionally different from sending it
> as "text/plain" -- the two messages have different semantics.
> 
> That will hold for anything compliant with HTTP/1.1.

So what would be your recommendation to the authors of CalDav and the 
Atom Publishing Protocol? What's the best way to add a new member to a 
container (storing the entity just like PUT, but letting the server 
decide on the URI)?

Best regards, Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 15:45:34 +0100

Jamie Lokier wrote:
> Julian Reschke wrote:
> ADDMEMBER is no different from POST in this respect:
> 
> You still have to "discover the service URI" to use with ADDMEMBER.

No, it would be the container resource itself. For CalDav, the calendar 
collection; for Atompub, the feed resource itself.

Best regards, Julian

---

From: Stefan Eissing <[email-masked]>
Date: Thu, 17 Feb 2005 17:34:02 +0100

Am 17.02.2005 um 17:05 schrieb Cyrus Daboo:
> Hi Julian,
>
> --On February 17, 2005 2:54:38 PM +0100 Julian Reschke  
> <[email-masked]> wrote:
>
>> (<http://greenbytes.de/tech/webdav/draft-reschke-http-addmember 
>> -00.html#r
>> fc.section.A.2>).
>
> I want to add that in the CalDAV case this URI naming restriction will  
> also apply to COPY and MOVE methods too, not just PUT. So we do need  
> to come up with a solution that at least works with those methods too.  
> Of course it is possible to emulate COPY and MOVE using PUT (client  
> downloads original data, creates it in new location) and DELETE  
> (remove old data in the case of MOVE) - but we should really have the  
> proper atomic operations for this.

It seems to me that however you solve the PUT/ADDMEMBER/POST problem,  
it can be used to solve the MOVE and COPY scenarios as well, at least  
for single, plain resources:

PUT/ADDMEMBER/POST a new, empty resource -> 201 with Location
COPY/MOVE to that new Location

Would this work?

//Stefan

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 18:18:25 +0100

Jamie Lokier wrote:
> Why can't you POST to the container resource?
> 
> That's what POST is for, after all.

Because in general, I will have no idea what the server will do with the 
POST.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 18:19:35 +0100

Cyrus Daboo wrote:
> Hi Jamie,
> 
> --On February 17, 2005 4:41:42 PM +0000 Jamie Lokier 
> <[email-masked]> wrote:
> 
> The WebDAV rfc has the following statement in it in Section 5.3 as a 
> justification for creating a new method (MKCOL in this case) rather than 
> using a special POST operation:
> 
>>    While the POST method is sufficiently open-ended that a "create a
>>    collection" POST command could be constructed, this is undesirable
>>    because it would be difficult to separate access control for
>>    collection creation from other uses of POST.
> 
> 
> Wouldn't the same issue be relevant here?
> 
> Interestingly the current WebDAV ACL document does not appear to mention 
> POST at all - even in the 'Normative' Method Privilege Table in Appendix 
> B. Is there a reason for that?

Yes. As you can do anything with POST, it's impossible to state what 
privileges you'll need.

> Whatever solution we come up with it ideally needs to work seamlessly 
> with WebDAV ACL.

That would be preferrable.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:21:06 +0100

Jan Algermissen wrote:
> Julian Reschke wrote
> 
>> Because in general, I will have no idea what the server will do with 
>> the POST.
> 
> 
> Hmm...the meaning of POST is that the server will process the message. 
> If you receive a 201,
> the server has created a resource.

But *what* does it contain? In general, I can not assume it will store 
the entity as sent (like PUT).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:39:12 +0100

Scott Lawrence wrote:
> That statement misses the point - it may be true that it's difficult to
> express the access control based just on the method, but that doesn't
> mean that it's difficult to implement appropriate access control in
> either the client or the server.  The method alone does not specify the
> operation - indeed, in the case of POST the full specification of the
> operation is deliberately expanded to include the body mime type and the
> body content.
> 
> I don't think you've shown how what you're trying to do is any different
> from what POST has always done.

It's a aubset with well-defined semantics. I consider this a feature.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:42:56 +0100

Jamie Lokier wrote:
> In general, you don't know what it will do with ADDMEMBER either.

I do, because I'll make the assumption that a server that advertises 
support for ADDMEMBER indeed implements what the spec says. Are you 
saying that I can't make that assumption?

> This is better illustrated with PUT.  Clearly, PUT is intended for
> creating or modifying an entity at a client-specific URI.
> 
> I have the impression you expect this is literally what PUT should do:
> that afterwards, GET from that URI, if permitted, will return the last
> entity that was put.

Yes.

> Plenty of HTTP servers which don't work like that.  Often, you can PUT
> to a URI ending in ".cgi" or ".php" for example, and (possibly
> depending on the put'd content type, possibly not) GET will not return
> the same entity; instead it will initiate processing using the
> resource now present at that URI.

I'd consider that a bug.

> You can PUT to a URI ending in ".html" and with content-type
> "text/html" even, but with some servers, GET from the same URI will
> initiate content negotiation and may return that entity, a different
> one, a transcoded one, a tidied one, etc., according to server-defined
> rules.  (Section 5.4 of WebDAV, RFC 2518 has more to say on this).

In which case the server shouldn't have supported PUT on that URI in the 
first place.

> Given that PUT has somewhat open-ended semantics - won't ADDMEMBER
> have similarly open-ended semantics?

I don't agree that it has open-ended semantics. But it's good that you 
raise this issue, because it's essential to understand the rational for 
ADDMEMBER.

> How is an ADDMEMBER whose effect is as open-ended as PUT, but which
> creates a resource at a server-selected URI, any different from POST -
> even for access control?
> 
> It seems to me that if you're considering making access control
> decisions on the difference between ADDMEMBER and POST, then you
> should really be making those decisions on the known behaviour of the
> request URI.  If you don't recognise the request URI, then you don't
> really know what ADDMEMMER _or_ POST will do, in general.

See above.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:47:53 +0100

Jamie Lokier wrote:
>                       ^^^^^^^^
> You cannot assume PUT will store the entity as sent either.
> 
> Read section 5.4 "Source Resources and Output Resources" in RFC 2518, WebDAV.
> (That part's not specific to WebDAV; it's a just a good explanation).

In fact, it says:

"However, if remote editing of the source resource(s) is desired, the 
source resource(s) should be given a location in the URI namespace. This 
source location should not be one of the locations at which the 
generated output is retrievable, since in general it is impossible for 
the server to differentiate requests for source resources from requests 
for process output resources."

>>From the client's point of view, once you have PUT an entity, in
> general you have no way to retrieve that exact entity.  Many servers
> will process the resource you created to produce the response to GET,
> instead of merely returning the entity you sent earlier.
> 
> Because you can't necessarily retrieve the entity sent, you can't make
> assumptions about what the server stores.  It might store only part of
> what you sent, or a transformed version.  What it actually stores --
> and the side effects of doing so -- are quite open.

You keep saying this, but it would be nice if you could point out where 
that kind of behaviour is described. RFC2616 says:

"The PUT method requests that the enclosed entity be stored under the 
supplied Request-URI."

And that's certainly what today's WebDAV servers implement and WebDAV 
clients do expect.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 21:07:15 +0100

Scott Lawrence wrote:
> On Thu, 2005-02-17 at 19:39 +0100, Julian Reschke wrote:
> 
>>Scott Lawrence wrote:
>>
> But you could just as easily and precisely define those semantics by
> using POST and defining the mime type and operations it supports.

In which case I couldn't use the content-type of my actual request body 
for the Content-Type request header, right?

> You won't get caught be firewalls and proxy servers that think they know
> better about what methods are legitimate (which you most assuredly will
> if you create a new method - ask the WebDav implementors), and you won't
> have changed the semantics of the method at all.

I am one of these WebDAV implementors, thanks. I haven't had any issues 
with issues for a long time.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 21:48:59 +0100

Jamie Lokier wrote:
>  ...
> And so, for ADDMEMBER to be useful, you need to specify it more
> tightly than "PUT except the created resource URI is selected
> differently".
> 
> If you do that, it makes sense.  If you don't, then ADDMEMBER is no
> different from POST.
> ...

OK, to summarize: your point is that PUT doesn't have the strict 
semantics (that I claim it has). Let's just agree that we disagree here.

> That said, even if you do specify ADDMEMBER more tightly, I still
> don't see how the difference would benefit you.  You say it could be
> useful to an intermediary, but can you give a plausible example?

Did I?

Best regards, Julian

---

From: "Jim Whitehead" <[email-masked]>
Date: Thu, 17 Feb 2005 13:04:39 -0800

A few comments on extending the functionality defined in -addmember-00:

1) One problem clients currently encounter with PUT is determining whether
they have permission to write to a specific location. At present, a client
must submit a PUT (or ADDMEMBER) with the entire entity body, then wait for
the response, or the authentication challenge. It would be very nice to have
a TESTPUT/TESTADDMEMBER method that sees whether a write (via PUT or
ADDMEMBER) would succeed, or if an authentication challenge is needed.

Yes, a server supporting ACLs could be queries to see if writing were
allowed, but it still doesn't provide an easy way to pre-determine the
authentication information.

2) It would be very helpful to allow ADDMEMBER to have atomic PUT+PROPPATCH
capabilities, to write both the resource and its metadata in one action.

- Jim

---

From: Helge Hess <[email-masked]>
Date: Thu, 17 Feb 2005 22:11:09 +0100

On 17. Feb 2005, at 22:04 Uhr, Jim Whitehead wrote:
> the response, or the authentication challenge.

Isn't that already solved with Expect: 100-continue?

http://zvon.org/tmRFC/RFC2616/Output/chapter14.html#sub20

Greets,
   Helge
-- 
http://docs.opengroupware.org/Members/helge/
OpenGroupware.org

---

From: Lisa Dusseault <[email-masked]>
Date: Thu, 17 Feb 2005 13:42:21 -0800

Rather than a new method, an ADDMEMBER specification could require 
support for the Expect mechanism which solves that problem rather 
neatly for any method.

Lisa

---

From: "Jim Whitehead" <[email-masked]>
Date: Thu, 17 Feb 2005 13:51:05 -0800

I don't think Expect allows you to easily determine what access control
challenge you'll receive. Am I wrong?

- Jim 

> -----Original Message-----
> From: Lisa Dusseault [mailto:[email-masked]] 
> Sent: Thursday, February 17, 2005 1:42 PM
> To: [email-masked]
> Cc: 'Julian Reschke'; 'Webdav WG'
> Subject: Re: [Ietf-caldav] [Fwd: draft-reschke-http-addmember-00]
> 
> Rather than a new method, an ADDMEMBER specification could 
> require support for the Expect mechanism which solves that 
> problem rather neatly for any method.
> 
> Lisa
> 
> > 1) One problem clients currently encounter with PUT is determining 
> > whether they have permission to write to a specific location. At 
> > present, a client must submit a PUT (or ADDMEMBER) with the entire 
> > entity body, then wait for the response, or the authentication 
> > challenge. It would be very nice to have a TESTPUT/TESTADDMEMBER 
> > method that sees whether a write (via PUT or

---

From: Helge Hess <[email-masked]>
Date: Thu, 17 Feb 2005 22:59:08 +0100

> challenge you'll receive. Am I wrong?

Not sure what you mean by "access control challange". It allows you to 
determine whether authentication is required or valid prior pushing the 
content. Access control must be determined with WebDAV ACL or something 
similiar.

It goes like this:
1) client sends HTTP request header with expect: 100-continue
2) server checks the 'authorization' header
3) if authorization fails for the given method+header, the server sends
    a 401 with www-authenticate
4) if auth is OK, it sends a 100 Continue

Pretty much what you were asking for initially?

Greets,
   Helge

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 22:28:35 +0100

Helge Hess wrote:
>> whether

Sort of. The problem is that it's extremely hard to implement using the 
Java servlet API (but that's "just" an implementation issue...).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 18 Feb 2005 00:05:56 +0100

Jamie Lokier wrote:
> which I took to mean you wanted to be sure of what would happen,
> perhaps with a view to a security policy or intermediary behaviour.
> My mistake.
> 
> So I'll humbly rephrase my question:
> 
> Even if you do specify ADDMEMBER more tightly (than POST), I still
> don't see how the difference would benefit you.  You say it is useful
> to have an idea what the server will do with a POST, in general.  In
> general means an arbitrary URI with unknown properties, because if
> it's a URI with known properties, you know exactly what POST will do.
> 
> How does specifying ADDMEMBER to have tighter semantics than PUT and
> POST benefit you when sending the request to an arbitrary URI with
> unknown properties?  Can you give an example?

The same way as COPY, LOCK, MOVE, VERSION-CONTROL, 
<insert-your-favorite-method-here>...

I don't think I understand your point. Any operation *can* be marshalled 
through POST, but that doesn't mean it's the best way to do it.

Best regards, Julian

---

From: "Jim Whitehead" <[email-masked]>
Date: Thu, 17 Feb 2005 15:27:57 -0800

> Pretty much what you were asking for initially?

Er, yes. Thanks.

- Jim

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 17 Feb 2005 15:50:39 -0800

On Feb 17, 2005, at 5:54 AM, Julian Reschke wrote:
> container; then let the server automagically move it somewhere else,  
> and report that in the Location response header  
> -00.html#rfc.section.A.2>).

That is not allowed in HTTP.

> 2) Atom's approach is to use service URIs to which the client can  
> POST, and to discover these service URIs by parting entity (feed)  
> bodies.

That is allowed.

> 2) Is specific to Atom and not applicable to other protocols.

No, it isn't. The client must "discover" the server's URI somehow,
whether that be via hypertext, save under dialogs, or an atom link.
They are all equivalent discovery mechanisms.  When Atom does a POST
it will send a representation of an entry (which happens to be just
another resource).  When a generic HTTP client does a POST it will
send a representation of a resource.

The server receives a POST message targeting a URI and containing
a representation.  It can do one of three things: 1) deny the request
with method not allowed (or any other error); 2) treat all POSTs
as the same; or 3) decide, based on the URI alone or in combination
with the representation, how to process the request.

In all cases, the client can assume the addmember semantics of POST
without any interoperability issues.  It is the server's responsibility
to provide resource references to the client that match the actions
called for by whatever it is the client is doing, and even if the
client picks some random URI the server is more than capable of
providing an appropriate response.

Geoff asked what happens if the resource already supports POST
for some other purpose.  The answer is one of

    1) if the implementation supports the addmember purpose
       in addition to its other duties, it will return 201 Created;

    2) if the implementation does not allow subsidiary resources
       to be created, then

       a) if the implementation is written according to HTTP,
          it will return 415 Unsupported Media Type; or

       b) if the implementation is clueless, it will attempt to
          parse the media type that it expects and will fail with
          some form of 4xx response. [Note that this is the case
          whether the method is POST or ADDMEMBER -- clueless
          implementations do not check the method either.]

       c) if the implementation is totally ignorant of all things
          HTTP, it will respond with 200 and a friendly error message
          about how this site only works with MSIE 5 (again,
          regardless of the method used).

So, the answer to Geoff's question is a new method is not necessary
to distinguish one POST from another POST because this was already
considered by HTTP in 1992

   http://www.w3.org/Protocols/HTTP/Methods/Post.html

and codified in RFC 1945, 2068, and 2616.  The reason it allows
the server so much leeway in its handling of the request is because
only the resource can determine, according to its own implementation
and purpose in life, the most appropriate way to store subsidiary
resources.  ANY other specification would assume that an HTTP
server is just another form of file server like FTP (which is not
its purpose in life, as I've explained a million times before).

If the client sends random unsafe requests to a Web server,
it will obtain randomly unsafe results regardless of the method
used.  If the server provides a URI as an authorable resource,
then POST to a collection has the semantics of adding a member
to that collection -- the semantics have not changed just because
webdav chose to ignore that feature of HTTP.

....Roy

---

From: Geoffrey M Clemm <[email-masked]>
Date: Fri, 18 Feb 2005 00:10:51 -0500

My main concern is that because of the popularity of SOAP,
many/most implementations fall into category 2b (i.e., "clueless"),
so a resource will say that it supports the POST operation,
but will actually fail it with some kind of 4xx response
because it cannot parse the body, or in the worst case,
will successfully parse the body and execute some potentially
harmful SOAP operation that was never intended by the client
(the client just wanted a subsidiary whose content was that body
to be created).

I'm rather surprised that clueless (e.g., SOAP :-) implementations
would parse the body of an unknown method (e.g., ADDMEMBER)
and treat it as if it were a POST call.  I would have expected
SOAP implementations to limit themselves to parsing only POST bodies.
But Roy knows way more about that kind of thing than I do ... 

But even if a clueless implementation will try to parse the
body of an unknown method like ADDMEMBER, I assume it is very
unlikely that it would say that it supports the ADDMEMBER
method on that resource, so that at least would give the client
a way of avoiding the problem (i.e., by first asking the resource
if it supports the ADDMEMBER function, before attempting it).

Cheers,
Geoff

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 18 Feb 2005 09:26:10 +0100

I think it is also not clear if its a hop-to-hop or end-to-end header. 
So chances of getting it working in an uncontrolled environment are not 
good.

//Stefan

Am 17.02.2005 um 22:28 schrieb Julian Reschke:

> Sort of. The problem is that it's extremely hard to implement using 
> the Java servlet API (but that's "just" an implementation issue...).
>
> Best regards, Julian
>
> --

---

From: Julian Reschke <[email-masked]>
Date: Fri, 18 Feb 2005 09:58:36 +0100

> So chances of getting it working in an uncontrolled environment are not 

 From <http://greenbytes.de/tech/webdav/rfc2616.html#header.expect>:

"The Expect mechanism is hop-by-hop: that is, an HTTP/1.1 proxy MUST 
return a 417 (Expectation Failed) status if it receives a request with 
an expectation that it cannot meet. However, the Expect request-header 
itself is end-to-end; it MUST be forwarded if the request is forwarded."

This seems to indicate that expectations not defined in RFC2616 (and the 
only one deployed there is "100-continue") will be next to impossible to 
deploy. Usage of "100-continue" shouldn't be problematic, though (expect 
that servers may not implement the amount of request header checking one 
may wish).

Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Fri, 18 Feb 2005 08:38:02 -0800

Yes, I agree there would need to be some added definition for it to 
work clearly and interoperably.  But I can't see how the mechanism 
itself is flawed other than being incompletely described and 
unimplemented :)

Lisa

On Feb 18, 2005, at 12:26 AM, Stefan Eissing wrote:

> So chances of getting it working in an uncontrolled environment are 
> not good.
>
> //Stefan
>

---

From: Julian Reschke <[email-masked]>
Date: Mon, 21 Feb 2005 22:46:16 +0100

Jamie Lokier wrote:
>>I fear that I may have lost the thread of your comment... you cannot use
>>Content-Type to send anything _but_ the content type of your request
>>body.  
> 
> 
> Roy said that POST can take into account the request content type to
> decide the action that is taken.
> 
> I believe Julian wants to "create resource" with an arbitrary content
> type - _without_ the content type having any effect on the action that
> is taken to create the resource.

Correct.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 21 Feb 2005 16:34:48 -0800

>> is taken to create the resource.

General-purpose HTTP servers will not allow that for security
reasons.  They use the media type internally as an extensibility
mechanism.

....Roy

---

From: Mark Baker <[email-masked]>
Date: Mon, 21 Feb 2005 22:46:55 -0500

Scott,

On Mon, Feb 21, 2005 at 09:56:29PM -0500, Scott Lawrence wrote:
> You already have what you need in POST -
> defining a new method does not add any new capability

Let me ask this; why do we have PUT when a valid effect of a POST
could also be to set the state of a resource?

IMO, it's because there are advantages to having messages which reflect
the expectations of their sender.

Mark.
-- 
Mark Baker.   Ottawa, Ontario, CANADA.        http://www.markbaker.ca

---

From: Geoffrey M Clemm <[email-masked]>
Date: Mon, 21 Feb 2005 21:24:50 -0700

Jamie Lokier <[email-masked]> wrote on 02/21/2005 02:32:47 PM:
> Geoffrey M Clemm wrote:
> I think that's a reasonable assumption.
> 
> But you should always try to avoid accessing unknown resources, even
> to query their capabilities:  There are implementations where sending
> OPTIONS to them will be interpreted as a POST or stateful GET :)

I'm not sure what you mean by "avoid accessing unknown resources".
The only way a client "knows" about a resource is by doing some discovery
method on it such as OPTIONS.

Cheers,
Geoff

---

From: Bernard Desruisseaux <[email-masked]>
Date: Tue, 22 Feb 2005 00:19:23 -0500

Ideally we would have a solution that would work for all methods that
may request the creation of a new resource (e.g, LOCK, PUT, COPY, MOVE,
MKCOL, MKWORKSPACE, MKACTIVITY, MKCALENDAR, ...).

Could the solution be as simple as specifying an empty query component
(e.g., "?") in the Request-URI for those methods, and have the methods
return the Location header as part of their response?

For instance,

   PUT /calendar/bernard/?

would request the server to create a new resource in the collection
/calendar/bernard/ and let the server chose the name of the resource.

Is there already a defined use for the query component for the methods
mentionned above?

Cheers,
Bernard

Cyrus Daboo wrote:

> also apply to COPY and MOVE methods too, not just PUT. So we do need to 
> come up with a solution that at least works with those methods too. Of 
> course it is possible to emulate COPY and MOVE using PUT (client 
> downloads original data, creates it in new location) and DELETE (remove 
> old data in the case of MOVE) - but we should really have the proper 
> atomic operations for this.
>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 22 Feb 2005 00:21:05 -0800

> could also be to set the state of a resource?

Because PUT means set the state of this resource, whereas POST
does not.

> the expectations of their sender.

Umm, think about that sentence and you will find it has no content.
Messages reflect the instruction of the sender.  POST does that.

What you are really saying is that there are advantages to the
client knowing the nature of a resource, encoding that nature
into the set of methods used, and thus forcing how its state
will be impacted by each messaged action.  That is what Julian
wants in an ADDMEMBER method.  You, of all people, should
be the first to recognize that slippery slope.

Yes, it has advantages, but it also has far-reaching disadvantages
that HTTP was designed to prevent. It causes clients to become
coupled to particular "types" of resources on the server, which
is just another from of implementation-dependency.  It leads to
absolutely moronic implementations that have to test each resource
for the set of methods it implements before performing a set of
actions that would have the same result as a generic POST.
[Which is to say that it will suck every bit as much as the other
methods introduced by webdav and its prodigy that created arbitrary
resource types for collections, versions, and now calendars.]

POST, in contrast, is a generic semantic that allows the resource
to determine its implementation-specific effects.  POST to a
collection means add a member.  POST to a PUT-able resource means
an atomic append. POST to a messaging gateway means "post this".
POST to any other resource means "process this". Note, however,
that to a client they all mean the same thing: take this data
and apply it in accordance with your nature.  The client does
not need to know anything more because the client already knows
what it wants to do (and knowing more cannot help it, since
only the server is allowed to know the implementation).

PUT versus PATCH might lead you to an interesting comparison,
at least in terms of seeing when two methods do have different
semantics.  A better example is POST versus PATCH, given that
PATCHing a collection could be defined as a set of add/remove
operations on the collection state. One could easily define a
patch media type for that, but there are also good reasons not to.

POST versus ADDMEMBER, however, does not compare at all because
POST is already defined to be "add a member." POST only seems to
have multiple definitions because the other definitions are
examples of various forms of partial state change within a
resource that may consist of identifiable subcomponents. It is
really just one semantic definition that manifests in different
ways based upon how the implementer views the resource to which
the POST is applied.  A file is just an ordered collection of bytes.
A collection is just a file with a compound media type. A gateway
is just a pipe, and a pipe is just a file with scrolling-window
state over time.

The IETF could give every form of implementation a different
type and every type its own set of methods for manipulating
state, but all that will accomplish is the death of another
protocol.  And no, I don't care what SOAP does via POST,
since SOAP has already proven to be a miserable failure.

Cheers,

Roy T. Fielding                            <http://roy.gbiv.com/>
Chief Scientist, Day Software              <http://www.day.com/>

---

From: Mark Baker <[email-masked]>
Date: Tue, 22 Feb 2005 08:37:58 -0500

On Tue, Feb 22, 2005 at 12:21:05AM -0800, Roy T. Fielding wrote:
> client knowing the nature of a resource,
[lots of other stuff that I agree with snipped]

Ah, so you're saying that ADDMEMBER isn't uniform?  Sorry, I wasn't
able to extract that from your other messages.  But can you please
explain your reasoning behind that belief?  From my POV, the draft
defines ADDMEMBER semantics to be a small, (seemingly) uniform
adjustment upon PUT semantics.  Can you give an example of a resource
for which ADDMEMBER wouldn't make sense?

Mark.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 17:36:02 +0100

Jamie Lokier wrote:
>>   The only way a client "knows" about a resource is by doing some
>>   discovery method on it such as OPTIONS.
> 
> 
> No: the way a client knows about a resource is that something asks the
> client to access the resource.
> 
> For example, a WebDAV client accesses a resource because the user
> tells the client to.  The user is allowing the client to assume that
> WebDAV methods are ok to try on the resource.

You're making assumptions about the knowledge a user has that IMHO are 
simply questionable. For instance, a user my use IE to follow a 
hyperlink to an Office document, and Office will (under some 
circumstances) LOCK the document, and later PUT and UNLOCK it. At least 
95% of the users will not be aware of what was going on technically.

> A CalDAV or Atom client accesses a resource that it is told to access.
> 
> In all cases, the client is told to try certain methods on certain
> URLs, by the user (or other agent).
> 
> If the user asks a client to try WebDAV methods on a resource that
> they don't know it's safe to try WebDAV methods on, then they can
> cause all sorts of havoc.

If that causes havoc, that's because the server is broken (be it 
misconfigured or just buggy).

> For example, some URLs will treat any method as form submission, even
> OPTIONS, so users should never ask a client which uses OPTIONS to
> access those URLs, because that is dangerous.

See above.

> There is simply no way for the client to "discover" whether a given
> URL supports the behaviour it's been asked to do, without causing
> potential harmful side effects.

Jamie, lots of clients are doing this today. I'm not aware of any 
"havoc" causing this.

> This is why the client has to assume the user gave it a URL with known
> behaviour in the first place.
> 
> This is why a client whose job is "add a member to a collection" may
> _assume_ that it's ok to use POST for this, if POST is specified as
> having the right behaviour _on that resource_.

So what do I do if a have a resource that accepts HTML form posts, SOAP 
requests and ADDMEMBER-like semantics on the same URL?

> The only technical reason why a new method would be required in HTTP
> is if the message semantics were different.  For example, POST is

Of course. That's why they are defined to be different from POST.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 17:49:34 +0100

> reasons.  They use the media type internally as an extensibility

Yes. But I'm not speaking of general-purpose servers, but of servers 
that *do* want to allow that (just like every off-the-shelf WebDAV server).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 17:56:44 +0100

Scott Lawrence wrote:
> On Mon, 2005-02-21 at 21:28 +0000, Jamie Lokier wrote:
> 
>>Scott Lawrence wrote:
>>
> But setting the content type of the resulting resource _is_ having an
> effect !?!
> 
> The point that I think Roy and I have both failed to communicate is
> this: POST does not in itself define what happens - the thing to which
> the request is POSTed (identified by the full URL) makes that decision. 

Well :-) I certainly am aware of that fact. That's the *reason* of 
proposing an alternate method that has strict semantics instead.

> That thing is free to use the URL, the content body, the time of day,
> the phase of the moon, or whatever it wants to when deciding what to do
> with the resource _and_you_get_to_choose_ because you are creating that
> thing.  If you want to define how you use POST in such a way that you
> say that the Content-Type of the body should be preserved in the new
> resource you are adding to the server, that's fine.  If you want to post
> an entity body in the request that encapsulates metadata about the
> resource with the resource itself (as is done in rfc1867 form file
> upload), that's fine too.  You already have what you need in POST -
> defining a new method does not add any new capability - it just adds
> ambiguity for the servers that don't know what it is (all intermediate
> systems already know that POST is not idempotent and the response to it
> can only be cached if the response so indicates - that's all you need).

I am fully aware that the desired semantics *can* be achieved with a 
well-defined message used with POST. The issue is that if multiple 
protocols (such as Caldav or Atompub) want to use the same semantics, it 
would be nice if they could use the same format, so that general purpose 
servers can take advantage of that. Whether that is some specific kind 
of POST, ADDMEMBER or PUT+Redirect (already dissed by Roy :-) is a 
separate issue here.

Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 18:10:16 +0100

Jamie Lokier wrote:
> ..
>>   and treat it as if it were a POST call.
> 
> 
> Yes...  I have seen numerous CGI implementations which parse a request
> as GET or POST, even if it has another method.
> 
> 

Jamie,

I'm not sure what your point is. Of course if a server mishaves that 
way, a client accessing it may cause problems. But that's the problem of 
the server, and I don't think there's anything we can do about that when 
discussing HTTP based protocols.

What am I missing?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 19:52:09 +0100

Scott Lawrence wrote:
> On Tue, 2005-02-22 at 17:36 +0100, Julian Reschke wrote:
> 
> Either:
> - Implement it very very carefully.

Hm. What exactly does that mean?

When POST is applied to that resource with a content type of 
"application/soap+xml", is it supposed to store the attached entity as a 
new resource (with content type "application/soap+xml", returning a 
Location response header with the URL of the new resource), or should it 
process it according to 
<http://www.w3.org/TR/2003/REC-soap12-part2-20030624>?

> or
> 
> - Change your server organization so that they use different URLs.
> 
> But that's just the point - from the perspective of HTTP, the proposed
> ADDMEMBER is not fundamentally different from POST.

No, it's a subset, thus it can not be "fundamentelly different". That's 
on purpose.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 19:59:41 +0100

Jamie Lokier wrote:
> Jeffrey Mogul wrote:
> 
>>If Jamie is arguing that we need to add new method because
>>implementations are not compliant with the specs for the old
>>methods, that seems to be a recipe for doom.
> 
> 
> I'm arguing _against_ adding a new method.

:-) On the other hand, arguing against new methods by principle may be a 
recipe for doom as well.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 20:19:19 +0100

> That is not allowed in HTTP.

Thanks for the clarification.

>> POST, and to discover these service URIs by parting entity (feed)  

Sure. I didn't mean to imply otherwise.

>       in addition to its other duties, it will return 201 Created;

How is it (the resource) supposed to distinguish?

> webdav chose to ignore that feature of HTTP.

I still have a hard time understanding what to learn from this. It seems 
that we're now discussing "when to define new HTTP methods" which is 
definitively an interesting topic...

1) If I understand you correctly, WebDAV should have said that POST to a 
WebDAV collection resource indeed is the operation I called ADDMEMBER. 
RFC2518 says in 
<http://greenbytes.de/tech/webdav/rfc2518.html#rfc.section.8.5>:

"Since by definition the actual function performed by POST is determined 
by the server and often depends on the particular resource, the behavior 
of POST when applied to collections cannot be meaningfully modified 
because it is largely undefined. Thus the semantics of POST are 
unmodified when applied to a collection."

It would be interesting to find out how the working group came to that 
decision (I don't remember seeing that discussion when I went through 
the archives some time ago; but I may have missed it).

2) RFC2518 introduces WebDAV collections as a new resource type (with a 
distinct creation method, MKCOL). How would a client use the POST method 
(as described above) to create a new collection then? In WebDAV, the 
only difference between a collection and a non-collection resource is 
containment; thus a WebDAV collection may respond to GET with any 
content type the server likes.

3) RFC3253 (DeltaV) defines "VERSION-CONTROL", which is a well-defined 
operation that puts a resource under version control (and may be applied 
to WebDAV collections as well). Of course that operation *could* have 
been defined using POST, but assuming POST to a WebDAV collection is the 
same as ADDMEMBER, how could I possible use POST to do something else 
then? So should RFC3253 have used POST instead? If the answer is "yes", 
how should it have resolved the conflict between the two differing 
semantics that are needed? If the solution is to use different URLs for 
POST, what would have been a good way to discover these URLs?

Wondering,

Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 20:23:56 +0100

Jamie Lokier wrote:
> You're missing that when you are doing CalDAV* requests, you _know_ the
> URL you are accessing is CalDAV compliant already.  (* - example).

How does a client do *any* kind of server feature discovery if *any* 
method (including things like OPTIONS) may cause harm because of broken 
server implementations?

> If you don't know that you have big problems.

Such as?

> So you don't need new methods just to express "intent".
> (Maybe for other reasons, but not this one).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 20:49:50 +0100

> Messages reflect the instruction of the sender.  POST does that.

Agreed. But for POST, this only seems to work where there is a tight 
coupling between client and server; otherwise the client will have no 
clue about the server will indeed do with the request.

> is just another from of implementation-dependency.  It leads to

Well yes; let's take a look at WebDAV collections, part of the WebDAV 
requirements written down in RFC2291 
(<http://greenbytes.de/tech/webdav/rfc2291.html#rfc.section.5.8>).

By defining a way to discover WebDAV collections 
(DAV:resourcetype/DAV:collection in PROPFIND response), a client can 
find out whether a resource is a WebDAV collection or isn't. Also, 
WebDAV defines COPY and MOVE as namespace operations that take 
containment into account.

I'm not sure why this is a problem; and in particular I'd like to know 
what a better solution would have been.

> actions that would have the same result as a generic POST.

Right, those are moronic implementations (because in general, a client 
can safely try the method and then handle HTTP's "method not allowed" 
status code gracefully).

On the other hand, using distinct methods allows to use a 
discovery/failure signalling approach that uses plain HTTP (OPTIONS 
method, "Allow" response header, 405 status).

> resource types for collections, versions, and now calendars.]

Nit: Versions aren't special resource types.

> an atomic append. POST to a messaging gateway means "post this".

Why can't a collection be PUT-able?

> since SOAP has already proven to be a miserable failure.

OK, so that sounds to me like resources should fall into distinct 
categories:

(A1) Collections vs (A2) Non-Collections
(B1) Put-able vs (B2) Non-Put-able

- resources of type A1 would understand POST request as having 
"ADDMEMBER" semantics

- resources of type A2 can (within reason) do anything they like with 
POST (as long as compliant to RFC2616)

- resources of type B1 will allow PUT (as defined by RFC2616) and 
understand POST as "append" operation

Let's consider a simple WebDAV server that supports only resources of 
types A1 and B1. What's the recommendation to do authoring operations 
beyond PUT on these resources? Define new methods (such as 
VERSION-CONTROL) or mint new related URLs to which existing methods can 
be applied?

Best regards, Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 22 Feb 2005 11:52:55 -0800

On Feb 22, 2005, at 5:37 AM, Mark Baker wrote:
> Ah, so you're saying that ADDMEMBER isn't uniform?

Actually, no, I was saying that ADDMEMBER as currently defined
is identical to POST.  Julian said that it wasn't identical because
his client would be able to expect one semantic, namely that a
201 result would cause the webdav collection to contain a new
member with the given media type.  That implies an additional
requirement that the target be a webdav collection, which isn't
uniform at all.

> Can you give an example of a resource

No, which is why it is POST by any other name.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 21:01:55 +0100

> requirement that the target be a webdav collection, which isn't

I'm not sure how you get to that conclusion. ADDMEMBER (as proposed) 
doesn't require the target resouce to be a WebDAV collection.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 21:47:43 +0100

Jamie Lokier wrote:
> ...
> If it's done, the behaviour of the server is unpredictable.  ADDMEMBER
> doesn't fix that.
> ...

You keep saying that :-) So are you saying that defining new methods 
never ever is the right approach, because there are broken servers out 
there?

If that's the case, let's just agree that we disagree.

> (Although it would shift the likelihood towards a simple failure
> rather than an unwanted side effect.  Is it worth creating a new
> method just to change the likelihood of a behaviour when a CalDAV
> client is pointed at the wrong URL?)

ADDMEMBER is a generic method. It doesn't have anything specific to do 
with CalDAV except that it *could* be used with CalDAV.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 21:49:19 +0100

Jamie Lokier wrote:
> Even when you can trust OPTIONS, that's not going to tell you if it's
> a CalDAV resource - only what methods are accepted.

I'm not interested whether it's a CalDAV resource. I'm interested in 
whether it supports ADDMEMBER.

> You do feature discovery - in other words, which URL points to a
> calendar - in other ways.
> 
> 
> Such as trying to treat a non-calendar resource as a calender! :)

And the problem resulting from that would be what exactly? A CalDAV 
calendar (as currently proposed) is just a WebDAV collection with some 
specific features.

Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 22 Feb 2005 13:01:57 -0800

> clue about the server will indeed do with the request.

The client doesn't need to know.  An HTTP client does not know the
reason why a request is being made, just as it does not know the
text inside the anchor of a hypertext link that was clicked on to
generate a GET request.  The user (agent) builds up state based on
a huge number of factors, including what is said in all past
representations received, degrees of trust, instructions that
may have been received via some other channel, configuration, etc.
That is what tells the user what to expect.  HTTP doesn't care.
What HTTP cares about is that the method is unsafe, the response
is not (by default) cacheable, a new resource has been created (201),
and it can be found at a given Location.

> containment into account.

Completely irrelevant -- COPY and MOVE are HTTP operations on any
collection, not just webdav collections, and will succeed or fail
regardless of whether the client previously checked its pacifier.

> what a better solution would have been.

There was no problem -- it was a solution to someone's angst.

> method, "Allow" response header, 405 status).

Which is completely unnecessary in HTTP.

> Nit: Versions aren't special resource types.

Sorry, I meant MKWORKSPACE.

> Why can't a collection be PUT-able?

It could have been, but MKCOL was defined instead (over my objection).

> VERSION-CONTROL) or mint new related URLs to which existing methods 
> can be applied?

Different URLs (see, for example, how the configurable name "index"
is a member of a collection and also the default response to GET
on that collection).  How a PUT or POST or any other method applied
to the collection is handled is just as configurable and may depend
on other aspects of the request (like content-type, authentication,
and more).  PUT is usually redirected to the index URL because
namespace operations are typically done one name at a time rather
than entire collections at a time, however that is dependent on the
implementation: when a client does a PUT of an XML document to a
JCR-backed implementation, it will create a collection (a document
is just one view of the collection).

Each resource has an implementation and that implementation defines
how the method impacts its state aside from what is already defined
by virtue of HTTP's method semantics.  The reason we don't need
ADDMEMBER is because POST already exists, a server must inspect
the media type already, and none of the overlapping uses of POST
are valid new members to add to a collection (soap envelopes or
www/x-form-urlencoded).  It is therefore possible to say that POST
has the add member effect on a webdav collection and be done.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 22:41:10 +0100

Jamie Lokier wrote:
> No I'm not saying that.
> One of the rationales given in this thread for ADDMEMBER is that the
> tighter semantics mean that clients can depend on it literally adding
> a new resource which is GETable, or failing with Method Not Supported.
> 
> I'm saying that particular rationale isn't logically sound.

ADDMEMBER is defined to do just that. If it would be accepted and become 
a standards-track RFC, yes, clients could rely on it just like they can 
rely on other HTTP methods today.

> It doesn't negate any other rationales you may have up your sleeve :)
> 
> 
> Indeed, but are there realistic applications in mind that would use
> ADDMEMBER without caring what kind of container resource it is, but do
> care that it is a container and not a form processor?

Yes. For instance it could be used for Atom-Pub, and another use case 
was just suggested a few days ago internally at SAP (where a specific 
backend system has a concept of containment, but doesn't allow 
user-selected URIs).

> If there are it makes more sense.  If there aren't, it's unneeded.
> 
> I think your exploration of the WebDAV text concerning why POST isn't
> used on containers is a good one, btw.

I appreciate all the feedback I'm getting, and I'm very interested in 
ways that would avoid adding a new method while keeping the intended 
benefits.

Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 23:10:40 +0100

> regardless of whether the client previously checked its pacifier.

Yes, but that doesn't mean it's irrelevant whether a resource is a 
collection or not. It just means that you don't *have* to check before 
you apply namespace operations.

> There was no problem -- it was a solution to someone's angst.

So how *should* a WebDAV collection have been defined?

> Which is completely unnecessary in HTTP.

The 405 status and the Allow response header are unnecessary?

> Sorry, I meant MKWORKSPACE.

The alternative would have been just to use standard resources, but in 
that case RFC3253 would have had to define some new way to indicate that 
something indeed is a workspace.

> It could have been, but MKCOL was defined instead (over my objection).

There is nothing in RFC2518 that prevents collections from being 
PUT-able (however the rational for not defining PUT on collections is a 
bit weird, see 
<http://greenbytes.de/tech/webdav/rfc2518.html#rfc.section.8.7.2>).

> namespace operations are typically done one name at a time rather

...when you say "typically", what implementation are you referring to? 
Is this implemented in Apache/mod_dav?

> www/x-form-urlencoded).  It is therefore possible to say that POST

Interesting. Why wouldn't a SOAP envelope be a valid new collection 
member? And what about other potential content types resources could 
already implement for POST (that just happen to be proprietary)?

Best regards, Julian

---

From: Mark Baker <[email-masked]>
Date: Tue, 22 Feb 2005 23:26:08 -0500

> requirement that the target be a webdav collection, which isn't

I get the DAV-collection-implies-non-uniform thing, and would be
against a revision which attributed that meaning to a 201 response.
But I'm just not seeing how ADDMEMBER==POST, sorry.

I still don't see how POST vs. ADDMEMBER is any different than POST
vs. PUT.  You previously said that PUT has "set the state of this
resource" semantics which is clearly different than POST.  IMO,
ADDMEMBER's semantics are very similar to PUT.  What (constraint?) am
I missing that suggests PUT is fine while ADDMEMBER isn't?

Mark.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 23 Feb 2005 00:24:07 -0800

> I missing that suggests PUT is fine while ADDMEMBER isn't?

You are missing that the target of PUT is the new resource,
whereas the target of POST and ADDMEMBER are both the collection
resource.  As such, ADDMEMBER's semantics has very little in
common with PUT (almost nothing, in fact, since any unsafe
extension method can return 201 and communicate just as much,
whereas PUT is unique in what it communicates by the 200/201).

....Roy

---

From: Mark Baker <[email-masked]>
Date: Fri, 25 Feb 2005 00:35:09 -0500

> whereas PUT is unique in what it communicates by the 200/201).

In case anybody's still following this thread 8-) ... I now find myself
in agreement with Roy, that POST is appropriate and that ADDMEMBER ==
POST.

I was looking at the interaction from the POV of a tweak on PUT
semantics, rather than as the submission of a document to a
state-preserving container (which I should have recognized it as
immediately, given my previous attempts at describing such a
model[1]).

I'm still not convinced that a POST extension (which declared the
expected type of the target resource) wouldn't improve the
self-descriptiveness of the message, but practically, that's pretty
much irrelevant for the purposes of this discussion.

 [1] http://www.markbaker.ca/2001/09/draft-baker-http-resource-state-model-01.txt

Mark.