Subject: RE: A case for GETSRC (or other mechanism to that effect)

From: "Clemm, Geoff" <[email-masked]>
Date: Fri, 1 Mar 2002 13:36:54 -0500

The problem with having multiple ways of doing things
is that it harms interoperability, when one server does one thing
(maintain the DAV:src property) and another server does another
(a GETSRC method or a Translate header).  Servers are always going
to have proprietary extensions, and widely distributed servers
(e.g. IIS) are going to have clients coded against those proprietary
extensions, but the goal of the standard protocol is to produce an
interoperable protocol, which means not providing different alternative
ways to produce the same result.

Cheers,
Geoff  

-----Original Message-----
From: CJ Holmes [mailto:[email-masked]]
Sent: Friday, March 01, 2002 1:04 PM
To: [email-masked]
Subject: RE: A case for GETSRC (or other mechanism to that effect)

>    From: CJ Holmes [mailto:[email-masked]]
>
>    Well, you can't always "just locate the source".  If the source
>    really is in a different location than the "normal" URI then your
>    DAV module probably has no knowledge of it.  Which means now you
>    have to teach JSP to be DAV-aware and answer PROPFIND requests, or
>    teach your DAV module all about what URIs are served by which other
>    modules and how the two URI spaces map to each other.

>
>My primary objection to GETSRC is that it represents a non-extensible
>direction to follow.  For example, one of the key purposes
>of PROPFIND was to provide semantic indexing of web resources, and the
>indexing of the display information should be significantly different
>from the indexing of the source information.  Once you have taught the
>DAV module to understand the difference between display URL spaces and
>source URL spaces, it can produce this kind of indexing.  Similarly,
>any other kind of method that could sensibly be applied to both the
>display and authoring resources can take advantage of the separation
>of display and authoring into separate identifiable URLs.

I agree with all of this!  I have no bones whatsoever to pick about 
the superiority of keeping source and display resources separate, or 
its happy consequences.  For an integrated content management system 
you would definitely want this, and it would be pretty darn cool.

What I disagree with is the _requirement_ to keep the two spaces 
separate, which burdens administrators with a lot of extra 
configuration, isn't understood by most content-generation software 
today (if any), and often confuses the end-users who can barely use 
Dreamweaver/Golive/FrontPage much less understand why their "browser" 
URL is different from their "Dreamweaver" URL.  For most 
installations, keeping the URIs the same is a desireable thing and 
does not cause anyone to lose any functionality that they care about. 
Most people don't care about indexing their source, and we have 
specialized crawlers already for indexing display content.

I'm just saying that the simple things should be easy.  It's a basic 
rule of design.

I'm not stuck on the GETSRC idea. I'll implement Translate almost as 
happily if that's the direction the group goes.  If the group doesn't 
come up with anything at all, then Translate is likely to become the 
de-facto standard, and you'll end up putting it into RFC-2518-bis2 
anyway.

cjh

--

---

From: CJ Holmes <[email-masked]>
Date: Fri, 1 Mar 2002 13:37:26 -0800

>ways to produce the same result.

Ok, let's try a different tack.

Let's keep the model that says source and display URIs are different, 
and keep/fix DAV:source, and leave all that the way it is.

Let's just require that DAV clients add a special field to the 
request so that servers know they are talking to a client that is 
likely to send MKCOL, PROPFIND, and other DAV-specific methods.

That way the URI model stays the same, and servers can tell when a 
GET request was generated by a DAV client.  Servers that only want to 
provide DAV access to source URIs may do so without creating new URI 
spaces.

Call it DAV-Enabled.  The value must be present but is ignored.  (If 
someone got ambitious it could be used by the client to indicate its 
capabilities.  But one problem at a time, OK?)

How's that for a solution?

cjh

--

---

From: CJ Holmes <[email-masked]>
Date: Fri, 1 Mar 2002 17:00:44 -0800

>It doesn't change the issue that you'd be using the same URI for different
>resources. That's a very basic problem, and no amount of syntactic sugar
>makes it go away.

Yeah, but it's OUR problem, and not yours.  If our DAV implementation 
is unsatisfactory to our customers then we have to change it.  But 
this would let us (and other implementors) give our customers what 
they want, which is DAV access to their source files with virtually 
no configuration necessary.

All we need from the protocol group is a sure way to know that a GET 
command originated from a DAV client.  Is that so much to ask?

cjh

--

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 2 Mar 2002 02:07:41 +0100

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of CJ Holmes
> Sent: Saturday, March 02, 2002 2:01 AM
> To: [email-masked]
> Subject: RE: DAV-Enabled field (was RE: A case for GETSRC)
>
>

To be honest: yes.

GET is GET. It doesn't matter what client it comes from. You shouldn't try
to change that.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 1 Mar 2002 17:23:06 -0800

> no configuration necessary.

No configuration is necessary with dav:source other than what was already
done to create the dynamic resource.

> command originated from a DAV client.  Is that so much to ask?

No, the problem is that your question starts off with an awful design
decision and expects the WG to validate that through another unnecessary
and completely lame hack.

A DAV client doesn't do a GET in order to start editing a resource.
It MUST, for many reasons, perform a PROPFIND on the resource.  When it
does that on a resource that is not capable of being edited, such as
a dynamic-content resource, it will receive the information necessary
to tell the it and the user that to go edit these other URI (as provided
by dav:source) in order to modify the content.  When the user picks one
of those to edit, the client software will then do a PROPFIND on that
other URI, and will continue this recursively until it gets to a resource
that is static and authorable.

There simply is no other way to implement authoring using the Web interface
without assuming that everything is a static file, and not assuming that
was the whole point of defining a WebDAV protocol in the first place.
If editing files was good enough, we'd just us FTP and rsync.

....Roy

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 2 Mar 2002 01:48:11 +0100

> Sent: Friday, March 01, 2002 10:37 PM
> To: [email-masked]
> Subject: DAV-Enabled field (was RE: A case for GETSRC)
>
>

---

From: Matthieu Chevrier <[email-masked]>
Date: Fri, 01 Mar 2002 17:58:33 -0800

> GET is GET. It doesn't matter what client it comes from. You shouldn't try

apart if your company's name is Microsoft, right? Because that's precisely
what they did with 'Translate'.

It seems like we are pretty much done with this thread. Just want to add one
little thing : in the SOAP specifications, POST requests MUST have a
SoapAction field. It is not specified what this field should contain, just
that it has to be there. And noone complained that it's breaking the holy
POST.

Soap is of course a completely different protocol compared to DAV, but it's
still interesting I believe to think about why the spec writers made this
choice (or shall I say a lame hack Roy?)

Regards,

   Matthieu.

---

From: CJ Holmes <[email-masked]>
Date: Fri, 1 Mar 2002 18:35:41 -0800

>done to create the dynamic resource.

I've tried to explain the real-world "hassle for no gain for most 
people" situation on this about five times now.  I'm not going to 
repeat it because apparently nobody on this list thinks such issues 
merit much weight.  But I guess what administrators have to do to 
configure their servers or how much we confuse hapless users is 
inconsequential in the face of the prospect of adding a mote of dust 
to the pure model of DAV.

>and completely lame hack.

It starts with trying to find a way to accommodate users who don't 
give a darn about using DAV on display resources, and would like to 
use DAV to edit their sources without screwing with the URL.  If it 
was just a minority of users, it wouldn't be an issue.  But it is a 
majority of users and installations, so it is an issue.

Translate exists because it solves this problem by identifying GETs 
from DAV clients.  If we have to use Translate to give our customers 
what they want, then so be it.  I was just hoping we could get 
something defined by a group of thoughtful people instead of 
something undocumented.

cjh

--

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 2 Mar 2002 13:22:15 +0100

CJ,

I understand your frustration.

However,

so far you have failed to answer the following question:

Are the source and it's output (from an HTTP point of view!) the same
resource? If they aren't, they MUST have different URIs. This is a
fundamental property of URIs and resources that you just can't throw away
because it *seems* to be convenient for you.

If you say they are, you'll have to convince us that the source code (of for
example an ASP page) and it's output are different representations of the
same thing.

For instance, let's take the example of an ASP page that creates the list of
current bug reports for a certain product "test". It's output is identified
by

	http://server/foo/bugs.asp?product=test

It can also be used for other products, so the *different* URI

	http://server/foo/bugs.asp?product=test2

represents a completely different resource which only happens to be produced
by the same script. Finally the script may have a default view (giving a
form), which is identified by:

	http://server/foo/bugs.asp

All these resources are different (and thus have different URIs).

Let's say that http://server/foo/bugs.asp happens to be WebDAV-compliant
resource. When you pass the URL to somebody else, s/he will ultimatively do
a GET against it. What GET does is defined in RFC2616, not in WebDAV. As

	http://server/foo/bugs.asp

identifies the *output* of the script, it cannot be used to identify the
script itself (otherwise, please show us!).

What you *can* do is add a DAV:source property that contains the URI of the
actual script (which will actually send the source text upon GET). Some
examples are:

	http://server/foo/bugs.asp?action=send-the-source

	http://server/foo/?source-of=bugs.asp

	http://server/foo,source/bugs.asp

	http://server/sources/foo/bugs.asp

	http://server:81/foo/bugs.asp

and so on. Note that only some of these formats require a second URL
namespace / virtual host / port.

Finally, please see what the HTTP spec (RFC2616) says in it's first
paragraph about GET:

"9.3 GET

The GET method means retrieve whatever information (in the form of an
entity) is identified by the Request-URI.
If the Request-URI refers to a data-producing process, it is the produced
data which shall be returned as the entity
in the response and not the source text of the process, unless that text
happens to be the output of the process."

Julian

> Sent: Saturday, March 02, 2002 3:36 AM
> To: [email-masked]
> Subject: Re: DAV-Enabled field (was RE: A case for GETSRC)
>
>

---

From: "Eric Sedlar" <[email-masked]>
Date: Mon, 4 Mar 2002 11:48:07 -0800

It's not clear to me that the generated output from GET on executables
actually is a different resource.  First of all, it doesn't make sense to
think of it as having properties separate from that of the script.  I doubt
if anyone actually doing the URL space mapping proposed would keep around a
set of dead properties for the generated output, but the case becomes
clearer when looking at the live properties.

If I have a directory full of .ASPs, or .JSP files, and I do a PROPFIND on
that directory, will the server actually execute all of the scripts in the
directory in order, dumping their output to a temp file, to ensure that
dav:getcontentlength is correct?  I bet nobody actually does that (it would
be "considered harmful" like PROPFIND depth infinity), and if you did it
would be stupid.  I haven't checked this, but I bet that most servers return
the length of the underlying script as the length of the generated output on
a PROPFIND depth 1.  The same difficulties apply to <get-etag> and many
other live properties.  The reason this doesn't matter in practice is that
returning the wrong etag or wrong content length for generated output is
irrelevant, since in the general case it cannot be cached, and it works
functionally just as if the resource was changed before the subsequent GET,
so it's ok if the value of <getcontentlength> is different in a PROPFIND on
its parent and the actual content length returned when you do a GET.

I think that the better model is if you think of the GET method more like
the "OPEN" action on Windows or Macintosh, which means to execute this file
if executable, or else display the contents of the document.  GET is really
like a simple case of the POST method (which was originally how it was
conceived).  To illustrate this point, let's say that a particular server
said that POST on a non-executable resource (like a static file) would dump
its contents (like a GET).  Now we can clearly see GET on a script as a
simple case of POST.

Julian's argument below gets at the heart of the problem--is each set of
possible output generated by a script a different resource?  First of all,
there are many different things that can affect the output of a resource,
and most of them are not in the URL (which is the only thing that can be
used to differentiate two resources).  While the URL arguments in the query
string are one set of arguments to the script, clearly information in the
header may also be used (even in a GET method).  For example, the
authentication headers may be used to choose whose shopping cart will be
displayed, or in POST, the form values appear in headers.  If, as Julian
suggests: GET /foo/my.jsp?arg1=val1  is a different resource than GET
/foo/my.jsp?arg1=val2, then POST /foo/my.jsp with a header Arg1: val1 should
be different than POST /foo/my.jsp with header Arg1: val2.  This obviously
makes no sense.

So, here are my arguments for why each possible output generated by a script
should not be a separate resource:

* they may not have separate URLs, since not all of the arguments that cause
different output may be in the URL string--they may be in headers, or may
depend on other server state

* it is impractical to manage dead properties for each possible output

* it is computationally difficult to return dead properties (such as
dav:getcontentlength) for each possible output

Therefore, I conclude that generated output is not a resource, and
especially not a dav-enabled resource.  I would recommend that server
implementers NOT return script output as dav enabled.

Now, on to the practical argument, which will demonstrate that dav:source is
a lot less usable than GETSRC/Translate:

let's say that I have a directory full of scripts and static files.  I want
to manage them from a authoring point of view.  If I want to implement
dav:source, I need to create some virtual URL space that corresponds to the
physical URL space where the actual script source is stored.  I can make the
virtual URL space return either the generated output, or the source, at
least from an implementation point of view.  For example:

Contents of /mydir:

   foo.jsp
   bar.html
   bar.gif

In case 1, let's say I make /scripts/src/<real-path> a virtual URL space
that returns the source of any executables (for DAV clients), so where GET
/mydir/foo.jsp will return the generated output, /scripts/src/mydir/foo.jsp
will return the source.  dav:source on /mydir/foo.jsp will point to
/scripts/src/mydir/foo.jsp.  The DAV client doesn't actually want to mess
with /mydir/foo.jsp since it is generated, has bogus live property values,
doesn't respond to things like PUT, and is generally not very helpful.  Now
I'm forced into a separate tree, which is certainly annoying from a
useability point of view, and extremely annoying when doing things like
copying with Web Folders, since when I drag & drop my app from the local
filesystem to the server, my .jsp files move to another place, and when I
copy the application back down to my local PC, the .jsps were moved (even if
I was smart enough to realize where they magically went to).  So, this is a
bad solution.

In case 2, let's say I make /script/output/<real-path> a virtual URL space
that returns the generated output for the executables.  Now the DAV client
is happy, since the server space looks like the space on the client.
Unfortunately, all of the <href> tags in any HTML output are screwed up vs.
the local copy (e.g if bar.html wanted a link to foo.jsp, or vice versa),
since neither a relative nor an absolute URL will work, and the URLs needed
are server implementation-dependent.

The reality of the world is that people want to be able to move their
content creation environment back and forth between local filesystems and
DAV servers.  Any usage of dav:source would break that abstraction, since
local filesystems aren't capable of implementing virtual URL spaces.  I
think these are the practical reasons why nobody has implemented dav:source
in the past 3 years.

So, it is clear that:

* generated output (from stuff like .jsps, .asps, cgi) does not make sense
as a DAV-enabled resource, since the only methods it is likely to respond to
intelligently is GET.
* separating the URL space of generated output and source causes usability
problems, making life awkward either for the DAV client, or for the HTML
linking

I think the argument for a GETSRC method becomes clear if you mentally
rename GET to LIMITED-VERSION-OF-POST.

Reponses to a couple of other points in more recent emails:

  Alan Babich says:

"Is the source the same entity as its output?" Julian and I agree that it is
not. RFC2616 section 9.3 says it is not.

  Response:  the issue is not whether or not the source and the output are
the same "entity", but the same DAV resource.  Entity and resource are
different things.  There are three general categories of things that can
cause the output of a generated script to vary:  URL information, e.g. the
query string; header information; or server-state (like in a database).
Clearly the URL isn't sufficient to differentiate between all the kinds of
server-generated output, so the various entities that can be generated by a
script cannot be mapped as separate resources, since they won't have
separate URLs.

  Roy Fielding says:

Any server that allows clients to vary GET responses based on the Translate
header field MUST include "Vary: Translate" in every response to the GET
method in order to remain compliant with HTTP/1.1.

  Response:  Actually, RFC2616 says: "An HTTP/1.1 server SHOULD include a
Vary header field with any cacheable response that is subject to
server-driven negotiation."  So, clearly, Microsoft is in compliance with
the HTTP specification since this is not a MUST requirement.  Clearly it is
acceptable to use header values to vary the content of generated state,
whether it be the Translate header or the Authentication header, and whether
or not the Vary header is supplied is a completely orthogonal issue.

Here's another way to look at the categorization of response entities for
executables:

1) the source
2) generated output that does not vary based on input arguments from client
3) generated output varying with input arguments from client encoded in
query string
4) generated output varying with input arguments from client encoded in
request headers
5) generated output varying with server-side state (like information in a
database)

I don't see why case #2 and case #3 (in Julian's example) should be awarded
its own status as a "real dav resource" when cases #4 and #5 are not.

I'd like to see someone propose a scheme for implementing dav:source that
solves the usability problem when mapping to the local filesystem, if they
believe in dav:source.

--Eric

> [mailto:[email-masked]]On Behalf Of Julian Reschke
> Sent: Saturday, March 02, 2002 4:22 AM
> To: CJ Holmes; [email-masked]

---

From: "Julian Reschke" <[email-masked]>
Date: Mon, 4 Mar 2002 23:27:34 +0100

Eric,

thanks for taking the time to write this long reply. I am putting some
comments inline...

> From: Eric Sedlar [mailto:[email-masked]]
> Sent: Monday, March 04, 2002 8:48 PM
> To: Julian Reschke; CJ Holmes; [email-masked]

It does. For instance, DAV:getcontenttype will almost certainly have a
different value, so will DAV:getcontentlength.

> clearer when looking at the live properties.

Well, I wouldn't expect a server to store dead properties for a "output"
resource, but this is really an implementation detail.

> dav:getcontentlength is correct?  I bet nobody actually does that

A conforming server will either report these properties as empty or as
non-existant (unless it's actually able to compute these values cheaply).

> (it would

IMHO this would be a bug.

> irrelevant, since in the general case it cannot be cached, and it works

Why do you say that generated output can not be cached? A big part of the
HTTP spec treats this very problem.

> its contents (like a GET).  Now we can clearly see GET on a script as a

GET doesn't dump "the content" of a resource. GET returns an entity which is
one of many possible representations of the resource, which may depend on
non-URL based information in the headers (in which case these headers should
be listed in the "Vary" header).

> header may also be used (even in a GET method).  For example, the

Yes.

> /foo/my.jsp?arg1=val2, then POST /foo/my.jsp with a header Arg1:

GET ... is not a resource, it's one representation of a resource.

URLs (as they are URIs as defined in RFC2396) identify resources. If they
differ, this is a good indication that the resource they identify differs as
well.

> val1 should

I don't understand the analogy to POST. POST is very different from GET in
it's semantics (it's not just a distinction in marshalling).

> should not be a separate resource:

I didn't say that. What I'm saying is that the set of outputs (that can be
generated by a GET on a URL) is a different resource than the source
resource.

> depend on other server state

That's fine. So all of them are the same abstract resource.

> * it is impractical to manage dead properties for each possible output

That's no problem. If you don't want to, don't.

> dav:getcontentlength) for each possible output

Again fine. Don't return them.

> Therefore, I conclude that generated output is not a resource, and

You can't conclude that, because the generated output *has* a URL, so it
*must* be a representation of an abstract resource.

> implementers NOT return script output as dav enabled.

That's fine. You don't have to.

> physical URL space where the actual script source is stored.  I

Actually, I'd say that you should create mappings for the outputs, not for
the sources, but this is just a matter of taste, I guess.

> can make the

That's something the client should do for you.

> I was smart enough to realize where they magically went to).  So,

Why don't you just edit *all* files in the source directory and leave the
output resources alone?

> local filesystems aren't capable of implementing virtual URL spaces.  I

Again, that's not true. You can author the *source* URL space just like
always. All you need the DAV:source property for is to discover where those
files sit.

> think these are the practical reasons why nobody has implemented

The DAV:source property is underspecified in the spec, that's the reason
it's not implemented. That doesn't make the concept itself bad.

> as a DAV-enabled resource, since the only methods it is likely to

I don't have any problem with this. If output resources aren't WEBDAV
enabled, you just lose the ability to use DAV:source to discover their
sources.

> problems, making life awkward either for the DAV client, or for the HTML

I don't agree.

> rename GET to LIMITED-VERSION-OF-POST.

GET and POST are fundamentelly different things.

> the same "entity", but the same DAV resource.  Entity and resource are

...the same HTTP resource...

> different things.  There are three general categories of things that can

Yes.

> script cannot be mapped as separate resources, since they won't have

If they don't have separate URLs, they aren't different abstract resources.
In the case of the shopping cart mentioned earlier, the resource identified
by the URL is "the shopping cart of the currently logged in user". It's no
problem if it's different for different users.

> the HTTP specification since this is not a MUST requirement.

SHOULD (as defined in RFC2119):

"This word, or the adjective "RECOMMENDED", mean that there
   may exist valid reasons in particular circumstances to ignore a
   particular item, but the full implications must be understood and
   carefully weighed before choosing a different course."

To still claim conformance, one would have to specify these valid reasons.
Can you think of any?

> Clearly it is

Yes, but by doing to you state that all entities that you return are
representations of the same resource. Are they?

> or not the Vary header is supplied is a completely orthogonal issue.

Yes. But you SHOULD indicate it in the "Vary" header.

> its own status as a "real dav resource" when cases #4 and #5 are not.

Because resources are *identified* by their URLs. If the URL is the same,
it's *by definition* the same resource.

> solves the usability problem when mapping to the local filesystem, if they

First of all, I strongly object to this line of argument. Even *if* you can
demonstrate that DAV:source as it is doesn't work, this doesn't mean that
this makes the Translate header an acceptable solution.

This having said, I don't see any problem having two separate URL spaces,
and to only author the source space. The output space *may* be DAV enabled,
and if it is, it should use DAV:source or something similar we'll come up
with to indicate where the source is.

Julian

---

From: CJ Holmes <[email-masked]>
Date: Mon, 4 Mar 2002 14:44:44 -0800

>be listed in the "Vary" header).

You mean like a "DAV-Enabled" header field?  In which case 
"DAV-Enabled" would appear in the Vary field of the response?

A "resource" is a rather malleable concept, and as you've stated here 
there may be many representations of that resource.  I don't see that 
it would be so horrible to allow the idea that one of the 
representations of a resource could be its raw source.  What 
representation you receive is a matter of server policy.  And some 
way of identifying that the server is talking to a DAV client would 
help with managing that policy.

cjh

--

---

From: "Julian Reschke" <[email-masked]>
Date: Tue, 5 Mar 2002 00:15:01 +0100

> Sent: Monday, March 04, 2002 11:45 PM
> To: DAV

CJ,

first of all: What is the *meaning* of "DAV-Enabled"? As you are proposing
this header, you should define what it means. Is an HTML editor that uses
PROPFIND/GET/PUT/LOCK to author HTML files "DAV-Enabled"? If yes, what do
you do if it has a test function that needs to access the output resource
rather than the source resource?

I think "Translate" is much clearer than "DAV-Enabled" because it at least
it says what you want.

And yes, if you *do* take the position that the source and it's output are
different representations of the same resource, then the "Translate" header
*does* make sense (and you should list in in the "Vary" header when
responsing).

As I've said numerous times: make up your mind whether the source and it's
output are the same resource or not. If they are, you can use "Translate".
If they aren't, they will have different URLs.

HTTP takes the position that they are different resources (see introduction
to "GET"), and I don't see how a different working group could change this
view.

> representations of a resource could be its raw source.  What

The most horrible thing being that the source resource doesn't have it's own
URL and thus cannot be properly referred to using just a URL. Could you
please explain why you think this particular problem *isn't* relevant?

> help with managing that policy.

No, "DAV-Enabled" vs. "Translate" is the completely wrong approach.
Following your proposal, a "DAV enabled" client never would want to GET the
output resource. This is obviously a wrong assumption.

---

From: CJ Holmes <[email-masked]>
Date: Mon, 4 Mar 2002 15:36:50 -0800

>I think "Translate" is much clearer than "DAV-Enabled" because it at least

Then let's spec Translate and make it part of DAV.  Interoperable 
implementations exist.

>If they aren't, they will have different URLs.

They are if the administrator says they are.  Again, that's policy. 
I don't set policy, I just make the tools my users want.

    The GET method means retrieve whatever information (in the form of an
    entity) is identified by the Request-URI. If the Request-URI refers
    to a data-producing process, it is the produced data which shall be
    returned as the entity in the response and not the source text of the
    process, unless that text happens to be the output of the process.

If DAV is responsible for processing GET, and it makes the policy 
decision that the correct output is the the same as the source text 
(and the security sub-system gives the go-ahead) then it should be 
able to return the source text if that's the policy.

>  > I don't see that

Because in practice most people don't want to view the source in a 
browser.  They use DAV for working with their source, and they only 
use it for working with their source.

>output resource.

Sure you could.  If the administrator decides that's a good thing, 
and wants to separate the URIs for source and display, then you could 
certainly GET the output resource with your DAV client.  And if 
DAV:source ever gets fixed and implemented then you could even have 
automatic linking between display and source  URIs.  Its all about 
how the administrator wants to set up the policy.

cjh

--

---

From: "Eric Sedlar" <[email-masked]>
Date: Mon, 4 Mar 2002 17:37:45 -0800

My comments inlined, with longer response at the end...

----- Original Message -----
From: "Julian Reschke" <[email-masked]>
To: "Eric Sedlar" <[email-masked]>; <[email-masked]>
Sent: Monday, March 04, 2002 2:27 PM
Subject: RE: DAV-Enabled field (was RE: A case for GETSRC)

> non-existant (unless it's actually able to compute these values cheaply).

The point is that you will not be able to get <dav:getcontentlength> and
<dav:getetag> that have the correct values, which the spec pretty clearly
states should be returned if it is a DAV-enabled resource.

> IMHO this would be a bug.

Whatever.  They return something other than the correct value.

> HTTP spec treats this very problem.

I say that in the GENERAL case it cannot be cached.  For example, a script
that returns the value of a row in a database table (say the current
inventory total on Amazon.com of Harry Potter books) cannot be cached, since
the return value is not simply a function of the input arguments in the
request method.  Those generated output entities that are strictly functions
of the input arguments can be cached, by using the techniques in HTTP/1.1.

> it's semantics (it's not just a distinction in marshalling).

Why is POST different in semantics from GET?  Since we are confining
ourselves to discussing generated output, I see them as just having
different ways of marshalling arguments.

> the sources, but this is just a matter of taste, I guess.

That option is covered in case #2.

> That's something the client should do for you.

It can't do this easily, because it doesn't know the general rule of mapping
the generated output URL to the source, it just knows the answer on a URL by
URL basis.

> Why don't you just edit *all* files in the source directory and leave the
See case #2, below, again.

> > are server implementation-dependent.

Julian, you have suggested that case #2 is the preferred solution (creating
the virtual URL space for the generated resources), but you haven't
addressed the linking problem between the static & generated content.  You
have not given a complete suggestion.

> always. All you need the DAV:source property for is to discover where
those

Are you suggesting that when dragging & dropping some Web Folders down from
a server, the client should examine the dav:source properties, and copy down
the entities returned by GET on those URLs those represent, rather than the
entities returned by the original URLs?  While this could be done, it
certainly breaks backwards compatability with almost all existing DAV
clients.  Plus, it could result in multiple copies of the source after the
download.

> it's not implemented. That doesn't make the concept itself bad.

What part of dav:source is underspecified?  I think it is pretty clear from
the spec.  It just doesn't solve the problem in a usable way.

> > problems, making life awkward either for the DAV client, or for the HTML

That's because you aren't considering backwards compatability.

> GET and POST are fundamentelly different things.

Why are they so fundamentally different?

> To still claim conformance, one would have to specify these valid reasons.

I'm sure MSFT could come up with some valid backwards-compatability reasons
or something.  The point is that an HTTP/1.1 client cannot expect anything
with SHOULD to always be the case

> Yes. But you SHOULD indicate it in the "Vary" header.

Great.  We can agree that the Vary header is a side issue and not relevant
for the purposes of this discussion.

> it's *by definition* the same resource.

Now you are arguing circularly.  You say that because the source and
generated output are different resources, they should have different URLs.
Now you are saying that because they have different URLs, they are different
resources.

> with to indicate where the source is.

My line of argument is that having a separate URL space for generated output
from the underlying executable doesn't work well because:

  * the generated "resources" won't have the correct values for many live
properties (other than dav:source!!!), and will probably (your words) not
support dead properties
  * the generated "resources" will only respond to the GET method, not other
WebDAV methods
  * creating a virtual URL space for generated outputs (as you suggest)
makes the <href> linking unworkable when doing the 1-1 sitecopy semantics
between local filesystems & servers that is common practice today, thus
making the separate URL space idea unusable if backwards compatability is
maintained.
  * just because the response body is different based on the Translate:
header isn't really a different argument than having a different response
body by varying the authentication header.  By this argument, each possible
response body that can be altered by varying a header is a different
resource, which clearly does not make sense.

dav:source is an implementation suggestion that requires separate URL
spaces.  GETSRC/Translate are various ways of handling the generated output
as the same URL.  My argument is against the separate URL space, not the
dav:source header as specified by RFC2518, which most people seem to agree
is broken (you believe it is underspecified, and I believe it is the wrong
architecture).

Julian, fundamentally your response is self-contradictory.  At one point,
you say:

1) "the set of outputs (that can be generated by a GET on a URL) is a
different resource than the source resource".

but later you say that:

2) "Because resources are *identified* by their URLs. If the URL is the
same, it's *by definition* the same resource".

If a set of generated outputs is generated differently by varying parameters
in the query string, statement #1 would imply that the entire set of valid
outputs is one resource.  Statement #2 implies that this set is a bunch of
different resources, since their URL is different.  You are using a
different number of resources per executable at different points to reply to
different points in my argument.  I'm saying there is one resource, used for
the script & generated output.  At first you are proposing 2, and later you
propose n, where n is the number of possible combinations of legal input
querystring values.  Which is it?

Julian's other argument is that you might want to have existing clients be
able to edit the source just by pointing to some URL.  Well, without
understanding dav:source they won't know what the URL is to use for the next
GET method.  So, clearly clients will need to be upgraded either to handle
Translate:/GETSRC or to handle the dav:source header.  Existing clients
won't cut it, so I don't by this reasoning, anyway.

--Eric

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 6 Mar 2002 13:46:45 +0100

Eric,

comments inline....

> states should be returned if it is a DAV-enabled resource.

Sorry? If your output is dynamically generated, you won't be able to supply
a content-length header upon GET (or HEAD) either (unless you compute the
result first before sending the headers). So it's perfectly acceptable not
to return a DAV:getcontentlength for resources like these.

> Whatever.  They return something other than the correct value.

Eric, not returning a value if you don't have it (and won't have it upon
GET) is correct.

> of the input arguments can be cached, by using the techniques in HTTP/1.1.

Others can be cached as well. For instance, a GET on http://www.cnn.com
(clearly something that falls under the category you just mentioned) tells
me that I can cache the resullt for 60 seconds:

> wget -S www.cnn.com
--13:07:43--  http://www.cnn.com/
           => `index.html'
Connecting to www.cnn.com:80... connected!
HTTP request sent, awaiting response... 200 OK
2 Server: Netscape-Enterprise/4.1
3 Date: Wed, 06 Mar 2002 12:07:42 GMT
4 Last-modified: Wed, 06 Mar 2002 12:07:43 GMT
5 Expires: Wed, 06 Mar 2002 12:08:43 GMT
6 Cache-control: private,max-age=60
7 Content-type: text/html
8 Connection: close

> different ways of marshalling arguments.

GET gets a representation of the resource (without affecting it). POST
(possibly) modifies the resource.

> the generated output URL to the source, it just knows the answer

Maybe this indicates that the protocol should allow a server to say that all
source resources for the output resources in a collection sit in some
specific other collection. I admit that if this is the case and the server
indicates this, thing for clients would be much easier.

> have not given a complete suggestion.

I'm not sure that I understand the issue. If you're talking about broken
links -- can't you use relative URI references instead?

> clients.  Plus, it could result in multiple copies of the source after the

I think this is what the spec says. I would expect that the result of
COPYing is the same as GETting and PUTting to a new location. I certainly
wouldn't expect to have the source code in one case and it's output in the
other case.

> the spec.  It just doesn't solve the problem in a usable way.

See
<http://lists.w3.org/Archives/Public/w3c-dist-auth/2001OctDec/0119.html>.

> That's because you aren't considering backwards compatability.

With...?

> Why are they so fundamentally different?

Because of their definition in the HTTP protocol?

> > >   Response:  Actually, RFC2616 says: "An HTTP/1.1 server
> SHOULD include
> a

Which means that implementing Translate without setting the Translate header
*will* break caching in proxies.

> > > 5) generated output varying with server-side state (like
> information in
> a
> > > database)
> > >

Yes.

> Now you are saying that because they have different URLs, they

Did I say that?

- Resources (RFC 2616):

"A network data object or service that can be identified by a URI, as
defined in section 3.2. Resources may be
available in multiple representations (e.g. multiple languages, data
formats, size, and resolutions) or vary in
other ways."

- URI (RFC 2616)

"As far as HTTP is concerned, Uniform Resource Identifiers are simply
formatted strings which identify--via name, location,
or any other characteristic--a resource."

So if two things have the same URI, they *must* be the same resource (by RFC
terminology). If the URIs differ, they *may* be different resources.

> properties (other than dav:source!!!), and will probably (your words) not

Not having values is something different from not having "correct" ones. I
simply don't see a problem here.

>   * the generated "resources" will only respond to the GET

Define "respond". They may not *implement* or *support* some other methods,
right.

> making the separate URL space idea unusable if backwards compatability is

I think it is *also* common practice to replicate the source URL spaces (and
not to touch the output URL spaces at all). In which case there's no
problem.

> resource, which clearly does not make sense.

I didn't say that. I asked the question: "do you consider the (ASP) source
code and it's output the same resource or not"? If they are, they can share
the same URL. If they aren't, they MUST not. HTTP (RFC2616) says they
aren't.

Varying the entity returned based on the time of day, the logged in user,
it's accept-language header and similar things is something completely
different from just returning the source code.

Yes.

> outputs is one resource.  Statement #2 implies that this set is a bunch of

No, it doesn't. If you vary parameters in the query string, you are using
*different* URLs. Statement 1 talks about the set of outputs generated by
GETs on a (*specific*) URL.

> querystring values.  Which is it?

No, I didn't.

What I'm saying is that:

- A URL identifies a resource. GETs on that URL return different
representations of the same resource.

- If the URLs differ (by varying the query string), they *may* identify
different resources (but they don't have to). For instance, I'd consider

	http://server/foo?test=1 and http://server/foo?test=2 different resources,
but

	http://server/foo?a=1&b=2 and http://server/foo?b=2&a=1 the same resource
(but to do this, you'll have to have special knowledge about the treatment
of query parameters)

> won't cut it, so I don't by this reasoning, anyway.

Yes, clients and servers will need to be upgraded. Did I say otherwise?

Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 6 Mar 2002 13:01:18 +0100

CJ,

> Sent: Tuesday, March 05, 2002 12:37 AM
> To: DAV
> Subject: Translate (was RE: DAV-Enabled field (was RE: A case for
> GETSRC))
>
>

Just because there's a proposal that I think is even worse doesn't make the
Translate header a good solution.

If you want to go ahead, you'll have to:

1) Clearly define what Translate means for every HTTP method defined in HTTP
1.1, WebDAV (and preferably ACL and deltaV).

2) Note that ths definition must be compatible with what the base specs do
today.

> I don't set policy, I just make the tools my users want.

I personally do not agree that this is a question of policy. It should be
*consistent*.

> able to return the source text if that's the policy.

DAV is not responsible for "processing" anything. DAV defines semantics of
new HTTP methods, and in particular, it doesn't define GET.

> use it for working with their source.

It's not a question of seeing the source in a browser, it's about being able
to *refer* to it. What do you do when somebody sends an email asking for a
link to the source resource? Do you send the URL and add wording to explain
what program will be needed to open it?

> how the administrator wants to set up the policy.

OK, so you agree that this is a problem with your proposal, while it isn't
with separate URLs?

---

From: CJ Holmes <[email-masked]>
Date: Wed, 6 Mar 2002 11:08:33 -0800

>OK, so you agree that this is a problem with your proposal, while it isn't

No, I don't agree that this is a problem.  In most cases it is the 
_desired_ effect because users generally only use DAV for messing 
with their source, and they don't care about using DAV with the 
output.  But they _do_ want to use the same URLs for both editing 
source and viewing output.  If the administrator wants something 
different he should be able to configure his server accordingly.

Separating "source" and "display" into different resources with 
different URIs is a very nice idea, but not terribly useful to most 
people.  Maybe once the DAV:source property is fixed and DASL gets 
off the ground, this will become a more useful configuration.

I believe the DAV spec needs to allow both scenarios, and the 
simplest way to do that is to either:

	- have a definitive way to know when a client expects the source or
	- know when a GET method originates from a DAV client

cjh

--

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 6 Mar 2002 20:19:58 +0100

> Sent: Wednesday, March 06, 2002 8:09 PM
> To: DAV
> Subject: RE: Translate (was RE: DAV-Enabled field (was RE: A case for
> GETSRC))
> ...

Again: a DAV-client may want to do a GET on the output (for testing), while
a non-DAV client may want to see the source code. So *this* distinction
doesn't make any sense at all.

---

From: CJ Holmes <[email-masked]>
Date: Wed, 6 Mar 2002 11:24:43 -0800

>terminology). If the URIs differ, they *may* be different resources.

I think we can all agree on that.

What we're disagreeing on is that the source can be the same URI as 
the output.  They can be two views of the same resource.  That is how 
most people think of it, and it would be practical to accomodate 
them.  But the current spec is very inconvenient in that it 
effectively prevents it (without explicitly forbidding it).

>simply don't see a problem here.

I think Eric mentioned that some of these values are required, so you 
can't not return them.

>not to touch the output URL spaces at all). In which case there's no

It is common because it is the only way to deploy DAV under the 
current spec.  It causes no end of confusion amongst people who are 
deploying/using DAV for the first time.  It is counter-intuitive, and 
is not really useful in most situations.

>Yes, clients and servers will need to be upgraded. Did I say otherwise?

Someone claimed that Translate/GETSRC would require implementation 
changes.  Eric was just pointing out that implementations have to 
change anyway because of these very issues, therefore the argument is 
irrelevant.

cjh

--

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 6 Mar 2002 20:39:08 +0100

> Sent: Wednesday, March 06, 2002 8:25 PM
> To: DAV

I have to not return them if they don't exist. I can't return the headers
upon HEAD or GET either. If you understand RFC2518 as saying that
DAV:getcontentlength MUST be returned any may not be an empty string, this
indicates that we need to fix RFC2518.

> is not really useful in most situations.

These are completely subjective arguments. Some people like the one
approach, some like the other.

> change anyway because of these very issues, therefore the argument is

Agreed, unless somebody claims that "adopting" the Translate header can be
done without breaking existing code.

---

From: CJ Holmes <[email-masked]>
Date: Wed, 6 Mar 2002 12:15:22 -0800

>approach, some like the other.

The argument may be subjective to you, but the number of dollars that 
have to be applied to remedy the situation are not.  It's the most 
common support problem for every organization that we've seen deploy 
mod_dav, our DAV, or other DAV solutions.  DAV + newbie users == high 
support costs.  Any many, many of those phone calls and emails turn 
out to be users who don't read the little pamphlet/webpage that the 
IT department prints explaining that they have to use a different URL 
from their DAV client.

Normally we fix support problems by fixing bugs, changing the default 
configuration, or making the UI more intuitive.

What is frustrating about this problem is that it can't be fixed 
because of the DAV spec.  There is no way to use a DAV client to 
modify source at the same URI as the output, even though that's what 
most people expect.

Using DAV:source isn't an answer because:
	(1) everyone who advocates it agrees that DAV:source as 
specified now is insufficient.
	(2) In a general-purpose web server there is a very real 
messiness with trying to get the right DAV:source property onto all 
of the dynamically generated resources.  A web server may have dozens 
of modules that dynamically generate content, all of which might have 
very different definitions the URI spaces that they handle.
	(3) I think client support for DAV:source (once the spec is 
fixed) will be a long time in coming because it will take time to 
understand the issues.
	(4) You don't get anything extra with DAV:source beyond what 
you could have with Translate.  (Until DASL and friends make the 
scene, that is.)

cjh

--

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 6 Mar 2002 21:27:52 +0100

> Sent: Wednesday, March 06, 2002 9:15 PM
> To: DAV
> ..

OK, let's fix it.

> very different definitions the URI spaces that they handle.

It's not clear to me why in this scenario using the same URI would be
better.

> fixed) will be a long time in coming because it will take time to

That we'll have to see.

> you could have with Translate.  (Until DASL and friends make the

I disagree. For instance, you get separate identifiers for separate things,
conform to HTTP, do not invent new methods, don't break proxies (...to be
continued...).

The remark about DASL is interesting. Could you elaborate?

---

From: CJ Holmes <[email-masked]>
Date: Wed, 6 Mar 2002 11:31:36 -0800

>doesn't make any sense at all.

There's nothing about GETSRC or Translate that would _prevent_ such a 
configuration.  If the administrator wants to support that, then he 
can configure his server accordingly.

cjh

--

---

From: CJ Holmes <[email-masked]>
Date: Wed, 6 Mar 2002 13:38:08 -0800

>It's not clear to me why in this scenario using the same URI would be

It would be better from the user's point of view.  And it is easier 
from the server's point of view because the DAV module only has to 
worry about what is on the file system, not all possible URIs that 
might be supported  by the sum total of all configurations of all 
modules running on the server.

>conform to HTTP, do not invent new methods, don't break proxies (...to be

Having separate identifiers is not a benefit.  As I have explained 
repeatedly, it is often a detriment.  *If* people want to view output 
with their DAV clients and view source with their browsers, *then* it 
is a benefit.  Since the benefit of splitting the source/display 
views into separate resources/URIs is very situation-specific, it 
should not be mandatory.

"Conform to HTTP" is debatable.  Returning the source for a GET if 
that's the intended policy could be interpreted as compliant with the 
HTTP spec, if you hold your head just so.

"Doesn't break proxies" is solvable with the Vary field, and is a 
minor implementation detail.

In short, in most cases you don't get anything.

>The remark about DASL is interesting. Could you elaborate?

DASL makes things more interesting because it allows for searching on 
properties.  You might have content with properties that need to be 
searchable, generated by sources with a very different set of 
properties.  (eg: a single source document that generates several 
display documents with searchable properties of different values.) 
Since in this case you _want_ different properties between 
source/display views you get a real benefit from splitting them into 
separate resources with separate URIs.

I confess to not having read the spec, but it sounds like ACL is 
interesting because you could allow the NaturalSciences department 
(as an example) to restrict GET, HEAD, and POST operations in some 
display spaces and assign authoring/editing rights internally in some 
other spaces.  In short, it might allow  for a very author-driven 
security system instead of an administrator-driven one (without 
resorting to editing .htaccess files).  Still, if you had a way to 
differentiate between GET and GETSOURCE, you wouldn't necessarily 
benefit from splitting the source/display resource, but it still 
might be very convenient to do so.

This is why I keep coming back to the desireability of DAV:source. 
It should be fixed, and its support encouraged.  But there also needs 
to be room for people who prefer the simplicity of the single URI, 
who just use DAV for messing with source, and who don't benefit from 
differentiating source/display URIs.

cjh

--

---

From: CJ Holmes <[email-masked]>
Date: Wed, 6 Mar 2002 13:44:40 -0800

>My remark referred to "know when a GET method originates from a DAV client",
>no to "GETSRC" or the Translate header.

I see your point.  Defining "Translate: F" would probably be better 
than just knowing if the client is DAV-enabled, since that tells you 
if the client wants the source or not.  That way you could have a 
client that could view output or source at the same URI.

Otherwise you would have to define the DAV-Enabled field to only be 
sent when a client wanted to GET the source view of a resource, which 
is a little silly.

cjh

--

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 6 Mar 2002 21:53:33 +0100

> Sent: Wednesday, March 06, 2002 8:32 PM
> To: DAV

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 7 Mar 2002 16:06:16 +0100

Consider a PROPFIND request like:

	<propfind xmlns='DAV:'><prop/></propfind>

I think this is clearly legal (by the DTD) and it can make sense if you're
for instance just getting a list of member URIs for a collection.

What format do we expect for the response body?

1)

<multistatus xmlns="DAV:">
  <response>
    <href>foobar</href>
  </response>
</multistatus>

I think this makes a lot of sense, but it breaks the DTD for the response
element.

2)

<multistatus xmlns="DAV:">
  <response>
    <href>foobar</href>
    <propstat>
      <prop/>
      <status>HTTP/1.1 200 OK</status>
    </propstat>
  </response>
</multistatus>

Conforms to the DTD, but isn't really logical (because in this case you may
expect a 200 propstat element in the case where all queried properties are
reported as 404 NOT FOUND as welll).

3)

<multistatus xmlns="DAV:">
  <response>
    <href>foobar</href>
    <status>HTTP/1.1 200 OK</status>
  </response>
</multistatus>

Conforms to the DTD as well, but isn't very logical. Why would the status
element on response be required, if it is not when properties *are*
reported.

Proposal:

allow format 1).

Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 8 Mar 2002 02:58:34 -0800

> different he should be able to configure his server accordingly.

You are saying that the protocol should have a one-to-one correspondence
with the way that a client application presents information to the user.
I am saying that the protocol must be unabiguous and compatible with
HTTP/1.1, but that doesn't stop the client from presenting it to the
user as if they were the same resource.  The only difference is the URL,
which is something an authoring client can and will present to the user in
a way that is meaningful to that user.  DAV is the access mechanism, not the
user interface.

....Roy

---

From: "Julian Reschke" <[email-masked]>
Date: Mon, 11 Mar 2002 14:09:33 +0100

Hi.

We have implemented the WebDAV Ordered Collections Protocol both in a client
and a server component and feel that it's complete and deserves to be
revived (the current Internet Draft has expired, and it wasn't submitted as
RFC).

I haven taken the task to update it to a new version (03), which is
available from our web site ([1]). The only changes made so far are:

- Updated contact information for Jim Davis.
- Specify charset when using text/xml media type.
- Made sure artwork fits into 72 columns.
- Removed "Public" header from OPTIONS example.

We are not aware of any open issues in the old draft ([2] doesn't list any)
and plan to submit this as an updated Internet Draft after the IETF meetings
are finished.

Feedback appreciated.

Julian

[1]
<http://greenbytes.de/tech/webdav/draft-ietf-webdav-ordering-protocol-latest
.html>
[2] <http://www-old.ics.uci.edu/pub/ietf/webdav/>