Subject: Ideas: GETSRC & MULTIPUT

From: "Jim Whitehead" <[email-masked]>
Date: Mon, 29 Oct 2001 12:14:10 -0800

I'm interested in the list's thoughts on two ideas for DAV improvements:

The first is to introduce a GETSRC method to support access to the
unprocessed source of a resource. This would decouple the dynamic response
of a resource (GET) from its static source (GETSRC).

The second is to introduce the MULTIPUT method to support "PUT with
PROPPATCH" scenarios. MULTIPUT would accept some subset of multipart MIME
packages and atomically write them to the server. This would support the
update of a resource and its metadata in one transaction.

- Jim

---

From: Hartmut Warncke <[email-masked]>
Date: Mon, 29 Oct 2001 21:49:42 +0100

Jim,

a great idea, to my mind especially something like the GETSRC mechanism is
needed,

Best, Hartmut

---

From: "Eric Sedlar" <[email-masked]>
Date: Mon, 29 Oct 2001 12:57:42 -0800

You should probably include the ACL method in the MULTIPUT.

These sound like good ideas, Jim.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Jim Whitehead
> Sent: Monday, October 29, 2001 12:14 PM
> To: WebDAV
> Subject: Ideas: GETSRC & MULTIPUT
>
>

---

From: "Julian Reschke" <[email-masked]>
Date: Mon, 29 Oct 2001 21:54:34 +0100

> Sent: Monday, October 29, 2001 9:14 PM
> To: WebDAV

Jim,

I think both address real-world needs, and non-proprietary solutions would
be welcome.

> of a resource (GET) from its static source (GETSRC).

Yes. This could replace Microsoft's "Translate" header. However, it would
also need to address which set of live properties (src or dynamic) PROPFIND
is supposed to report.

See also the discussion in [1].

> update of a resource and its metadata in one transaction.

Sounds good.

[1] <http://lists.w3.org/Archives/Public/w3c-dist-auth/2001JulSep/0149.html>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 29 Oct 2001 14:25:20 -0800

> of a resource (GET) from its static source (GETSRC).

That would, of course, be completely contrary to the HTTP design and
further propagate the mistake of not accessing properties as resources.
Just use the source link.

> update of a resource and its metadata in one transaction.

Please choose a better name -- MPUT and MULTIPUT means multiple PUT requests,
which was already tried and abandoned.  Maybe you can call it PKG.  Or just
define a mapping to transactions and leave them as separate requests.

....Roy

---

From: "Jim Whitehead" <[email-masked]>
Date: Mon, 29 Oct 2001 16:02:57 -0800

Roy,

> Just use the source link.

The source link didn't work. Telling people to use the source link is not,
IMO, productive.

GETSRC could be defined as something like, "returns to the client a
representation of the resource suitable for editing in an authoring client",
so we avoid issues of getting at the "true" state of the resource. GETSRC
wouldn't have the same caching properties as GET; I don't see that as being
a big problem for authoring.

So, which specific HTTP design principle(s) does this proposal violate?

> Please choose a better name -- MPUT and MULTIPUT means multiple
> PUT requests, which was already tried and abandoned.  Maybe you can call
it
> PKG.  Or just define a mapping to transactions and leave them as separate
> requests.

I don't want to do arbitrary transactions, since that dramatically increases
the implementation burden. But I'd be happy to call it something different.

- Jim

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 29 Oct 2001 16:21:57 -0800

> The source link didn't work. Telling people to use the source link is not,

Under what conditions did it not work?  Do you mean just never implemented?

> So, which specific HTTP design principle(s) does this proposal violate?

The one that calls for different resources to be accessed using different URI.
The set of source URI for some other URI are different resources.  Since
there can be many more than one source per representation, the notion of
creating a new method to avoid one indirection is just wrong.

....Roy

---

From: "Adam Freeman" <[email-masked]>
Date: Mon, 29 Oct 2001 16:51:01 -0800

Hello,
While the link property is a little funky, it addresses a common problem:
having two (OR MORE) different sets of properties for the same resource.
Having a GETSRC and a GET for the same resource limits you to having two
representations of the resource, the original and the modified (or authored
or whatever you want to call it).  What if you want a third representation
of that resource?  With the link property, you can accomplish that (I
think).

GETSRC should return the same thing that a GET without any properties should
return.  It seems redundant.  Why not just have more than one set of
properties.  GET with blank properties is the same as GETSRC.
@m

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Roy T. Fielding
Sent: Monday, October 29, 2001 4:22 PM
To: Jim Whitehead
Cc: WebDAV
Subject: Re: Ideas: GETSRC & MULTIPUT

> So, which specific HTTP design principle(s) does this proposal violate?

The one that calls for different resources to be accessed using different
URI.
The set of source URI for some other URI are different resources.  Since
there can be many more than one source per representation, the notion of
creating a new method to avoid one indirection is just wrong.

....Roy

---

From: "Jim Whitehead" <[email-masked]>
Date: Mon, 29 Oct 2001 17:16:41 -0800

> Under what conditions did it not work?  Do you mean just never

The source link was never implemented.

The two reasons I have heard for this (there may be more) are:

(a) Server implementors did not want to handle the namespace complexity of
automatically creating new areas of the namespace for source resources. That
is, on most servers, the source resources would be "synthetic" -- the server
would be making up the URL for the source resource, and would be managing
interactions with this synthetic resource.

(b) There was insufficient direction given to client implementors on how to
use and handle the source link, especially in the case of multiple sources
for a given resource. The user interfaces of almost all existing DAV clients
implicitly assume there is just a single "source" representation for a
resource. Perhaps DAV's closeness to a traditional network file system
protocol is coming back to haunt us. Certainly, it would involve significant
UI and operational shifts to accommodate more.

I think existing client UIs could handle a standard mechanism to get at 1
source representation for a resource. I do not think they can handle a
mechanism that provides access to multiple source representations. Efforts
to do so will, like the source link, be ignored. I think it is instructive
that the only mechanism that has gained any traction so far is the Translate
header, which provides access to one source representation.

- Jim

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 30 Oct 2001 00:32:45 -0800

> The source link was never implemented.

Well, that's a hell of a lot easier to fix than to rewrite a standard and
all of the models upon which it is based.

> interactions with this synthetic resource.

Huh?  Then how are they going to handle the different access control and
behavior necessary for synthetic versus static resources?  You can't PUT to
a CGI script's synthesis URI and expect to edit the source because it is the
Common Gateway Interface, which necessarily requires that the PUT be gatewayed
through to the script itself.  Trust me on this one -- the notion of GETSRC
as you described it cannot be implemented on the Web.  That only makes sense
for files, not resources, and even then only files that do not contain any
subsidiary resources as inclusions.

> UI and operational shifts to accommodate more.

So the solution is to make it impossibe to edit resources that consist of
multiple sources?  That'll really piss off my customer who has spent most
of this year creating a site that is entirely controlled by a content
management system, where each resource consists of a dynamic mapping to a
database entity catalog.  Each Web page consists of dozens of source elements.
It isn't using webdav right now, but it would be a shame if webdav couldn't
support existing CMS functionality.

> header, which provides access to one source representation.

If it doesn't support the Web, then it doesn't qualify for the name WebDAV.
Feel free to define yet another distributed filesystem protocol within any
other working group of the IETF.  I'd rather wait until it gets implemented
correctly than reduce the standard to something that doesn't meet the need.

....Roy

---

From: "Julian Reschke" <[email-masked]>
Date: Tue, 30 Oct 2001 13:09:56 +0100

> Sent: Tuesday, October 30, 2001 2:17 AM
> To: Roy T. Fielding
> Cc: WebDAV
> Subject: RE: Ideas: GETSRC & MULTIPUT
>
>
> ...

I don't quite agree. If a client is able to select an optional source
resource, it should't have problems offering a selection multiple sources.
However, the server would need to assist in labeling them.

<quote>
Currently, the RFC only gives the following example:

   <?xml version="1.0" encoding="utf-8" ?>
   <D:prop xmlns:D="DAV:" xmlns:F="http://www.foocorp.com/Project/">
     <D:source>
          <D:link>
               <F:projfiles>Source</F:projfiles>
               <D:src>http://foo.bar/program</D:src>
               <D:dst>http://foo.bar/src/main.c</D:dst>
          </D:link>
          <D:link>
               <F:projfiles>Library</F:projfiles>
               <D:src>http://foo.bar/program</D:src>
               <D:dst>http://foo.bar/src/main.lib</D:dst>
          </D:link>
          <D:link>
               <F:projfiles>Makefile</F:projfiles>
               <D:src>http://foo.bar/program</D:src>
               <D:dst>http://foo.bar/src/makefile</D:dst>
          </D:link>
     </D:source>
   </D:prop>

In this example the resource http://foo.bar/program has a source property
that contains three links. Each link contains three elements, two of which,
src and dst, are part of the DAV schema defined in this document, and one
which is defined by the schema http://www.foocorp.com/project/ (Source,
Library, and Makefile). A client which only implements the elements in the
DAV spec will not understand the foocorp elements and will ignore them, thus
seeing the expected source and destination links. An enhanced client may
know about the foocorp elements and be able to present the user with
additional information about the links. This example demonstrates the power
of XML markup, allowing element values to be enhanced without breaking older
clients.
</quote>

This is extremely confusing (at least it was to me), because:

- the individual link elements have a src element that is identical to the
request URI (which isn't shown). Why does it exist at all?

- there's no common way for a client to decide what kind of line this is.

So, taking this in account and moving to a XLink-compatible syntax would
give us instead:

   <D:prop xmlns:D="DAV:" xmlns:F="http://www.foocorp.com/Project/">
     <D:source-set>
          <D:source xlink:href="http://foo.bar/src/main.c"
xlink:role="UriDescribingTheRole" xml:lang="en">
            source file
          </D:source>
          <D:source xlink:href="http://foo.bar/src/main.lib"
xlink:role="UriDescribingTheRole" xml:lang="en">
            library file
          </D:source>
          <D:source xlink:href="http://foo.bar/src/makefile"
xlink:role="UriDescribingTheRole" xml:lang="en">
            makefile
          </D:source>
     </D:source-set>
   </D:prop>

A UI could then use the text content of the D:source element as a
human-readable description of the type of link.

---

From: "Dylan Barrell" <[email-masked]>
Date: Tue, 30 Oct 2001 09:18:00 -0500

Transactions are overkill for this simple single operation.

I prefer a single put with a package made up of different parts - I don't
care what we call that.

--Dylan

> Sent: Monday, October 29, 2001 5:25 PM
> To: Jim Whitehead
> Cc: WebDAV
> PUT requests,

---

From: "Matt Timmermans" <[email-masked]>
Date: Tue, 30 Oct 2001 14:11:27 -0500

GETSRC is evil:

1) It's strictly less powerful than the source link.  What are you going to
do about synthetic _collections_, and multiple synthetic resources with the
same source, or a synthetic document with a source collection (i.e., a
compiled source tree)?

2) It's incompatible with existing DAV clients.  Everyone who writes a DAV
client has to know that they must set Translate: F if they want to work
properly with IIS.  You'd force everyone to retrofit to call GETSRC in place
of GET.

3) It doesn't even satisfy Microsoft's requirements.  Translate: F works for
all methods, not just GET.

4) It breaks the symmetry that HTTP requires between GET and PUT.

> From: Jim Whitehead

I think the big one is that, in most cases, it isn't necessary.  If the web
site has a source tree, it's not unreasonable to expect the web site authors
to know where that source tree is.  They are the authors, after all.

> interactions with this synthetic resource.

There isn't a requirement to automatically synthesize parts of the
namespace.  In fact, what I can easily do Today does everything but fill in
the source links.  I just have to map two virtual directories to the same
location -- one view tree at http://me.com/* and one source tree at
http://me.com/~src/*.  On the latter directory, I can disable script
execution, grant source editing and browsing permissions, and require
stronger authentication.

What Today's web servers don't do is fill in the source link, because in
most cases they can't know how to do it properly.  Oh sure, a Web server
might be able to supply a sourcelink for /cgi/mycgi.exe, but it can't
provide a source link for /cgi/mycgi.exe/doc1, because it doesn't know where
my CGI gets its information from.  Similarly, of course, the Web server
can't implement GETSRC on /cgi/mycgi.exe/doc1.  It's the CGI's job to fill
in those source links if it wants to.

> [...]
> I think existing client UIs could handle a standard mechanism
> to get at 1 source representation for a resource. I do not think they can
handle a
> mechanism that provides access to multiple source representations.

You're right, but a "source representation" is not necessarily an entity
body.  All we really need is some language that tells a client how to
extract a URI for the source from the source link property, i.e., a single
source link, or a "main" source link, or some such thing.

What the source link gets you is a File/Edit item on the menu when you view
the page in a Web browser.  The appropriate thing to do when that item is
selected is clear if you have a single source link -- you PROPFIND it to
determine DAV:resourcetype and DAV:getcontenttype, and then open the link in
whatever client application is appropriate for editing that type of
resource.  If the source resource is a jsp or asp page, then the source file
will open in the appropriate editor.  If the source resource is a
collection, then it will open in your default WebDAV explorer.

GETSRC has a much reduced scope.

> Efforts

That's not all that Translate: does.  Translate: F disables script
processing.  If you PUT to a CGI with Translate: F, you overwrite the CGI.
If you leave out the header, then the CGI processes the request.  If you
PROPFIND mycgi.exe with Translate:F, you get properties of the executable
file.  Without the header, you might get DAV:resroucetype=DAV:collection and
some children.  If you PROPFIND mycgi.exe/somedir with Translate: F, you get
404 or 409.  If you leave out the header, the CGI might give you the
properties of a synthetic DAV-compliant resource.

In short, Translate: F and GETSRC are not equivalent in functionality, and
MS's decision to use Translate: F does not support adding GETSRC to the RFC.

Moving on...

There is, however, a real problem with the source link in 2518 -- it's a
property, and only works for resources that are DAV-compliant.  A CGI, for
example, is a simple way to synthesize a Web page.  There is no reason for
that synthetic resource to be DAV-compliant at all.  CGI authors don't want
to go through all the trouble of implementing PROPFIND just so they don't
have to remember where the source tree is.  How many other methods will CGI
authors have to implement to convince the browser that it should do a
PROPFIND in the first place?

If we want to provide a really useful link to source representations, it
needs to be a simple header that CGIs can just stick in their responses.  We
could define the DAV-source: <URL> header to point the the "primary source"
for that resource.  We can keep the existing source link for DAV-compliant
synthetic resources if we describe the mapping between the two.

---

From: "Jim Whitehead" <[email-masked]>
Date: Tue, 30 Oct 2001 13:58:36 -0800

> database entity catalog.  Each Web page consists of dozens of

How would you propose that authoring tools present this scenario to their
users?  The tool does a PROPFIND, gets back the source property, and finds
there are dozens of links. Now what?

You could pop up a list of the sources, and say "pick one". But without
providing any guidance as to which source corresponds with a particular
aspect of the browser-based view, the user doesn't have much guidance on
which one to select. It doesn't sound like a compelling user experience.

There are two kinds of authoring tools: ones that are structure aware, and
ones that are not. Structure-aware tools (i.e., a tool that understands a
given page is comprised of N other pieces) will be programmed to know the
relationship between sub-elements and the end-user visible Web page, and
will have developed a naming convention that it understands. The source
property won't provide these tools much value.

Structure unaware tools just allow the user to pick a URL and edit it. There
is no notion of structure. Such a tool could use the source property to find
other parts, but could still only allow a user to pick one of them to
individually edit. A person using these tools would eventually need to
understand the naming convention used by the parts and the whole.

Hence, I'm not convinced the source property, as currently defined, provides
much value to any known class of client.

There are really several classes here:

(a) when the mapping of resource to authorable subparts is 1::many
(b) when the mapping of resource to authorable subparts is 1::1 (as with
server-side includes), and the authorable rendition of the resource isn't
returned by GET
(c) when the mapping of resource to authorable subparts is 1::1, and the
authorable rendition of the resource *is* returned by GET (e.g., a Word
document)

Something like "the inverse of PUT" would be handy for case (b). WebDAV only
provides good support for cases (c) (Office, Acrobat, drive mappers) & (a)
(Go Live, Dreamweaver).

> other working group of the IETF.  I'd rather wait until it gets
> implemented correctly than reduce the standard to something that doesn't
> meet the need.

Today, the most frequently used WebDAV client is Web Folders.  A valid way
to view WebDAV is as a network file system protocol. While this was never my
goal, the fact remains that the majority of WebDAV users would never notice
if PROPPATCH, LOCK, and UNLOCK were turned off. This is changing as more
full-featured collaborative authoring clients become available, and enter
into widespread use. But right now, and into the forseeable future, DAV as
Web Filesystem is an important user model.

- Jim

---

From: Greg Stein <[email-masked]>
Date: Wed, 31 Oct 2001 03:00:43 -0800

> much value to any known class of client.

That is an improper conclusion. You only posed two cases, and both deal with
a resource that is comprised of many other resources. If a resource is
produced from a *single* item, then the source property is of tremendous
value. That unaware tool will take me there, and I can get my editing done.
Nothing complicated there, yet incredibly useful.

> provides good support for cases (c) (Office, Acrobat, drive mappers) & (a)

WebDAV also provides quite fine support for (b). I've got empirical evidence
of people doing exactly that. I've done it myself.

Don't warp your cases so that you can draw a desired conclusion :-)

The 1::many problem is hard, sure. But the existing solution provides
structure for that. Maybe you could argue some extensions, such as a
DAV:displayname for each link pair, but the concept is sound.

> Web Filesystem is an important user model.

Okay, fine. But what does that have to do with GETSRC or the source
property? Great -- so people can browse... what about it? The capabilities
offered by Web Folders are also going to be independent of the source stuff
we're discussing.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/