Subject: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)

From: "Lisa Dusseault" <[email-masked]>
Date: Tue, 7 Oct 2003 11:48:51 -0700

> > What if we ditched the DTD entirely and simply use English?
> 
> I'd prefer to keep them and continue to use them the same way 
> as currently done in RFC2518, RFC3253 and the various drafts 
> closing completion.

Does that mean we go back to what was in RFC2518 before the 'bis'
changes started?  Then we've failed to address the issue where
we had interoperability problems because implementations did not
allow extension elements in places where the spec ought to allow
them.  

I know this is a failure of the implementors to follow the text 
in the spec to its logical conclusion, but as we've seen a clear
spec is crucial to easing the way to interoperability.

I can't tell what the consensus is here.  I thought originally
there was a consensus to clarify which elements could be extended
and in what way.  I attempted to do this by altering the DTD, 
and I also proposed doing this by replacing DTDs with English.
The third option is to go back to RFC2518 DTDs and ignore the 
problem.  I haven't seen a fourth concrete option.

> All that's needed is the following general clarification:
> 
> "This document uses XML DTD fragments as a purely notational 
> convention. WebDAV request and response bodies can not be 
> validated due to the specific extensibility rules defined in 
> section 23 of [RFC2518] and due to the fact that all XML 
> elements defined by this specification use the XML namespace 
> name "DAV:". In particular:
> 
> - element names use the "DAV:" namespace,
> - element ordering is irrelevant,
> - extension elements (elements not already defined as valid 
> child elements) may be added anywhere, except when explicitly 
> stated otherwise,
> - extension attributes (attributes not already defined as 
> valid for this
> element) may be added anywhere, except when explicitly stated 
> otherwise."

This doesn't seem sufficient to me unless we simply decide not
to address the problem.  RFC2518 already had general language of
this sort to say that generally extensibility should be allowed.
To solve the problem, we need to specify for every element how
it may be extended, and what it means to extend it. Or we can 
decide not to solve the problem.

> > This would tend to drive us, the spec writers, to include more 
> > information, more
> > guidance, rather than less.  That's because the DTDs don't provide an
easy
> > place to say whether an element may be extended with new
> > elements, or under
> > what conditions an element is required.

> If this is the main issue, clarifying that an "ANY" content model 
> indicates that the element is *not* extensible (contrary to the 
> default case) may suffice (note that this would apply to the 
> content of the DAV:prop element in PROPFIND marshalling).

This seems opposite to what I'd expect "ANY" would mean.  The prop
element is the *most* extensible because all implementations know
you can put any element child in here, as long as it is a property name.

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Tue, 7 Oct 2003 21:14:11 +0200

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Lisa Dusseault
> Sent: Tuesday, October 07, 2003 8:49 PM
> To: 'Julian Reschke'; 'Geoffrey M Clemm'; [email-masked]
> Subject: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)
>
>
>
>

Nobody suggested not doing anything at all. However, I also don't remember
any agreement about removing all formal definitions (be it DTD or a
different syntax) in favor of just using plain English. In fact, there
doesn't seem to be an issue on the RFC2518 issues list related to this at
all.

> problem.  I haven't seen a fourth concrete option.

Then you've missed it :-). I've been trying to come up with language that
absolutely clarifies what the DTD fragments say. It's in the ordering draft
that we all discussed just 3 months ago. I think the very same language
should be used in all WebDAV specs (this may be somthing we want to sneak
into the ACL spec during the AUTH period...).

In fact, you're quoting that fourth option below :-).

> decide not to solve the problem.

Again: no, we don't need to specify that, and we can't. If an element is
extensible, there's no point in discussing what an extension could mean. It
could mean anything, that's the point of an extension. On the other hand, if
an element can't be extended (in RFC2518, that seems to be just <prop> and
<resourcetype>), then that's all we need to say as well.

> you can put any element child in here, as long as it is a property name.

Sorry, Lisa, but this is just plain wrong. <prop> is not extensible
*because* it already has a defined semantics for each possible child
element. That's *why* the content model say's "ANY". It means: you could put
any element here, and the spec defines what that means.

On the other hand, elements with closed content models (such as <propfind>)
are the elements that *can* be extended. Looking at the DTD in RFC2518,
"ANY" is used exactly three times:

prop: whatever child element you put here, it identifies a property
resourcetype: whatever child element you put here, it identifies a resource
type
owner: content model is completely by definition

So, with the exception of DAV:owner, "ANY" is used exactly in those cases,
where the element can *not* be extended, and I'd be really surprised to hear
that this wasn't on purpose.

Hmm. Maybe the issue is that there's a disagreement about what "extensible"
mean. When I say "extensible", I'm talking about the thing described in
RFC2518, Appendix "Example - Unknown XML Element"...

Julian

--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: "Lisa Dusseault" <[email-masked]>
Date: Tue, 7 Oct 2003 14:11:55 -0700

> Nobody suggested not doing anything at all. However, I also 
> don't remember any agreement about removing all formal 
> definitions (be it DTD or a different syntax) in favor of 
> just using plain English. In fact, there doesn't seem to be 
> an issue on the RFC2518 issues list related to this at all.

That's not what I said - I also don't recall any agreement about
removing all formal definitions.  Instead, I believe we decided to 
address a certain problem, which was interoperability problems due
to implementations having trouble accepting extensions in some 
elements.  Assuming we still have consensus about addressing this
problem, I'm trying to determine consensus about *how* to address
this problem.

> Then you've missed it :-). I've been trying to come up with 
> language that absolutely clarifies what the DTD fragments 
> say. It's in the ordering draft that we all discussed just 3 
> months ago. I think the very same language should be used in 
> all WebDAV specs (this may be somthing we want to sneak into 
> the ACL spec during the AUTH period...).

I'm sorry, I don't see how the ordering draft does anything 
signficantly different here. For example, this is from the 
ordering draft: 

   <!ELEMENT orderpatch (ordering-type?, order-member*) >

This is exactly like RFC2518, and I couldn't find additional text 
to specify whether orderpatch can have custom child elements.

> To solve 
> > the problem, we need to specify for every element how it may be 
> > extended, and what it means to extend it. Or we can decide not to 
> > solve the problem.
> 
> Again: no, we don't need to specify that, and we can't. If an 
> element is extensible, there's no point in discussing what an 
> extension could mean. It could mean anything, that's the 
> point of an extension. On the other hand, if an element can't 
> be extended (in RFC2518, that seems to be just <prop> and 

Yes, we can do that.  For example, the text for RFC2518bis says:

		The prop XML element is a generic container for 
            properties defined on resources.  All elements inside a 
            prop XML element MUST define properties related to the 
            resource.  No other elements may be used inside of a prop 
            element. 

That's an example of a "negative" restriction, where the element 
being defined can't be extended in just any way.  The opposite example
might be to define the element as extensible in the DTD:

   <!ELEMENT resourcetype ANY > 

And we could add more text for example (made up on the spot:

	it is expected that resourcetype will be extended with new 
	standard or custom child elements when new resourcetypes are 
	defined.  Resources types that are unrecognized should
	be treated as regular resources.  Resource type values that 
	contain DAV:collection as well as additional unrecognized 
	elements or child elements within DAV:collection should be 
	treated as collections.

Yes, it's a lot of work to do this thinking, but I think we ought to do 
it -- and perhaps throw away the DTD if it gets in our way or confuses
readers.

> *because* it already has a defined semantics for each 
> possible child element. That's *why* the content model say's 
> "ANY". It means: you could put any element here, and the spec 
> defines what that means.

We must have different meanings of the word extensible.  I'm 
trying to adapt DTDs to our XML usage here.  Since the list of 
elements you can put in the 'prop' element is not limited, I call
it extensible.  Would you like me to use a different word for this?
I use this word in this way to distinguish from the <href> element
which is not extensible because it can only contain URIs.

> On the other hand, elements with closed content models (such 
> as <propfind>) are the elements that *can* be extended. 
> Looking at the DTD in RFC2518, "ANY" is used exactly three times:
> 
> resourcetype: whatever child element you put here, it 
> identifies a resource type
> So, with the exception of DAV:owner, "ANY" is used exactly in 
> those cases, where the element can *not* be extended, and I'd 
> be really surprised to hear that this wasn't on purpose.

Would it have been any different if the <prop> element had been
defined in a DTD like this:

   <!ELEMENT prop (creationdate?, displayname?, getcontentlength?,
			getcontentlanguage?, getcontentlength?, 
			getcontenttype?, getetag?, getlastmodified?,
			lockdiscovery?)> 

This would be very much in spirit with the rest of RFC2518, 
because it defines the known elements from the rest of the spec
that can appear in the 'prop' element.  Because of the general
extensibility rules it doesn't change the way it can be used.
Does that change the 'prop' element from what you call unextensible
to what you call extensible?  I think you're making a distinction
without a difference here.

> Hmm. Maybe the issue is that there's a disagreement about 
> what "extensible" mean. When I say "extensible", I'm talking 
> about the thing described in RFC2518, Appendix "Example - 
> Unknown XML Element"...

Heh heh- I said the same thing before I read so far.  

So perhaps we can agree on some more helpful terminology: 

 - *fully extensible* elements like 'response' can contain any
elements not defined in RFC2518 and their meaning is not pre-defined.
 - *partially extensible* elements like 'prop' aren't limited to
a known set of child elements but the meaning of child elements is
strictly defined.  
 - *unextensible* elements (possibly lockscope should be?) have a known and
limited set of child elements allowed.

Given that elements like 'response' and 'prop' are extended (by adding
currently unknown child elements) in different ways - one has a
known meaning, and one has an unknown meaning - surely we can say more
about this in RFC2518bis.  Because otherwise, implementors tend to
assume that the 'resourcetype' element can only contain 'collection' 
or nothing.

Back to the question I'm trying to ask: how should we make it clearer
to implementors that an element like 'resourcetype' can have more than
just 'collection' sub-element or nothing?  Should we attempt to do this
in the DTD (if so how) or in English?  If we do so in English, does the
DTD become misleading, drawing the reader away from the important parts
of the element's definition?

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 8 Oct 2003 01:20:47 +0200

> From: Lisa Dusseault [mailto:[email-masked]]
> Sent: Tuesday, October 07, 2003 11:12 PM
> Subject: RE: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)
>
>
>

OK, we're in agreement. In particular, one issue I do remember are certain
clients that insist in popping up warning messages when a server extends a
response in an unexpected way. However, I fear that RFC2518 already was very
clear about the extensibility rules, so I'm not sure that any amount of
additional language would have avoided that.

The main issue is that we are (were) using DTD fragments, but we also
state(d) that WebDAV messages can't be validated. Many people seem to be
confused about that. The options here some to be:

- try harder to explain it
- choose a schema language that allows to express RFC2518's extensibility
rules (as far as I understand, that would require Relax NG, XML Schema
*won't* do that)
- do not use any formal language at all (this I think would be a mistake)

> to specify whether orderpatch can have custom child elements.

- The additional language about what DTD fragments mean is in section 1
(Notational Conventions)

- In fact, the standard RFC2518 rules apply, so all of these can have custom
child elements -- just like in RFC3253 and ACL.

> being defined can't be extended in just any way.  The opposite example

Yes. It's not extensible. The DTD should state ANY, and probably the
description should state explicitly what this means.

>    <!ELEMENT resourcetype ANY >

No no no. This does not mean that the element is extensible. It means that
the DTD already expects "ANY" content, thus meaning already has been
assigned to arbitrary child elements. Thus the element is *not* extensible.
What's extensible here is the set of valid resource types, but whatever you
put into DAV:resourcetype will always identify a resource type.

> it -- and perhaps throw away the DTD if it gets in our way or confuses

I think we should both keep the DTDs as formal (and normative definition
with the known caveats), *and* explain what it means (when there's a chance
it's not clear).

> it extensible.  Would you like me to use a different word for this?

Yes. It is definitively not extensible in the way described by the "RFC2518
extensibility rules", described in the appendix (which state that any
element you don't know must be ignored).

> which is not extensible because it can only contain URIs.

Fascinating. I'd say it's extensible. For instance, I client should handle

	<d:href xmlns:d="DAV:">foo</d:href>

the same way as

	<d:href xmlns:d="DAV:">foo<foobar/></d:href>

(I'm not saying that I'd expect many clients to handle this gracefully, but
RFC2518 definitively allows this -- maybe that's a candidate for the issues
list).

> 			getcontenttype?, getetag?, getlastmodified?,

Yes. That would mean that I could do a PROPFIND/prop/foobar and the server
would be allowed to ignore the foobar element (which would be wrong).

> extensibility rules it doesn't change the way it can be used.

Yes it does. A server must respect *all* child elements of prop, whether
known or unknown. Thus <prop> is one of the few protocol elements that are
*not* extensible, because the spec already assigns meaning to all possible
child elements (ANY!).

> without a difference here.

Yes it does, and I just described the difference.

> > about the thing described in RFC2518, Appendix "Example -

(actually, I should have been referring to section 14)

> limited set of child elements allowed.

Agreed, except I'd prefer something different than "partially extensible",
and I'm not sure that the class of "unextensible" actually exists.

> assume that the 'resourcetype' element can only contain 'collection'

OK, so we just need to find a good name.

> of the element's definition?

Both. The DTD should be normative and just say "ANY" (like it did in
RFC2518). We also should clearly state what "ANY" means regarding
extensibility (it means that the extensibility rules from section 14 do NOT
apply). In addition, it won't hurt to restate it in plain english -- but I
think the DTDs should stay (unless we can replace them with a better
*formal* definition).

Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Tue, 7 Oct 2003 17:32:21 -0700

> Both. The DTD should be normative and just say "ANY" (like it 
> did in RFC2518). We also should clearly state what "ANY" 
> means regarding extensibility (it means that the 
> extensibility rules from section 14 do NOT apply). In 
> addition, it won't hurt to restate it in plain english -- but 
> I think the DTDs should stay (unless we can replace them with a better
> *formal* definition).

The DTD cannot be normative - it doesn't give the whole picture.
An appendix can't be normative.  The DTD fragments in subsections
of section 12 could be normative in combination with the XML
extension rules *and* the specific element value rules, but 
the spec doesn't specifically say this.

What if we omitted the summary DTD in appendix 1 (23.1), 
and instead had only DTD fragments with each property?  THen the 
reader of the spec would be more likley to read the whole
definition, rather than the shortcut of going to the appendix and
only seeing part of the whole picture.

Another idea.  As long as we're stating what ANY means, why limit 
ourslves.  Eg. we could use pseudo-DTD syntax -- extending the definitions
to say more of what we want them to say. We could call it a WebDAV-DTD
if it offends people to redefine XML DTD.

E.g. 
   <!ELEMENT activelock (lockscope, locktype, depth, owner?, timeout?,
   locktoken?, *) >

   <!ELEMENT prop ANY_PROP >

I don't know how we'd go about formally extending DTDs to do this 
or if that's important, but since the DTD in appendix 23.1 isn't 
normative presumably that would not be illegal.

In my opinion the important thing to keep our eyes on here is to 
make the spec as readable, as implementable, and as conducive to
interoperability, as we can.  "Rules" about DTDs or definitions are
there as our tools, to reshape into better tools if we can, to 
achieve our main goals in this spec.

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 8 Oct 2003 10:01:34 +0200

> Sent: Wednesday, October 08, 2003 2:32 AM

I can be *made* normative. That's what I'm trying to do.

> An appendix can't be normative.  The DTD fragments in subsections

Yes it can. Quoting from RFC223bis (07):

      4.7e Appendixes

         Many RFC documents have appendices, some of which may be very
         extensive.  Common practice is to position Appendixes at the
         very end of a document, after the references.  However, a
         significant set of RFCs have large and dense Appendix sections
         for technical details, which are actually an integral part of
         the document.  In this case, it can be difficult to locate the
         references.  We therefore recommend that, in general,
         references follow the Appendixes in an RFC.

I agree that informative parts are usually in the appendix, but that doesn't
mean that any appendix is automatically non-normative.

> the spec doesn't specifically say this.

That's what we should fix.

> only seeing part of the whole picture.

That sounds like a good idea.

> normative presumably that would not be illegal.

We could, but I'm not sure why we would want to do that. The DTD fragments
define allowable *syntax*. The allowable content model for PROP *is* ANY.

> achieve our main goals in this spec.

Yes. IMHO the basis should be a formal description that is normative, and in
those cases where we feel that people may not "get it", add additional
English language.

Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Wed, 8 Oct 2003 09:24:45 -0700

> I agree that informative parts are usually in the appendix, 
> but that doesn't mean that any appendix is automatically 
> non-normative.
> 
> > of section 12 could be normative in combination with the 
> XML extension 
> > rules *and* the specific element value rules, but the spec doesn't 
> > specifically say this.
> 

Got any suggestions?  I'll try to come up with something here.

> definition, 
> > rather than the shortcut of going to the appendix and only 
> seeing part 
> > of the whole picture.
> 

I'll do that, then.

> > ourslves.  Eg. we could use pseudo-DTD syntax -- extending the 
> > definitions to say more of what we want them to say. We 
> could call it 
> > a WebDAV-DTD if it offends people to redefine XML DTD.
> >
> > E.g.
> > I don't know how we'd go about formally extending DTDs to 
> do this or 
> > if that's important, but since the DTD in appendix 23.1 isn't 
> We could, but I'm not sure why we would want to do that. The 
> DTD fragments define allowable *syntax*. The allowable 
> content model for PROP *is* ANY.

Sure, but you could also say the allowable content model for 
'response' element is ANY.  I will attempt to make this clearer
with English alongside the regular DTD although I still think 
the spec could be clearer without something else formal or 
semi-formal that worked better for us than DTDs.

lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 8 Oct 2003 23:10:28 +0200

> Sent: Wednesday, October 08, 2003 6:25 PM
> ..

We could, but we shouldn't. The spec should use ANY if and only if it
assigns a meaning to ANY type of content. This is the case for <prop> (what
ever child element you find, it identifies a property) or <resourcetype>. It
is not the case for <response>.

> semi-formal that worked better for us than DTDs.

Options:

1) keep the DTDs as they are and clarify what they mean (that's what I've
been trying to do),

2) extend the DTD syntax somehow,

3) switch to something that may allow to formally express what we need (as
far as I understand, only Relax NG can do this).

If there's interest in option 3), I can test that. However, I have my
serious doubts that people are willing to learn yet another syntax just to
fix a very minor issue with the DTD notation.

Julian

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 8 Oct 2003 17:39:10 -0400

I guess I'd go with (1), with (3) as a backup choice.
In particular, (2) is likely to cause more confusion than clarity.
If you're going to do new things, I think it's better to
select a standard new thing, rather than define a non-standard
extension to a standard.

Cheers,
Geoff

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Thu, 9 Oct 2003 15:00:52 -0700

I looked at where this current conversation was going and I thought it best to respond here, where it seems to have started.  I am arriving in the middle of a long-running discussion of DTDs, but I thought these observations might help to calibrate the discussion. (I finally took a course on XML and Web Services and used the occasion to dig into the specifications more than I had up until now.)

1.	ARRANGING FOR DTD VALIDITY OF A WebDAV XML 1.0 "document" 

The key thing to keep in mind is that 

If you intend for a WebDAV "document" to be an XML 1.0 document that is [DTD] valid,

	(a) the ANY case in an element markup declaration means any element that is declared in the DTD, not some undeclared or unspecified element.
	(b) Elements having no markup declaration in the DTD are not permitted in [DTD] valid XML 1.0 documents.  (This is specific technical language around what it means for an XML 1.0 document to be valid as opposed to simply well-formed.)

2.	What that might mean, generally, is that the best external DTD that can be created for WebDAV is one that is a skeleton that must be supplemented by additional markup declarations (perhaps in an internal DTD subset) to account for any extensions. If no extensions are used (e.g., no elements and attributes from any other namespace), the external DTD will work fine by itself. 

3.	When it gets too difficult to handle that, or there is too much variability to deal with, I recommend simply not providing a Document Type Declaration (the <!DOCTYPE ...> structure) and fall back to the WebDAV XML being merely well-formed (and conforming to the WebDAV rules but without asserting [DTD] validity).  

4.	NAMESPACES

I notice that "DAV:" is referred to as the namespace, when, according to the Namespace specification and other specifications since (such as XML Schema), it is technically a prefix.  That is, attribute

	xmlns:DAV="some-URI-for-the-WebDAV-namespace"

establishes the "DAV:" prefix for QNames that refer to local names of the WebDAV namespace.

It is possible to deal with this (a namespace known in advance) and not even require that the prefix be "dav:" in creation of a WebDAV DTD.  This is done, for example, in the DTD that exists for XML Schema documents.  The XML Schema DTD can be used to validate schema documents and provides a way to select alternatives to the exemplar namespace prefixes "xs": and "xsi:".  This is done in a way that requires no modification to the DTD.

5.	RUNNING OUT OF GAS

If a DTD is used for validation of a document, the DTD must account for all of the QNames that can be used for elements and attributes.  That includes all of the namespace declarations and the prefixes that they will declare.  (xmlns:prefix attributes are often declared to be #FIXED and the namespace URI is then regulated too.)

Because of the requirement to rigidly and completely define a document's structural vocabulary, even with the promiscuous allowance of ANY elements, there are times when it doesn't work to have a DTD.  One certainly can't be required if there is meant to be tolerance for arbitrary addition of attributes and elements in any document instance.  The ideas of a fixed, known-in-advance DTD and arbitrary additions of elements and attributes to documents are incompatible.  The alternative, as in the case of RDF, is just too difficult because there basically has to be a custom DTD per RDF document instance, and they have to be maintained together as new versions of the document are created.  (I've actually done that but only for simple cases.)

I find DTDs very useful and often create one simply as a design device when building XML documents.  It is very helpful to have the support of my validating XML editor to let me know when I have something to clean up.  I've even found it useful to use DTDs and XML Schema together.  

Does this shed any light or have I just muddied a situation that you have already struggled through many times?

Regards,

-- Dennis

Dennis E. Hamilton
------------------
AIIM DMware Technical Coordinator
mailto:[email-masked] | gsm:+1-206.779.9430
http://DMware.info
   ODMA Support: http://ODMA.info
OpenPGP public key fingerprint BFE5 EFB8 CB51 8781 5274  C056 D80D 0C3F A393 27EC

-----Original Message-----
Sent: Tuesday, October 07, 2003 11:49

easy

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Fri, 10 Oct 2003 11:44:07 +0200

> [mailto:[email-masked]]On Behalf Of Dennis E. Hamilton
> Sent: Friday, October 10, 2003 12:01 AM
> To: Lisa Dusseault; 'Julian Reschke'; 'Geoffrey M Clemm';
> [email-masked]

Hi Dennis,

just a few line-comments.

> ...
> 	(a) the ANY case in an element markup declaration means any
> element that is declared in the DTD, not some undeclared or
> unspecified element.
> ...

Thanks for the reminder. I had forgotten about that. It basically means that
by definition you can't express the content model of <prop> using a DTD.

> ...
> I notice that "DAV:" is referred to as the namespace, when,
> according to the Namespace specification and other specifications
> since (such as XML Schema), it is technically a prefix.  That is,
> attribute
>
> ...

No, no. "DAV:" is indeed the namespace URI for WebDAV. There are some cases
where it's *also* used as prefix, but that's just a coincidence.

> ...

Regards, Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Fri, 10 Oct 2003 13:45:13 -0700

Ah well, I have learned just enough to be dangerous.

Julian, thanks for the response.  

I have continued to look at this since I wrote yesterday's note.  Here are responses to your points, followed by a brain dump about reconciliation of DAV and XML:

1.	Yes, I forgot that DAV: is also the URI that is spoken of as the "DAV namespace." 

1.1	I find it better to refer to the "DAV namespace" or "DAV namespace URI" rather than the "DAV: namespace" especially because namespaces are often versioned (rather than revised [;<), so the identification in XML will be with different URIs over time.  Ditto for the lock-token namespace.

1.2	In the context of this discussion, the DAV namespace is not a barrier to having a DTD.  It is prefixes (that is QNames) that must be dealt with carefully when a DTD is used.  However, this is completely possible, as the XML Schema folk demonstrated.

2.	You have your finger on a critical and important point.  

2.1	It is not possible to create a generic DTD that can be used for validation of any DAV XML 1.0 "document".  That's specifically because of the RDF hack and the ability to ad lib property QNames.  (I call anything where there is an application technique that maps QNames to URIs as the RDF hack, since RDF seems to have been the first to do it. This creates a variety of problems, some of which are noted in a recent review of the latest RDF working documents.  See <http://lists.w3.org/Archives/Public/www-rdf-comments/2003OctDec/0017.html>.  I don't want to go deeper into this, though it has become an interoperability concern because of a problem concerning when the special application knowledge has to be applied to handle an RDF (or any RDF hack) correctly. [Aside: There is now an approach to (lexical) datatypes in the latest RDF working documents that could well be adapted to DAV property values at some point.]

2.2	It is possible to create a DTD for some specific (family of) DAV XML 1.0 "document".  It might be a loose DTD, in that there are more XML 1.0 documents that satisfy it than are acceptable as DAV XML "documents."  

2.3	It might not be possible to express the content model of <dav:prop> for other reasons, at least as an accurately-constrained grammar.  XML is rather limited in the power of language that can be defined by markup declarations.  Although it looks equivalent to BNF, it isn't.  There are further constraints that limit XML markup vocabularies to ones that can always be handled by greedy, non-backtracking parsers.  The limitations on mixed content models reflect these required simplifications.  Nevertheless, (2.1) is compelling in the case of DAV application of XML 1.0.

2.4	It is important to keep case (2.1) and  case (2.2) distinct.  With regard to wanting valid XML documents, if that is indeed the desire, there is a broader question.  Namely,

<dh:braindump>

3.	What's a DAV XML 1.0 "document"?

3.1	I have been using quotes because there is the notion of an XML 1.0 document, and then there are notions of applications where XML, including XML documents, may occur.  

3.2	What I don't find in WebDAV (I haven't looked at 2518bis) is any clear specification of the relationship of DAV request bodies, DAV response bodies, and, for that matter, DAV documents-as-content, as an application of XML 1.0.

3.3	In the examples in the DAV specification, an XML declaration (<?xml version="1.0" ...?>) is always present.  Although not required by the XML 1.0 Specification, this is tantamount to a declaration that an XML 1.0 document is present (or something equivalent to an XML external entity).  This is a strong statement with regard to what one can count on:
	3.3.1	The only entity declarations that may be presumed by default 
	3.3.2 The well-formedness condition (and existence of a single, though un-named, root element)
	3.3.3 The reservation of attributes and elements with names of the form xml... (including all capitalizations) as exclusively defined by the XML specification or other approved specifications by the XML authority (whoever that might be) -- that is, use of xmlns in all of its variants, use of xml:lang, xml:base the one for whitespace control, etc.
	3.3.4 The definition of what a Document Type Declaration (the <!DOCTYPE ...> structure) is and the rules for markup declarations (for element types, attribute types, entity references, ...).

3.4	An example of (3.2) is the fact that it is not clear that the XML declaration is required for DAV XML documents and whether a Document Type Declaration is forbidden (as it is for SOAP, if I recall correctly).  I can't find where the DAV specification settles the matter, so I don't know if a DAV processor is expected to deal with it if it is there, and whether or not validation is expected or not.

3.5	Because DAV XML 1.0 documents are for an application of XML, and DAV processors are expected to deal with some specific things (such as the DAV version of the RDF hack), there are further considerations.

	3.5.1 These are out-of-band agreements that are not part of XML 1.0 compliance but that impact how the DAV use of XML is to be interpreted.  An important consideration is whether the application is presumed to follow the basic XML processing.  That is, the application could be viewed as fed by an XML processor that is designed to operate in an application-ignorant manner.  

	3.5.2 There is some application context of a transport nature that applies even to the XML 1.0 embedding in HTTP header and response bodies.  This is a problem that SOAP has addressed (I am not sure "dealt with" holds, though) and the SOAP HTTP binding and the WS-I Base Profile 1.0a are worthy of review with regard to coherence between HTTP, MIME types, and DAV XML 1.0 bodies.  (I keep thinking there is a mistake in the WS-I conclusion about byte-order marks, but it is more valuable to have a standard that can be consistently followed than be right about that.  There is a miss-reading of the XML 1.0 specification about this, and it has been perpetuated in the folklore around SOAP.)  

	3.5.3 Side Note: There is no prohibition on unusual encodings for XML documents.  The DAV specification goes too far in presuming there is a limitation.  There is a requirement that a processor always support utf-8 and utf-16.  In the absence of an encoding declaration [in the XML declaration or from context] it must be one or the other.  Note that the default encoding for MIME content type text/xml is ASCII. Note further that encoding="ASCII" does not entail encoding="utf-8".  And all of the DAV specification examples do match up encodings properly.    Whatever the encoding methodology, XML 1.0 documents are always taken to be expressed in Unicode:  That is, the abstraction of the character stream out of the medium is always Unicode.  This means that encoding characters that have no counterpart in (the XML-specified subset of) Unicode may not be used.  This is important with regard to XML 1.0 processors which should probably only see Unicode, on the way in and on the way out.  It would seem that there is a minimum subset of Unicode that any encoding must have corresponding character codes for in order to carry arbitrary XML 1.0 documents at all.

	3.5.4 Because DAV XML 1.0 is an application of XML, it is wise to consider that all XML well-formedness and any DTD validation (if invited) or non-validation (even with a Document Type Declaration present) rules will have been carried out first, before the XML stream is delivered for further application employment (e.g, via a validating DOM processor).  And there may be a desire to carry DAV XML 1.0 in some neutral way as pure XML 1.0 documents.  So there is need for care here.

	3.5.5 I am unclear on its status, but there is a proposal (at least an I-D) to use MIME-type designations like application/xml+DAV as a way to signify that there is an application-processing agreement applicable to the XML being carried in the medium.  This, by the way, would be useful as a type for DAV XML 1.0 documents as content, as well as an useful signal in content-type headers.  (Something for the DAV job jar.)

3.6	Coming back to the specific case of DTD, 

	3.6.1	It might be more appropriate to specifically reject the use of Document Type Declarations in DAV XML 1.0 documents that are used in HTTP bodies and headers and certainly for a MIME type that asserts DAV application.  

	3.6.2 This deals with section 17.7 of the specification too, because there are therefore no external entities and a DAV processor does not have to accommodate such things (or any internal entities other than the standard default ones, &amp; &lt;, ..., plus the encoding notation for Unicode characters).  [You might want to review the XML 1.1 Working Draft in this regard too, just to see if there is anything to anticipate here.]

	3.6.3 Although a Document Type Declaration might be prohibited, it is still possible to describe what the validity is expected to be if a DAV XML 1.0 document were submitted to an XML validator with a DTD constructed according to some prescribed scheme.  There is a hack for this by treating the DAV XML 1.0 document, separated out into a file with its XML declaration, as an external entity of a tiny document that validates the DAV XML 1.0 document by inclusion.  I have used that to apply different XSL transformations to the same XML document without modifying the XML document to add XSL processing instructions.
	I know I saw this trick demonstrated in a W3C document, but I have been unsuccessful finding it again.  (I remember the trick, just not the source.)

OK, end of brain dump.

</dh:braindump>

Relax.  My next class starts on October 16 and I will quiet down for another 8 weeks ...

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Friday, October 10, 2003 02:44
To: [email-masked]; Lisa Dusseault; 'Julian Reschke'; 'Geoffrey
M Clemm'; [email-masked]
Subject: RE: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)

Hi Dennis,

Regards, Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 11 Oct 2003 13:04:03 +0200

Comments inline...

> Sent: Friday, October 10, 2003 10:45 PM
> To: Julian Reschke; Lisa Dusseault; 'Geoffrey M Clemm';
> [email-masked]
> 1.1	I find it better to refer to the "DAV namespace" or "DAV
> namespace URI" rather than the "DAV: namespace" especially
> because namespaces are often versioned (rather than revised [;<),
> so the identification in XML will be with different URIs over
> time.  Ditto for the lock-token namespace.
> ..

Well. There are no plans to change the namespace URI for DAV:. Doing that
would be an incompatible change breaking all existing code (see for instance
how XSLT deals with versioning).

> 1.2	In the context of this discussion, the DAV namespace is not
> a barrier to having a DTD.  It is prefixes (that is QNames) that
> must be dealt with carefully when a DTD is used.  However, this
> is completely possible, as the XML Schema folk demonstrated.

Is it worth the effort? Can you supply (a link to) an example?

> 2.1	It is not possible to create a generic DTD that can be used
> for validation of any DAV XML 1.0 "document".  That's
> specifically because of the RDF hack and the ability to ad lib
> property QNames.  (I call anything where there is an application
> technique that maps QNames to URIs as the RDF hack, since RDF
> seems to have been the first to do it. This creates a variety of
> problems, some of which are noted in a recent review of the
> latest RDF working documents.  See
> <http://lists.w3.org/Archives/Public/www-rdf-comments/2003OctDec/0
> 017.html>.  I don't want to go deeper into this, though it has
> become an interoperability concern because of a problem
> concerning when the special application knowledge has to be
> applied to handle an RDF (or any RDF hack) correctly. [Aside:
> There is now an approach to (lexical) datatypes in the latest RDF
> working documents that could well be adapted to DAV property
> values at some point.]

I'm not sure where the hack is. In WebDAV, property names are identified by
namespaced XML names (as a pair of namespace URI and local name), not as
URI.

> ...
>
> 3.2	What I don't find in WebDAV (I haven't looked at 2518bis)
> is any clear specification of the relationship of DAV request
> bodies, DAV response bodies, and, for that matter, DAV
> documents-as-content, as an application of XML 1.0.

Some WebDAV methods use XML documents as request/response bodies. There is
no notion of DAV-documents-as-content. That's it.

> 3.3	In the examples in the DAV specification, an XML
> declaration (<?xml version="1.0" ...?>) is always present.
> Although not required by the XML 1.0 Specification, this is
> tantamount to a declaration that an XML 1.0 document is present
> (or something equivalent to an XML external entity).  This is a
> strong statement with regard to what one can count on:

Actually the content types we use indicate that as well, therefore the XML
declaration is not necessary. However there are clients/servers that fail
(or did fail) in absence of the XML declaration.

> ...
>
> 3.4	An example of (3.2) is the fact that it is not clear that
> the XML declaration is required for DAV XML documents and whether

As the spec doesn't say that it is required, it isn't.

> a Document Type Declaration is forbidden (as it is for SOAP, if I
> recall correctly).  I can't find where the DAV specification

As the spec doesn't say it's forbidden, it's allowed. Hoewever it does say
that servers/clients must not attempt to validate the documents.

> ...
>
> 	3.5.1 These are out-of-band agreements that are not part of
> XML 1.0 compliance but that impact how the DAV use of XML is to
> be interpreted.  An important consideration is whether the
> application is presumed to follow the basic XML processing.  That
> is, the application could be viewed as fed by an XML processor
> that is designed to operate in an application-ignorant manner.

Of course. Section 8 states:

"The following new HTTP methods use XML as a request and response format.
All DAV compliant clients and resources MUST use XML parsers that are
compliant with [REC-XML]. All XML used in either requests or responses MUST
be, at minimum, well formed. If a server receives ill-formed XML in a
request it MUST reject the entire request with a 400 (Bad Request). If a
client receives ill-formed XML in a response then it MUST NOT assume
anything about the outcome of the executed method and SHOULD treat the
server as malfunctioning."

> 	3.5.2 There is some application context of a transport
> nature that applies even to the XML 1.0 embedding in HTTP header
> and response bodies.  This is a problem that SOAP has addressed
> (I am not sure "dealt with" holds, though) and the SOAP HTTP
> binding and the WS-I Base Profile 1.0a are worthy of review with
> regard to coherence between HTTP, MIME types, and DAV XML 1.0
> bodies.  (I keep thinking there is a mistake in the WS-I
> conclusion about byte-order marks, but it is more valuable to
> have a standard that can be consistently followed than be right
> about that.  There is a miss-reading of the XML 1.0 specification
> about this, and it has been perpetuated in the folklore around SOAP.)

I'm not sure to what you're referring here...

> 	3.5.3 Side Note: There is no prohibition on unusual
> encodings for XML documents.  The DAV specification goes too far
> in presuming there is a limitation.  There is a requirement that

Where...?

> a processor always support utf-8 and utf-16.  In the absence of
> an encoding declaration [in the XML declaration or from context]
> it must be one or the other.  Note that the default encoding for
> MIME content type text/xml is ASCII. Note further that
> encoding="ASCII" does not entail encoding="utf-8".  And all of
> the DAV specification examples do match up encodings properly.
> Whatever the encoding methodology, XML 1.0 documents are always
> taken to be expressed in Unicode:  That is, the abstraction of
> the character stream out of the medium is always Unicode.  This
> means that encoding characters that have no counterpart in (the
> XML-specified subset of) Unicode may not be used.  This is

...such as control characters other then CR, LF and TAB...

> important with regard to XML 1.0 processors which should probably
> only see Unicode, on the way in and on the way out.  It would
> seem that there is a minimum subset of Unicode that any encoding
> must have corresponding character codes for in order to carry
> arbitrary XML 1.0 documents at all.

An encoding that can carry arbitrary XML documents must support all Unicode
characters allowed in XML names (as those can't be escaped).

> 	3.5.4 Because DAV XML 1.0 is an application of XML, it is
> wise to consider that all XML well-formedness and any DTD
> validation (if invited) or non-validation (even with a Document
> Type Declaration present) rules will have been carried out first,
> before the XML stream is delivered for further application
> employment (e.g, via a validating DOM processor).  And there may

A processor is allowed to deliver data to the application before it has
finished checking wellformed-ness. Otherwise XML processors could not be
used in streaming contexts. However, if a wf-error is detected, it must be
signalled and the application must abort processing.

> be a desire to carry DAV XML 1.0 in some neutral way as pure XML
> 1.0 documents.  So there is need for care here.
>
> ...
>
> 	3.6.1	It might be more appropriate to specifically reject
> the use of Document Type Declarations in DAV XML 1.0 documents
> that are used in HTTP bodies and headers and certainly for a MIME
> type that asserts DAV application.

Why does it matter? The spec says that they must be ignored, and in real
life I haver never seen a client or server sending a DOCTYPE. I don't think
there's an issue here.

> 	3.6.2 This deals with section 17.7 of the specification
> too, because there are therefore no external entities and a DAV
> processor does not have to accommodate such things (or any
> internal entities other than the standard default ones, &amp;
> &lt;, ..., plus the encoding notation for Unicode characters).
> [You might want to review the XML 1.1 Working Draft in this
> regard too, just to see if there is anything to anticipate here.]

--> <http://lists.w3.org/Archives/Public/w3c-dist-auth/2002OctDec/0148.html>

Regards, Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Sat, 11 Oct 2003 14:30:38 -0700

Hi Julian,

I am responding to your last comment first, since I think this is an issue to have be clear right away.  This is one of the places where I see missing precision in definition of the DAV reliance on XML 1.0:

1.	Accepting that the WebDAV specification says that WebDAV XML is not to be validated, that is not the same as saying that any <!DOCTYPE ...> Document Type Declaration can be ignored. 

2.	XML 1.0 gives specific instructions about what must be done with a Document Type Declaration even for a non-validating processor. The key statement is in XML 1.0 Recommendation section 5.1, Validating and Non-Validating Processors <http://www.w3.org/TR/REC-xml#proc-types>.  

3.	The use of Document Type Declarations in non-validating situations is illustrated in the RDF Primer.   See <http://www.w3.org/TR/2003/WD-rdf-primer-20031010/#example48> and the discussion immediately preceding the example.  The official (candidate) RDF Schema for OWL uses this very technique (<http://www.w3.org/TR/2003/CR-owl-ref-20030818/#appB>). 

(These illustrate the RDF hack, by the way.  In some parts of RDF, (adlib) QNames *are* mapped to URIs by concatenation of the namespace URI that the prefix stands for, plus the local-name part, into a new URI.  The <!DOCTYPE ...> is used to set up entity declarations that are used in xmlns:... attributes and in directly writing the URI form in literals and attribute values.)

4.	Rather than have so much sensitivity to out-of-band nuances, I think it would be cleaner and more interoperable (for DAV, not arbitrary XML) to have the DAV application of XML 1.0 specify that a Document Type Declaration must not be present.  Then (1) the XML can't be presumed to be validatable, and (2) there is no confusion about the validating versus non-validating use as there is when one is provided.
	Since, as you say, it doesn't seem to be used, it might be a good idea to simplify here and say that it is not meant to be.
	I don't have any skin in this particular game.  I just think it is a good idea, especially since there doesn't seem to be any demonstration of interoperability with Document Type Declarations present.

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Saturday, October 11, 2003 04:04
To: [email-masked]; [email-masked]
Subject: RE: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)

Comments inline...

[ ... ]

[ ... ]

Regards, Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Sat, 11 Oct 2003 23:19:08 -0700

Gee, Namespaces are versioned all of the time without breaking any existing code.  I didn't mean to suggest obsolescing anything, only that it is useful to speak in a way where a new URI might be usable for a *later* namespace.  The existing namespace URI would still work, and refer to the current namespace.  A future namespace might have the current one as a proper subset and that would work too, even though there is a different URI for it.

I realize that there are DAV headers for this, as part of feature coordination and leveling, It is often handy to also rev the namespace for anything specific to the use of XML.  For example, with the SOAP XML binding to HTTP, a closer analog to WebDAV, I would think, the SOAP 1.2 Recommendation establishes a different namespace than is used for the SOAP 1.1 level, and the SOAP 1.2 specification deals with how to deal properly with SOAP 1.1 as a legacy.  This supports the versioning of XML Schema definitions too, since XML Schema definitions tend to be targeted to specific namespaces.

So I wasn't anticipating any kind of versioning of namespace URIs that would break compatibility with or even usage of the current specification.  I see it as keeping a namespace stable and known, whether or not it is a proper subset of a future one.

Namespace versions will already have to be dealt with when properties, for example, are taken from other XML-mapped vocabularies, such as Dublin Core, which has a specific namespace for its current 1.1 level of element definitions.  Something like that will doubtless happen with RDF, if it isn't underway already in Friday's Last-Call Working Draft documents.

Having said that, I am not wedded to the idea. I was just suggesting a way of speaking that would allow for specific-namespace versioning with minimal editorial impact in future revisions of the DAV specification.  I assume that 2518bis is far enough along that one would not want to mess with it.  In any case, it is an editorial nuance, nothing substantive.

I do think of namespace versioning as a key provision for preservation of interoperability over time, like versioning COM interfaces (and their UUIDs), and never changing them.  Or providing for versioning in Java package names.  Or even like W3C specification URLs where you can refer to a specific edition or simply the current latest of the progression.

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Saturday, October 11, 2003 04:04
To: [email-masked]; [email-masked]
Subject: RE: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)

Comments inline...

[ ... ]

---

From: Geoffrey M Clemm <[email-masked]>
Date: Sun, 12 Oct 2003 09:30:05 -0400

By "breaking all existing code", I believe Julian was referring
to the result that no client that only understands V1
of the namespace can work against a server
that only understands V2 of the namespace, and vica versa.

This means that each time you rev the namespace, you introduce
an interoperability nightmare for both clients and servers,
because inevitably, new clients and servers will be written that
only understand the "current" version of the namespace, which
means that client and server implementors that care about interoperability
will have to implement variant code for every new rev of the namespace.

A significant percentage (possibly even a majority) of the hard
design work of WebDAV and its extensions has been 
how to add in functionality in a way that is compatible with the
previous standards.  So I agree with Julian that the DAV namespace
should never be "rev"ed, but rather every new WebDAV protocol has
to figure out how to live within the single existing DAV namespace.

Cheers,
Geoff

Dennis wrote on 10/12/2003 02:19:08 AM:

> 
> Gee, Namespaces are versioned all of the time without breaking any 
> existing code.  I didn't mean to suggest obsolescing anything, only 
> that it is useful to speak in a way where a new URI might be usable 
> for a *later* namespace.  The existing namespace URI would still 
> work, and refer to the current namespace.  A future namespace might 
> have the current one as a proper subset and that would work too, 
> even though there is a different URI for it.
> 
> I realize that there are DAV headers for this, as part of feature 
> coordination and leveling, It is often handy to also rev the 
> namespace for anything specific to the use of XML.  For example, 
> with the SOAP XML binding to HTTP, a closer analog to WebDAV, I 
> would think, the SOAP 1.2 Recommendation establishes a different 
> namespace than is used for the SOAP 1.1 level, and the SOAP 1.2 
> specification deals with how to deal properly with SOAP 1.1 as a 
> legacy.  This supports the versioning of XML Schema definitions too,
> since XML Schema definitions tend to be targeted to specific namespaces.
> 
> So I wasn't anticipating any kind of versioning of namespace URIs 
> that would break compatibility with or even usage of the current 
> specification.  I see it as keeping a namespace stable and known, 
> whether or not it is a proper subset of a future one.
> 
> Namespace versions will already have to be dealt with when 
> properties, for example, are taken from other XML-mapped 
> vocabularies, such as Dublin Core, which has a specific namespace 
> for its current 1.1 level of element definitions.  Something like 
> that will doubtless happen with RDF, if it isn't underway already in
> Friday's Last-Call Working Draft documents.
> 
> Having said that, I am not wedded to the idea. I was just suggesting
> a way of speaking that would allow for specific-namespace versioning
> with minimal editorial impact in future revisions of the DAV 
> specification.  I assume that 2518bis is far enough along that one 
> would not want to mess with it.  In any case, it is an editorial 
> nuance, nothing substantive.
> 
> I do think of namespace versioning as a key provision for 
> preservation of interoperability over time, like versioning COM 
> interfaces (and their UUIDs), and never changing them.  Or providing
> for versioning in Java package names.  Or even like W3C 
> specification URLs where you can refer to a specific edition or 
> simply the current latest of the progression.
> 
> -- Dennis
> 
 
> > From: Of Dennis E. Hamilton

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Sun, 12 Oct 2003 13:12:15 -0700

Geoff,

Thanks.  I didn't realize until your note that we have different nightmares, and both of them are about interoperability.

I also agree that the level of attention that you suggest in 

will make it unnecessary to revise the namespace, although you don't entirely solve my nightmare with that approach.  The one problem that I have is that a changed or modified namespace is not construed by everyone (and definitely not me) as a "single existing ... namespace." 

Also, I am talking about namespace versioning, not revision.  Any change to a namespace is a revision, in my book.  A version is a differently-identified derivative, in my book.  Maybe there is better nomenclature for that. I am interested in suggestions for that.

Meanwhile ...

1.	Silent Up-Leveling Dangers

The java.lang "namespace" is more-or-less maintained the way you describe.  There are deprecated classes and methods, but they are still there.  There are also new elements that are added to the namespace from time to time.  

Reliance on new additions creates down-level compatibility problems, and it happens in a kind of blissful unawareness.  But if you distribute a class file, it can be found to not run on earlier versions of the JVM and its libraries precisely because an added feature has been relied on without concern for (or probably even any awareness of) version dependency.  For example, I am fond of

	double enteredValue = java.lang.Double.parseDouble(someInputString);

and in using it I am producing a program that will not operate with versions of Java prior to JDK 1.2.  If I make use of the 

	java.lang.String.matches(some-regular-expression) 

method, my program will not operate with any version of Java prior to JDK 1.4.

When I first started using Java (earlier this year in a course), I was meticulous about determining the earliest version my code would work with, and saying so in prominently placed comments and any documentation.  I have gotten lazy and I don't do that any longer.  I should probably build a tool to figure it out for me and then let me decide if I want to make changes to expand the down-level compatibility of whatever I produce for widespread usage. It is probably a hack off of JavaDoc.

2.	My nightmare

What has me wake up in the middle of the night is any situation that I have stepped over where (1) there is any change to the semantics of the same name between two versions of something or (2) there is vocabulary [local names] in one and not the other, and (3) I haven't provided for explicit differentiation so that (4) I can be specific in what interface contract I am depending on or providing, and (5) it is then clear what is a bug and what isn't.

3.	My sense of interoperability

3.1	I do not find it a nightmare to encounter a clearly-versioned use of XML that says namespace-version-2 is acceptable and name-space-version-1 is not.  I find that honest and clean.  It makes a clear declaration of where interoperability is not provided.  

3.2	I also see no problem in an XML application saying that namespace-version-2 is preferred and acceptable and that namespace-version-1 will be honored and responded to in complete compliance with the rules that apply to namespace-version-1.  That for me is the best of interoperability.  I could even foresee a specification (or an interoperability profile that applies to a specification) where acceptors of namespace-version-2 are required to accept namespace-version-1 and, in so doing, behave as if namespace-version-2 has never been heard of.  

3.3	Even better is that there is only namespace-version-1 and any client who uses namespace-version-1 will never be surprised by any response and any acceptor of namespace-version-1 will never be surprised by an unexpected legal use (as the result of some revision that has occurred).  That's a wonderful contract, as long as it can be maintained.  Clearly, if DAV accomplishes that, there is nothing I could possibly quarrel with.  

4.	Wrap-up

I find that the provision for namespace versioning, even if there is never more than one version, is a bit different than the care one takes to foster interoperability and then make clear and readily confirmable what the interoperability agreement is in actual situations.  Of course, namespace applies only to the XML part and not to DAV protocol in the context of HTTP, DAV methods and all the rest.

I suspect our nightmares overlap, but not completely. 

As I said, this is not a proposal.  And your feedback is really useful.  It is valuable to notice that we might have similar but not the same nightmares around breakdowns in interoperability.  I need to document the guidelines that I intend to follow in work that I do, and what I think that provides.

-- Dennis

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Geoffrey M Clemm
Sent: Sunday, October 12, 2003 06:30
To: [email-masked]
Subject: Re: Versioning Namespaces

Cheers, 
Geoff 

[ ... ]

---

From: "Julian Reschke" <[email-masked]>
Date: Sun, 12 Oct 2003 22:18:25 +0200

> Sent: Saturday, October 11, 2003 11:31 PM
> To: Julian Reschke; [email-masked]
> Subject: Ignoring Versus Not Validating <!DOCTYPE ...>
>
>
>
> Hi Julian,
>
> I am responding to your last comment first, since I think this is
> an issue to have be clear right away.  This is one of the places
> where I see missing precision in definition of the DAV reliance
> on XML 1.0:
>
> 1.	Accepting that the WebDAV specification says that WebDAV
> XML is not to be validated, that is not the same as saying that
> any <!DOCTYPE ...> Document Type Declaration can be ignored.
>
> 2.	XML 1.0 gives specific instructions about what must be done
> with a Document Type Declaration even for a non-validating
> processor. The key statement is in XML 1.0 Recommendation section
> 5.1, Validating and Non-Validating Processors
> <http://www.w3.org/TR/REC-xml#proc-types>.

True. I was oversimplifying.

> 3.	The use of Document Type Declarations in non-validating
> situations is illustrated in the RDF Primer.   See
> <http://www.w3.org/TR/2003/WD-rdf-primer-20031010/#example48> and
> the discussion immediately preceding the example.  The official
> (candidate) RDF Schema for OWL uses this very technique
> (<http://www.w3.org/TR/2003/CR-owl-ref-20030818/#appB>).
>
> (These illustrate the RDF hack, by the way.  In some parts of
> RDF, (adlib) QNames *are* mapped to URIs by concatenation of the
> namespace URI that the prefix stands for, plus the local-name
> part, into a new URI.  The <!DOCTYPE ...> is used to set up
> entity declarations that are used in xmlns:... attributes and in
> directly writing the URI form in literals and attribute values.)
>
> 4.	Rather than have so much sensitivity to out-of-band
> nuances, I think it would be cleaner and more interoperable (for
> DAV, not arbitrary XML) to have the DAV application of XML 1.0
> specify that a Document Type Declaration must not be present.
> Then (1) the XML can't be presumed to be validatable, and (2)
> there is no confusion about the validating versus non-validating
> use as there is when one is provided.
> 	Since, as you say, it doesn't seem to be used, it might be
> a good idea to simplify here and say that it is not meant to be.

I think RFC2518bis is saying that (see
<http://greenbytes.de/tech/webdav/draft-ietf-webdav-rfc2518bis-04.txt>,
section 4.4).

Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Sun, 12 Oct 2003 22:30:14 +0200

> Sent: Sunday, October 12, 2003 8:19 AM
> Subject: Versioning Namespaces
>
>
>
> Gee, Namespaces are versioned all of the time without breaking
> any existing code.  I didn't mean to suggest obsolescing

I think that depends on what you understand with "breaking". If the DAV:
namespace URI gets versioned that is *changed*, no existing code will be
able to process messages using the new namespace, even if the local names
match and do have the same semantics.

> anything, only that it is useful to speak in a way where a new
> URI might be usable for a *later* namespace.  The existing
> namespace URI would still work, and refer to the current
> namespace.  A future namespace might have the current one as a
> proper subset and that would work too, even though there is a
> different URI for it.

That would mean that new elements are put into a new namespace, or existing
elements are getting changed and now use a new namespace. That's possible,
but it doesn't (IMHO) really means that the namespace is versioned -- you
are just adding a second namespace for new elements, and the old one isn't
changed at all. That would be a compatible change, however it would mean
that when creating messages, you always have to worry about picking the
right namespace URI. One of the reasons why XSLT2 elements use the existing
XSLT namespace URI.

> legacy.  This supports the versioning of XML Schema definitions
> too, since XML Schema definitions tend to be targeted to specific
> namespaces.

I don't know a lot about SOAP, but SOAP 1.2 is the first revision of the
spec issued by a (sort-of) standards body, so it makes perfect sense to
restart and not to worry too much about legacy.

> for its current 1.1 level of element definitions.  Something like

Funny that you mention that, because I came across this issue just a few
weeks ago (the SAP EP portal will support extraction of RFC2731-formatted
custom metadata from remote HTML pages, for instance as support for
crawlers).

From a WebDAV point of view, DC properties with the same local name differ
when they use different namespace URIs. It seems that DC people have found
this to be a problem as well. Looking at
<http://dublincore.org/documents/2001/10/26/dcmi-namespace/>, they seem to
recommend to use unique (and hopefully) stable namespace URIs for their
elements.

> that will doubtless happen with RDF, if it isn't underway already
> in Friday's Last-Call Working Draft documents.
>
> Having said that, I am not wedded to the idea. I was just
> suggesting a way of speaking that would allow for
> specific-namespace versioning with minimal editorial impact in
> future revisions of the DAV specification.  I assume that 2518bis
> is far enough along that one would not want to mess with it.  In
> any case, it is an editorial nuance, nothing substantive.
>
> interfaces (and their UUIDs), and never changing them.  Or
> providing for versioning in Java package names.  Or even like W3C

Doing versioning right is *hard*. I think a few months ago it was suggested
that the W3C TAG would pick up the topic, and they rejected that :-)

Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Mon, 13 Oct 2003 11:17:15 -0700

Thanks, Julian.

I looked at 2518bis-04 and section 4.4 does not speak to the presence or absence of material in XML document headers.  It says that the given DTD (in 24.1 and also throughout the document) is informational and is not meant to be usable for validation of DAV XML 1.0 documents.  [We have already discussed that ANY, as used in the DTD, doesn't accomplish the intended purpose.]

Section 4.4 does not speak to rules for the appearance of <!DOCTYPE ...> or anything else in the XML Prolog and Document Type Declaration [REC-XML section 2.8, 
<http://www.w3.org/TR/REC-xml#sec-prolog-dtd>] for DAV XML documents.

I also checked bis-04 sections 8.1.1, 15, 18.5, 18.6, 19, 24, 24.1, and 24.2 and there does not seem to be anything to resolve this question.  There are mildly contradictory statements in 8.1.1 and 18.6, though. I didn't look elsewhere.

I continue to recommend that the presence of a <!DOCTYPE ...> declaration be forbidden in the XML Prolog of DAV XML in HTTP request bodies and response bodies.  This will also take care of 18.6, since the only way external entities may be declared is in the DTD.

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Sunday, October 12, 2003 13:18
To: [email-masked]; Julian Reschke; [email-masked]
Subject: RE: Ignoring Versus Not Validating <!DOCTYPE ...>

[ ... ]

Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Mon, 13 Oct 2003 20:28:47 +0200

Dennis,

RFC2518 hasn't forbidden a DOCTYPE so far, and as far as I can tell, this
never has caused an interop problem. So please let's stay focused in
identifying only those things that are broken.

What the spec should say is that a recipient MUST NOT attempt to validate a
message (either using a hard-wired DTD or a DOCTYPE declaration when
present). You are right that section 4.4 currently doesn't state this, so
this should be put onto the issues list (-> Jason, listening? :-).

Regards, Julian

> Sent: Monday, October 13, 2003 8:17 PM
> I looked at 2518bis-04 and section 4.4 does not speak to the
> presence or absence of material in XML document headers.  It says
> that the given DTD (in 24.1 and also throughout the document) is
> informational and is not meant to be usable for validation of DAV
> XML 1.0 documents.  [We have already discussed that ANY, as used
> in the DTD, doesn't accomplish the intended purpose.]
>
> Section 4.4 does not speak to rules for the appearance of
> <!DOCTYPE ...> or anything else in the XML Prolog and Document
> Type Declaration [REC-XML section 2.8,
> I also checked bis-04 sections 8.1.1, 15, 18.5, 18.6, 19, 24,
> 24.1, and 24.2 and there does not seem to be anything to resolve
> this question.  There are mildly contradictory statements in
> 8.1.1 and 18.6, though. I didn't look elsewhere.
>
> I continue to recommend that the presence of a <!DOCTYPE ...>
> declaration be forbidden in the XML Prolog of DAV XML in HTTP
> request bodies and response bodies.  This will also take care of
> 18.6, since the only way external entities may be declared is in the DTD.
>
> -- Dennis
>

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Mon, 13 Oct 2003 12:13:47 -0700

Just a couple of things on the practical use of namespace versioning.  

I just want to sharpen some cases as long as we are discussing this.  I don't have any basis for suggesting that a particular one applies to DAV (or not).

1.	Although I have seen cases where there is an added namespace (for example, Dublin Core uses two), normally the idea is that the new namespace includes everything needed.  Sometimes namespaces are partitioned/subsetted for convenience or logical appropriateness, but that is another matter (RDF uses two in that way (usually prefixed rdf: and rdfs:), and so does XML Schema (usually prefixed xs: and xsi:).

2.	If there is legacy interoperability, which is generally a strong requirement as well as practical reality, then the usual idea would be that uses of the original namespace would continue to do so, and the new namespace would be used only where features of that namespace, not available with the earlier namespace, are required.
	There is the matter of people specifying the newer namespace out of convenience or carelessness, and this causing processors that only support the original namespace to reject the material.  It is like updating your Microsoft Office Suite and not doing Save As ... to the oldest compatible version.  On the other hand, the Microsoft Office application actually has all of the information necessary to correctly do a Save As ... to the oldest compatible version. And a Java processor could have all of the information necessary to correctly specify the oldest versions of JDK that a compiled class is definitely compatible with [;<).
	Note that changing the namespace without versioning the name does not make this particular problem go away, it just changes the way the problem shows up.  Versioning the namespace has it show up sharply and up front, and that may or may not serve the requirements of a particular application setting. I find that it has a strong accountability quality that I prefer, as I mentioned in the exchange of nightmares with Geoff.  But, you know,  your nightmare may vary.

3.	For changes such that the new namespace does not recognize the old as a subset, there are more problems, obviously.  That is the case for SOAP, where, like it or not, SOAP 1.1 is in serious de facto use (and is the level used in the WS-I Base Profile 1.0a for interoperable use in web services).  The SOAP 1.2 namespace does not include the SOAP 1.1 nomenclature and elements as a proper subset.  There are substantial and important differences.  And SOAP 1.2 accounts for what the SOAP 1.1 behavior will be on receiving a SOAP 1.2 request. It also specifies how SOAP 1.2 processors can properly accept SOAP 1.1 and also how to properly reject SOAP 1.1 by using responses that a SOAP 1.1 requester will interpret correctly.

I have my own druthers among that set.  If the namespace can be held fixed and not changed as Web DAV is revised, that certainly preserves maximum interoperability with ones own legacy.  I'm for it.

I am grateful for this discussion.  There is a lot here that I am going to pay attention to as I look at applying the XML technology set in other situations, and in writing about it and encouraging best practices.  This has been useful to me whether or not it is particularly apt in the context of DAV.

Thank you,

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Sunday, October 12, 2003 13:30
To: [email-masked]; Julian Reschke; [email-masked]
Subject: RE: Versioning Namespaces

Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Mon, 13 Oct 2003 14:41:11 -0700

A.	RECOMMENDATION

I have the following recommendations with regard to the DAV DTD in section 24.2 of 2518bis-04.

1.	It is possible to provide a DTD and account for use of the DAV namespace and namespace prefixes.  There will need to be some special way to account for the occurrence of ad hoc elements in the specific cases where elements and attributes from other namespaces may be introduced.  My thoughts on this are incomplete, so it would be necessary to try some things first to make the DTD one that can actually be used and confirmed with a validating processor in the presence of ad hoc elements.  (See note at the end.)

2.	The form of the DTD should be changed to be an external DTD. (The form illustrated in 24.2 and in rfc2518 is of an internal DTD given as part of a <!DOCTYPE ...> declaration).  An external DTD is one that has a URL and can be invoked by reference from a <!DOCTYPE ...> declaration.  At the same time, an internal DTD can also be provided.  The internal DTD is used to provide supplemental information to be used for validating a given XML document instance.  It is even possible to issue a PUBLIC identifier for such an external DTD so that it can be cached by validators without having to always be fetched through the URL (which is taken as a hint).  

3.	The DTD can be parameterized such that documents can use a different prefix (or the default prefix) value for occurrence of DAV namespace elements and attributes.  Since all of the work is in the DTD, and the technique is simple, I suggest that it be applied.  (More about that below).

4.	This recommendation does not alter my suggestion that <!DOCTYPE ...> declarations be forbidden as part of the DAV profile for acceptable XML in DAV request bodies and response bodies.  It is valuable to have a way to validate these elements as part of creation, processing, and trouble-shooting in specific settings, even when one does not want to imply validity or validation as a condition of their occurrence as DAV protocol elements.  The material in 24.2 can be useful for those ancillary purposes as well as for clarifying the DAV reliance on XML.

5.	There are many different XML documents (that is, having different root elements) that arise as DAV protocol elements.  There can continue to be covered a single DTD, since under validation conditions the particular root to use is named in the <!DOCTYPE ...> declaration, one way or another.

B.	PARAMETERIZING A DTD

I would suggest that the same parameterization scheme that is used for the XML Schema Definition DTD.
(See <http://www.w3.org/TR/xmlschema-1/#nonnormative-schemaDTD>.)  The paragraph preceding the schema itself would also apply for rfc2518bis.) The technique (for the DAV namespace only, here) is to define two parameter entities:

	<!ENTITY % dp "dav:">  
		<!-- the prefix that is used on 
		     DAV-namespaced local names -->
	<!ENTITY % ds ":dav">   
		<!-- the suffix that is used on 
		     xmlns attributes that specify
		     the prefix for DAV-namespaced
		     local names -->

(I chose prefix "dav:" arbitrarily.  Any other default, such as "D:" could be used.)

These are internal to the external DTD to be defined.   They are used everywhere in the DTD where DAV
namespace element types and DAV namespace attribute types are declared.  (There might not be any such attribute types -- most attribute types are local to an element type and their names do not require a
prefix.  It is only when an attribute from a different namespace is used that this comes up.)

The entity declarations above establish default parameter values for the parameter entity references %dp and %ds.  These references occur only in the DTD, not in the XML document proper.

If the document to be assessed for validity uses a different prefix, the parameter declarations can be over-ridden.  For example, to use the default namespace for a DAV document, one could assess validity using a validating parser and 

	<?xml version="1.0" encoding="ASCII" ?>
	<!DOCTYPE error SYSTEM "url-of-the-DTD"
		[ <!ENTITY %dp "">
              <!ENTITY %ds ""> 
              ] >
	<error xmlns="DAV:">
		<!-- the rest of an actual DAV error element.  
                 The DTD can check that the correct xmlns
		     is present and also have it not defaulted.
		     -->
		<forbid-external-entities>
	</error>

To use the prefix without overriding, it can work like this:

	<?xml version="1.0" encoding="iso8859-1" ?>
	<!DOCTYPE dav:propfind SYSTEM "url-of-the-DTD" >
	
	<dav:propfind xmlns:dav="DAV:">
		<dav:prop>
			<dav:creationdate/>
			<dav:getlastmodified/>
		</dav:prop>
		<dav:deadprops/>
	</dav:propfind>
 
This case is so simple because only DAV namespace elements are used.  (To a DTD validator, a ":" is just another name character.)

This form of schema parameterization cannot be normative because there will be valid documents that are excluded.  However, every acceptable DAV XML will have a form that is valid under this DTD, and that will be good enough for most situations where some assessment of validity is desired.  It can also support establishment of recommended practice.

C.	DEALING WITH AD HOC ELEMENTS

In order to have an ad hoc element, there needs to be some approach, in the external DTD, that allows for the internal DTD to provide additional element and attribute definitions so that the ad hoc material can be confirmed as valid.  

For example, the section 24.2 external DTD can easily provide definitions for DAV-defined properties.  However, there needs to be a way to define the acceptance of additional property elements under <%dp;prop>.  Suppose for now there is a parameter % op for "other properties" that effectively defaults to an empty set of choices.

The following example, however clumsy, illustrates what it takes to do the job with a valid XML document:

	<?xml version="1.0" encoding="utf-8" ?>
	<!DOCTYPE dav:propertyupdate SYSTEM "url-of-the-DTD"
	   [ <!ENTITY %op "dc:creator">
               <!-- Add dc:creator to acceptable 
		        prop elements -->
	     <!ELEMENT dc:creator (#PCDATA)>
	     <!ATTLIST dc:creator
	               xmlns:dc   CDATA #FIXED 
		                    "http://purl.org/dc/elements/1.1/"
	         <!-- Unfortunately, this will default xmlns:dc 
                    if we don't show it explicitly,
                    but this is the best we can do. -->
              ] >

	<dav:propertyupdate xmlns:dav="DAV:">
	    <dav:set>
	        <dav:prop>
	            <dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">
			Dennis E. Hamilton
			</dc:creator>
			<dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">
			Orcmid, junior nymph of the Orcan Conclave
			</dc:creator>
	        </dav:prop>
          </dav:set>
	</dav:propertyupdate>

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Saturday, October 11, 2003 04:04
To: [email-masked]; [email-masked]
Subject: RE: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)

Comments inline...

[ ... ]

[ ... ]

Regards, Julian

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 14 Oct 2003 06:21:49 -0400

Dennis wrote on 10/12/2003 04:12:15 PM:
> The one problem 
> that I have is that a changed or modified namespace is not construed
> by everyone (and definitely not me) as a "single existing ... 
namespace." 

Yes, there still is the problem of determining what functionality
within that namespace is supported by a given server.  That's what
the DAV header is all about.

> Also, I am talking about namespace versioning, not revision.  Any 
> change to a namespace is a revision, in my book.  A version is a 
> differently-identified derivative, in my book.  Maybe there is 
> better nomenclature for that. I am interested in suggestions for that.

I don't think "revision" is the right way to model what happens to 
the DAV namespace.  In particular, revision commonly implies a 
set of alternatives, of which you can pick one.  The WebDAV
specs do not define a sequence of alternatives, but rather a set
of standard "features", any combination of which can be supported
by a WebDAV server(with some constraints, e.g. WebDAV level "2" support
requires WebDAV level "1" support).  The set of features supported
by a server is indicated in the DAV response header.
 
> The java.lang "namespace" is more-or-less maintained the way you 
> describe.  There are deprecated classes and methods, but they are 
> still there.  There are also new elements that are added to the 
> namespace from time to time. 
> 
> Reliance on new additions creates down-level compatibility problems,
> and it happens in a kind of blissful unawareness.  But if you 
> distribute a class file, it can be found to not run on earlier 
> versions of the JVM and its libraries precisely because an added 
> feature has been relied on without concern for (or probably even any
> awareness of) version dependency.  For example, I am fond of
> 
> and in using it I am producing a program that will not operate with 
> versions of Java prior to JDK 1.2.  If I make use of the 
> 
> When I first started using Java (earlier this year in a course), I 
> was meticulous about determining the earliest version my code would 
> work with, and saying so in prominently placed comments and any 
> documentation.  I have gotten lazy and I don't do that any longer. 
> I should probably build a tool to figure it out for me and then let 
> me decide if I want to make changes to expand the down-level 
> compatibility of whatever I produce for widespread usage. It is 
> probably a hack off of JavaDoc.

This is a problem for Java classes because they do not have the
"supported feature" mechanism that is provided by the WebDAV DAV
header.  WebDAV clients understand that they must deal with servers
that support different combinations of WebDAV features.

One of the most difficult aspects of defining a WebDAV protocol
extension is to invent the minimal number of new features needed
for that extension (preferably just one, but for some sophisticated
extensions such as DeltaV, several are required).

> 2.   My nightmare
> 
> What has me wake up in the middle of the night is any situation that
> I have stepped over where (1) there is any change to the semantics 
> of the same name between two versions of something or (2) there is 
> vocabulary [local names] in one and not the other, and (3) I haven't
> provided for explicit differentiation so that (4) I can be specific 
> in what interface contract I am depending on or providing, and (5) 
> it is then clear what is a bug and what isn't.

In WebDAV:

(1) is not allowed, except in extreme cases where the previous semantics
was so poorly defined that interoperability would improve by changing the
semantics to something closer to what was actually being implemented
(e.g. "lock-null resources" in 2518).

(2) is explicitly allowed, but must be accompanied by a new DAV feature
that allows a client to discover what vocabulary is supported by a given
server.

which addresses (3), (4), and (5).

> 3.1   I do not find it a nightmare to encounter a clearly-versioned 
> use of XML that says namespace-version-2 is acceptable and name-
> space-version-1 is not.  I find that honest and clean.  It makes a 
> clear declaration of where interoperability is not provided. 

The WebDAV approach of defining DAV features provides significantly
superior interoperability.  A client that requires features X, Y, and
Z just checks whether X, Y, Z are supported by a given server.
If they aren't, the client can fail, or can fall back to the subset
of features the server does support.
 
> 3.2   I also see no problem in an XML application saying that 
> namespace-version-2 is preferred and acceptable and that namespace-
> version-1 will be honored and responded to in complete compliance 
> with the rules that apply to namespace-version-1.  That for me is 
> the best of interoperability.  I could even foresee a specification 
> (or an interoperability profile that applies to a specification) 
> where acceptors of namespace-version-2 are required to accept 
> namespace-version-1 and, in so doing, behave as if namespace-
> version-2 has never been heard of. 

That is what the WebDAV DAV feature mechanism gives you.
 
> 3.3   Even better is that there is only namespace-version-1 and any 
> client who uses namespace-version-1 will never be surprised by any 
> response and any acceptor of namespace-version-1 will never be 
> surprised by an unexpected legal use (as the result of some revision
> that has occurred).  That's a wonderful contract, as long as it can 
> be maintained.  Clearly, if DAV accomplishes that, there is nothing 
> I could possibly quarrel with. 

It does, so we have no quarrels (about that, anyway, I'm sure that 
there are some other aspects of the WebDAV protocols that will warrant
a quarrel or two :-).

> 4.   Wrap-up
> 
> I find that the provision for namespace versioning, even if there is
> never more than one version, is a bit different than the care one 
> takes to foster interoperability and then make clear and readily 
> confirmable what the interoperability agreement is in actual 
> situations.  Of course, namespace applies only to the XML part and 
> not to DAV protocol in the context of HTTP, DAV methods and all the 
rest.

The WebDAV feature mechanism deals with all of these issues, i.e. a
"feature" that appears in the DAV header defines a set of node types
in XML, a set of DAV methods, a set of headers, and a set of properties
(in the DAV: namespace).  And it works by requiring a DAV protocol 
extension
to respect (and re-use when appropriate) every DAV feature that has been
defined by a DAV protocol with an earlier IETF RFC number.  This makes
writing WebDAV protocol extensions an interesting challenge, but it sounds
like you appreciate why we would impose this requirement on ourselves.

Cheers,
Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 14 Oct 2003 06:21:57 -0400

> Relax.  My next class starts on October 16 and I will quiet down for

Dennis:

Thanks for spending time on these topics!  It is always useful to 
re-explore
these kinds of issues, since it is likely other folks have similar 
questions
and concerns, and since there are usually at least some points that we 
have
missed (e.g. the question about the suitability of ANY to describe the
desired syntactic behavior of DAV:prop).

Looking forward to hearing again from you in your next break from classes.

Cheers,
Geoff

---

From: "Julian Reschke" <[email-masked]>
Date: Tue, 14 Oct 2003 16:43:29 +0200

Dennis,

I'm not sure what you're trying to achieve.

On the one hand, we're trying to clarify that WebDAV messages MUST not be
validated using DTDs. On the other hand, you're trying to come up with a
clever way to enable DTD validation. As far as I understand, it will never
work for arbitrary now legal WebDAV messages, so why bother?

After all, the initial question was about whether we want to continue using
DTD fragments to specify some constraints on WebDAV messages -- after all,
this is what all published and soon-to-be published WebDAV-related specs do.
So if there's an issue with that notation, we should fix just that, nothing
more... In particular, if "fixing" the DTD notation essentially means to
make it extremely complicated to read, this will be in fact
contra-productive.

> 			<dc:creator
> xmlns:dc="http://purl.org/dc/elements/1.1/">

Nit: that isn't a meaningful request. Each WebDAV property only can have a
single value, so having it twice in propertyupdate will just result in the
"last" value being set.

Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Tue, 14 Oct 2003 12:31:06 -0700

Hi Julian,

I'm surprised once again.  OK, there are several topics here:

1.	Having a generic DTD for WebDAV, or at least one that is generic as possible considering that actual documents can introduce ad hoc elements and attributes in various places.

There was a question about how to do it and sources on the technique.  I have addressed that question.  I have only addressed the question in the context of section 24.2 of bis-04, because I saw immediate application there.  It is also an useful cross-check on the specification and it provides an operable DTD that people can use in working with the specification, creating DAV protocol elements manually, etc., under the understanding that it is non-normative.

2.	Having a way that someone could develop a quasi-generic DTD for a family of WebDAV XML documents of interest.  

I think that comes out of (1).  Since this is a restricted-use situation, a precise DTD might actually be possible for many situations in practice, and I demonstrated a way to have that.  It is also possible in many places where ad hoc properties and attributes are either not provided for or don't happen to be used.

3.	Using XML DTD Fragments (markup declarations) to illustrate/explicate (not specify) the structures that apply for specific elements of WebDAV.  

Since that can't be "normative" as a DTD anywhere that allows ad hoc elements in the content model, or ad hoc attributes of an element, I would think  it is a nit whether you use a convention of supposing a representative (explicit) namespace prefix for QNames in the specification, as many XML-family specifications do, or you show the use of parameters in the specimen markup declarations.  It's the difference between

	<!ELEMENT dav:error ...> 
	<!ATTLIST dav:error 
                xmlns:dav (DAV:) #REQUIRED
	          ... >
and	<!ELEMENT %dp;error ...>
	<!ATTLIST %dp;error 
                xmlns%ds; (DAV:) #REQUIRED
	          ... >

where the second parameterizes the use of namespace prefix, but which might not be so appealing in the flow of the exposition.  Either way, I would not show the <!ATTLIST ...> everywhere.  The assumption of such an <!ATTLIST ...> would be explained in the documentation of fragment usage in the specification.

I am not addressing the question of DTD fragments with my proposal for section 24.2.  One can do 24.2 independent of whether markup declarations are used elsewhere in the specification, and whether they are of the same literal form.  

4.	Concerning the question of whether or not Document Type Declarations (<!DOCTYPE ...>) should be allowed in the XML document prolog of DAV request and response bodies, I see my recommendation for 24.2 as orthogonal to that.  

Also, I don't believe there is agreement to disallow Document Type Declarations at this time.  (I will keep repeating that presence of a Document Type Declaration and using a non-validating processor are separate but interacting matters.)

It is valuable to have a DTD that is crafted in such a way that it can be used to establish validity of a document in proper form (e.g., no defaulted attribute values for required attributes) for use with or without validation, with or without a Document Type Declaration.  I think one can come close to that as a practical matter.  I find it valuable.

I find interactions with the requirement for use of a non-validating XML processor to be problematic.  A way to assure a predictable, interoperable non-validating process case is to omit Document Type Declarations in requests (as a practice, even if not a requirement of 2518).  There is absolutely no doubt about validation in that case. So omission of a Document Type Declaration is a good practice for obtaining clearly-predictable behavior in DAV requests (and appropriate safety, regarding any concern for reference to external parameters and external XML entities as identified in section 18.6 of bis-04).  

Dealing with a server that provides Document Type Declarations in responses is something I don't want to think about.  In practice, that borders on a hostile act, since it imposes processing on the client and has the interpretation of the response be unpredictable in some cases.  It raises the bar for universal client processors.

5.	Why do I consider (4) and some of the other items when we are "only" talking about HTTP extensions and what happens in that narrow band of what the DAV functioning is all about -- the exchange of protocol elements?  

I answer with a question: Why is it important to use XML in that narrow setting? I am considering the setting and the desire people have to be able to understand the rules, apply tools that are available and supported in the industry, and also be able to check their work (or the work of their software) as well as trouble-shoot.  That strikes me as valuable.  

6.	Is it worth it?  I don't know.  I demonstrated the degree to which it could be done.  Developers of other XML-family specifications have found it worthwhile.  Does it hurt?  How can it?
Would I do it?  Yes.

This can clearly be done.  Whether it is included in the specification or not seems to be the question.  We need some other countries to be heard from.

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Tuesday, October 14, 2003 07:43
To: [email-masked]; [email-masked]
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

Dennis,

[ ... ]

Julian

---

From: "Stanley Guan" <[email-masked]>
Date: Tue, 14 Oct 2003 13:24:05 -0700

Hi,

I'm new on this mailing list.  So, forgive me if my questions were
brought up before.

For security consideration, external XML entities are considered
vulnerable to denial of service attack. So, I agree that WebDAV
messages MUST not be validated using DTDs.  Or it can be
optional, if an implementation opt to do that.

Anyone else have ever thought of using XML Schema, instead of
DTD, to validate WebDAV messages?  Any security concerns?

Will appreciate your inputs!

-Stanley

----- Original Message -----
From: "Dennis E. Hamilton" <[email-masked]>
To: "Julian Reschke" <[email-masked]>; <[email-masked]>
Sent: Tuesday, October 14, 2003 12:31 PM
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

Hi Julian,

1. Having a generic DTD for WebDAV, or at least one that is generic as
possible considering that actual documents can introduce ad hoc elements and
attributes in various places.

There was a question about how to do it and sources on the technique.  I
have addressed that question.  I have only addressed the question in the
context of section 24.2 of bis-04, because I saw immediate application
there.  It is also an useful cross-check on the specification and it
provides an operable DTD that people can use in working with the
specification, creating DAV protocol elements manually, etc., under the
understanding that it is non-normative.

2. Having a way that someone could develop a quasi-generic DTD for a family
of WebDAV XML documents of interest.

I think that comes out of (1).  Since this is a restricted-use situation, a
precise DTD might actually be possible for many situations in practice, and
I demonstrated a way to have that.  It is also possible in many places where
ad hoc properties and attributes are either not provided for or don't happen
to be used.

3. Using XML DTD Fragments (markup declarations) to illustrate/explicate
(not specify) the structures that apply for specific elements of WebDAV.

Since that can't be "normative" as a DTD anywhere that allows ad hoc
elements in the content model, or ad hoc attributes of an element, I would
think  it is a nit whether you use a convention of supposing a
representative (explicit) namespace prefix for QNames in the specification,
as many XML-family specifications do, or you show the use of parameters in
the specimen markup declarations.  It's the difference between

<!ELEMENT dav:error ...>
<!ATTLIST dav:error

where the second parameterizes the use of namespace prefix, but which might
not be so appealing in the flow of the exposition.  Either way, I would not
show the <!ATTLIST ...> everywhere.  The assumption of such an <!ATTLIST
...> would be explained in the documentation of fragment usage in the
specification.

I am not addressing the question of DTD fragments with my proposal for
section 24.2.  One can do 24.2 independent of whether markup declarations
are used elsewhere in the specification, and whether they are of the same
literal form.

4. Concerning the question of whether or not Document Type Declarations
(<!DOCTYPE ...>) should be allowed in the XML document prolog of DAV request
and response bodies, I see my recommendation for 24.2 as orthogonal to that.

Also, I don't believe there is agreement to disallow Document Type
Declarations at this time.  (I will keep repeating that presence of a
Document Type Declaration and using a non-validating processor are separate
but interacting matters.)

It is valuable to have a DTD that is crafted in such a way that it can be
used to establish validity of a document in proper form (e.g., no defaulted
attribute values for required attributes) for use with or without
validation, with or without a Document Type Declaration.  I think one can
come close to that as a practical matter.  I find it valuable.

I find interactions with the requirement for use of a non-validating XML
processor to be problematic.  A way to assure a predictable, interoperable
non-validating process case is to omit Document Type Declarations in
requests (as a practice, even if not a requirement of 2518).  There is
absolutely no doubt about validation in that case. So omission of a Document
Type Declaration is a good practice for obtaining clearly-predictable
behavior in DAV requests (and appropriate safety, regarding any concern for
reference to external parameters and external XML entities as identified in
section 18.6 of bis-04).

Dealing with a server that provides Document Type Declarations in responses
is something I don't want to think about.  In practice, that borders on a
hostile act, since it imposes processing on the client and has the
interpretation of the response be unpredictable in some cases.  It raises
the bar for universal client processors.

5. Why do I consider (4) and some of the other items when we are "only"
talking about HTTP extensions and what happens in that narrow band of what
the DAV functioning is all about -- the exchange of protocol elements?

I answer with a question: Why is it important to use XML in that narrow
setting? I am considering the setting and the desire people have to be able
to understand the rules, apply tools that are available and supported in the
industry, and also be able to check their work (or the work of their
software) as well as trouble-shoot.  That strikes me as valuable.

6. Is it worth it?  I don't know.  I demonstrated the degree to which it
could be done.  Developers of other XML-family specifications have found it
worthwhile.  Does it hurt?  How can it?
Would I do it?  Yes.

This can clearly be done.  Whether it is included in the specification or
not seems to be the question.  We need some other countries to be heard
from.

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Tuesday, October 14, 2003 07:43
To: [email-masked]; [email-masked]
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

Dennis,

[ ... ]

Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 15 Oct 2003 10:42:47 +0200

> [mailto:[email-masked]]On Behalf Of Stanley Guan
> Sent: Tuesday, October 14, 2003 10:24 PM
> To: [email-masked]
> Subject: Re: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)
>
>
>
> Hi,
>

Sure.

> optional, if an implementation opt to do that.

I think we need to be precise here. As far as I understand, the XML
recommendation does only define one very specific form of validation, and it
is based on the document/message declaring it's document type.

Exactly this kind of validation is completely useless in XML based
protocols: it's completely irrelevant whether a document conforms to a DTD
that the *sender* provides. It would be only interesting to validate against
the DTD expected by the *recipient*. Doing the latter of course is
completely up to the recipient -- however it must be aware of the fact that
the DTD (fragments) in RFC2518 and related specs only describe part of the
constraints, and that a recipient MUST accept way more message variations as
the DTDs (per XML rec) allow.

> DTD, to validate WebDAV messages?  Any security concerns?

If the schema or the reference to the schema is provided by the sender of
the message, I think the same concerns apply. If the schema is hardwired
into the recipient, none apply.

On the other hand, I don't see any big advantage in using XML Schema as
replacement in WebDAV specs. It only solves one particular problem (DTDs
ignorance of namespaces), but is a lot harder to read. If we really decide
not to use DTD syntax anymore, we should consider a schema language that can
*really* express the DAV extensibility rules, and that's easy to read by the
(human) readers of the spec. As far as I understand, Relax NG (compact
syntax) would qualify.

You may also want to check out RFC3470 ("Guidelines for the Use of
Extensible Markup Language (XML) within IETF Protocols", section 4.7.

Regards, Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 15 Oct 2003 12:06:28 +0200

Dennis,

I appreciate the time you're spending thinking about this subject. The
question that started this thread was what to do with the DTD notation that
is used in current WebDAV specs and that disappeared in RFC2518bis. My
position is that we should keep it and make sure that the spec clearly says
what it means for developers.

Other comments inline...

> 4.	Concerning the question of whether or not Document Type
> Declarations (<!DOCTYPE ...>) should be allowed in the XML
> document prolog of DAV request and response bodies, I see my
> recommendation for 24.2 as orthogonal to that.
>
> Also, I don't believe there is agreement to disallow Document
> Type Declarations at this time.  (I will keep repeating that
> presence of a Document Type Declaration and using a
> non-validating processor are separate but interacting matters.)

Yes. They haven't been forbidden before, and they don't seem to cause harm.

> Dealing with a server that provides Document Type Declarations in
> responses is something I don't want to think about.  In practice,
> that borders on a hostile act, since it imposes processing on the
> client and has the interpretation of the response be
> unpredictable in some cases.  It raises the bar for universal
> client processors.

Nope. The spec should clearly state that messages must be processed in
non-validating mode. In this case, it's really irrelevant whether a DOCTYPE
is present (except for the well-known recursive entity substition problem).

> 6.	Is it worth it?  I don't know.  I demonstrated the degree
> to which it could be done.  Developers of other XML-family
> specifications have found it worthwhile.  Does it hurt?  How can it?
> Would I do it?  Yes.
>
> This can clearly be done.  Whether it is included in the
> specification or not seems to be the question.  We need some
> other countries to be heard from.

I don't argue with automatic "checking" (avoiding "validation" because it
implies DTD to many readers) of messages based on a formal description being
useful. However, we were discussing the role of DTD fragments in the spec
*as they are*. My take is that it's best to

- first get a consensus about what they are supposed to mean - I've been
repeating my proposal many times now (before *this* discussion started), and
I'd really like to see whether we finally agree on that

- then find out whether we can keep that syntax - I think it's worthwhole
keeping it for the simple reason that it's present on published specs

If we keep the syntax *and* agree about what it means, it will be simple to
specify a process by which an arbitrary WebDAV message can be transformed
into a format that *can* be validated.

Julian

---

From: "Stanley Guan" <[email-masked]>
Date: Wed, 15 Oct 2003 09:51:35 -0700

Julian,

Thank you for your comments.

Personally I'm working on the implementation of XML Schema. So, I'm
talking more from the XML Schema perspective.  Sorry for the bias. See
my response below.

-Stanley

----- Original Message -----
From: "Julian Reschke" <[email-masked]>
To: "Stanley Guan" <[email-masked]>; <[email-masked]>
Sent: Wednesday, October 15, 2003 1:42 AM
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

> the DTDs (per XML rec) allow.

Yes, it matters only on how recipients use the DTD, or possible other
schemas,
to validate the message.  RFC 2518 didn't dictate whether the recipient
should
use DTD, XML Schema, or Relax NG to validate the XML message.  Right?

> into the recipient, none apply.

If the schema is provided by the sender, say using SchemaLocation, it should
be ignored by the recipient from the same security consideration.  XML
Schema
spec says SchemaLocation only provides a hint, an implementation can
rightfully ignore the information provided by the sender.

In this case, I was thinking of hardwiring the schema to the recipient.

> (human) readers of the spec. As far as I understand, Relax NG (compact

I'm not a big fan of XML Schema either.  But, I think XML Schema WG is
trying hard to correct some of the problems in its original design.
However,
managing namespaces is a big concern.  Current approach for new extensions
is just extending DAV: namespace.  This introduces a versioning control
issue.

Currently, DAV extensions are using XML structures in a limited way.
To handle these structures, I think, XML Schema can provide good
support for its constraint specification and address extensibility by using
its "extension" or similar mechanisms.  What I'm trying to say here is:
TRUE, the whole XML Schema spec. is hard to read; but, if you carefully
enough to use a subset of its features, it's still a good tool for message
validation.

Lastly, XML Schema has been widely supported by most software
vendors.

> Extensible Markup Language (XML) within IETF Protocols", section 4.7.

Sure!

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 15 Oct 2003 19:13:55 +0200

> Sent: Wednesday, October 15, 2003 6:52 PM
> To: [email-masked]

Ok :-)

schemas,
should
> use DTD, XML Schema, or Relax NG to validate the XML message.  Right?

Right. And it doesn't need to. The recipient can of course validate the
message with any technology it feels adequate. To be adequate, it must
somehow be able to express the special WebDAV extensibility rules.

Schema
> rightfully ignore the information provided by the sender.

Correct.

> In this case, I was thinking of hardwiring the schema to the recipient.

That's fine.

However,
> is just extending DAV: namespace.  This introduces a versioning control

Such as...? Fact is, one of the most successful XML applications -- XSLT --
uses *exactly* the same extensibility mechanism as WebDAV. If a schema
language isn't able to express this kind of extensibility mechanism, that's
unfortunate but it doesn't necessarily mean that the mechanism itself is
flawed.

I don't want to change WebDAV. In fact, we can't change WebDAV unless we go
back to "proposed standard", and it's very unlikely that at this point of
time, we'd get big players such as Microsoft to update their code.

So what this issue is about is to *keep* the current extensibility rules
while considering a better (hopefully formal) way to express them in the
spec. DTDs can't do that well, nor can XML Schema. Relax NG may be able to
do it.

> enough to use a subset of its features, it's still a good tool for message

OK, so *how* do you express the following rules in XML Schema?

- in addition to the specified child elements, *any* other element is
allowed (at any position!), unless it's specified in RFC2518,

- arbitrary properties are allowed

- ordering is irrelevant

As far as I understand, XML Schema can't do that (as the extensibility rules
explicitly allow new child elements from the DAV: namespace).

> Lastly, XML Schema has been widely supported by most software

I don't want to see another xml-dev permathread migrating here. We just need
the schema language for the *formal notation*. It's completelely irrelevant
here whether it is or will be implemented.

What *is* relevant is whether we can find a notation that is better suited
for writing the spec. I seriously doubt that XML Schema can be a candidate
(if only for it's verboseness).

Regards, Julian

---

From: "Stanley Guan" <[email-masked]>
Date: Wed, 15 Oct 2003 11:04:46 -0700

Julian,

If you're just looking for a schema language for the *formal notation*,
then any choice will be fine with me :-).

> allowed (at any position!), unless it's specified in RFC2518,

I think, what you need here is
   <xs:any namespace="##other" processContents="lax"/>
specified in a "choice" component.

A question here: if the server claims that it only supports "versioning",
but not "acl", is it an error if the user sends  a request to find the
"DAV:acl" property?

> - arbitrary properties are allowed

I think, you can collect all server supported DAV: properties in
a single complexType using "choice" component.  Then, using
"extension" component to extend different capabilities into the
final set.

> - ordering is irrelevant

"choice" is the best candidate.  "all" is somewhat limited.

Best,

Stanley

---

From: "Julian Reschke" <[email-masked]>
Date: Wed, 15 Oct 2003 20:32:48 +0200

> Sent: Wednesday, October 15, 2003 8:05 PM
> To: [email-masked]

Last time was dicussed I was told that this will not allow new extension
elements from the DAV: namespace.

> but not "acl", is it an error if the user sends  a request to find the

No.

> "extension" component to extend different capabilities into the

Sorry. I wanted to say "attributes".

> "choice" is the best candidate.  "all" is somewhat limited.

Thanks,

Julian

---

From: "Stanley Guan" <[email-masked]>
Date: Wed, 15 Oct 2003 13:41:14 -0700

Julian,

See my comments inline!

Thx,

-Stanley

----- Original Message -----
From: "Julian Reschke" <[email-masked]>
To: "Stanley Guan" <[email-masked]>; <[email-masked]>
Sent: Wednesday, October 15, 2003 11:32 AM
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

> elements from the DAV: namespace.

True.  But, new DAV extension elements should be explicitly listed in
the "choice" component.  So, any bogus element in DAV: namespace
can be caught.

> Sorry. I wanted to say "attributes".

I thought we want to be loose on what can be allowed at element
level.  Within each element, don't we want all attributes to be
explicitly spelt out?  Why do we need arbitrary attributes to be
allowed on any specific DAV: element?

If elements belong to the following category:
   <xs:any namespace="##other" processContents="lax"/>,
then any arbitrary attributes can be allowed in them.

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Wed, 15 Oct 2003 14:50:41 -0700

Hi Julian,

1.	Yes, I understand that this was spun off of a different question.  I came around to the 24.2 proposal because I started looking at bis-04 to get up to speed on what new wordings there might be on DTD and XML-related topics.  

1.1	I propose to keep the proposal of DTD approach for 24.2 separate from the one about in-line use of markup declarations in the 2518bis's descriptions of the DAV use of XML.

1.2	Probably the thing to do is to build such a DTD, play with it, and see if it is worth including in 24.2 (and also at some well-known URL where it will stay).  [Is the W3C support of WebDAV such that we could get space on the W3C server, or would ietf.org be more appropriate?  Or webdav.org?]

	-	-	-	-	-	-	-	-	-	

2.	I do want to point out an over-simplification that keeps creeping into this discussion.   

2.1	<!DOCTYPE>-required processing.  

When a <!DOCTYPE ...> declaration is present, even a non-validating processor must do a lot of work.  It is definitely not the same as just parsing through the declaration and going on one's merry way.  There is the small matter of entity declarations and whether or not the receiving processor will read external sources for parameter entities and external XML entities (it being the receiver's option to do so).
  
And then there are attributes to be concerned about.  
	
Also, the standalone attribute of the XML declaration can influence the non-validating behavior, although it doesn't reduce the amount of work that must be done.  

There are some XML 1.0 errata that pertain to these discussions and to WebDAV use of XML, <http://www.w3.org/XML/xml-V10-2e-errata>: E61, E60, E58, E57, E55, E49, E48, E47, E45, E43, E42, E41, E39, E38, E36, E29, E25, E23, E22, E21, E20, E19, E16, E15, E14, E12, E8, and E7.  I have allowed for these in my comments.  A few pertain to non-validating behavior, attribute processing, etc.  There are also a number of others regarding Unicode and the correct specifications to use that I have omitted from the above list.

2.2	Entity References and Recursion.  

The issue of recursive expansion of entity references is a red herring. Recursion, directly or indirectly, in processing a parsed entity reference is a violation of *well-formedness* and it is a bug for a processor not to reject its occurrence (since all ill-formed XML must be rejected by an XML processor).  This is clear in [REC-XML] section 4.1 Character and Entity References, <http://www.w3.org/TR/REC-xml#sec-references>.  Look at the syntax on Entity References and the notes entitled [WFC: No Recursion] for the two cases.  (A non-validating processor can fail to notice a case of recursion that a validating processor might detect, but that's only if the recursion depends on an external entity that is not read by the non-validating processor.)

There are some technical nuances around when an entity reference is a parsed entity reference, but that doesn't seem to figure here.  The point is that all cases of recursive expansion are prohibited.

It is still trivially possible to send a short XML document with internal DTD that results in a gigantic expansion, as noted in the discussion of security considerations for XML Media types (<http://www.ietf.org/rfc/rfc3023.txt>).  Extend this pattern to, say, cx32, and open the document in Internet Explorer:

	<?xml version="1.0" encoding="utf-8" standalone="yes"?>
	<!-- XML-bloat-test.xml demonstrates exponential 
	     expansion of internal entities -->
	<!DOCTYPE doc 
             [ <!ELEMENT doc (#PCDATA)>
 	         <!ENTITY  c10 "0123456789 ">
               <!ENTITY  cx2 "&c10;&c10;">
               <!ENTITY  cx3 "&cx2;&cx2;">
               <!ENTITY  cx4 "&cx3;&cx3;">
               <!ENTITY  cx5 "&cx4;&cx4;">
               <!ENTITY  cx6 "&cx5;&cx5;">
               <!ENTITY  cx7 "&cx6;&cx6;">            
                ] >                
	<doc>
	&cx7;
	</doc> 

	<!--      ** END OF XML-bloat-text.xml **      -->   

2.3	Attribute Declarations and Default Values.

Any attribute declaration that is read by a non-validating processor must be honored.  This applies to normalization of attribute values based on their type.  If the attribute has a default value, the processor must supply that default value when the attribute is not explicitly present in the start tag of the element for which it is defined.

Although it is not considered good practice to rely on default values and proper normalization for attributes whose declarations might *not* be read by a non-validating processor, any attribute declarations in an internal DTD must be read.

The idea is that an XML processor delivers a processed stream of elements as if every defaulted attribute had been explicitly supplied, every recognized attribute is normalized as appropriate.  All parsed entities have their spacing rules applied, with internal entity references resolved.  The DTD is usually not delivered to the application.  The XML processor makes it unnecessary for the application to have to deal with the DTD under normal conditions.

You can create an XML document with declared attributes and default values and observe this expansion by opening the XML document in Internet Explorer 6.0.  (There's an inconsistency in some defaultings of xmlns attributes, but other cases work.)

[I am omitting all nuances related to what are called Notations, and how attributes of type ENTITY and ENTITIES work for non-validating processors.  That's not less to deal with.]

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Wednesday, October 15, 2003 03:06
To: [email-masked]; [email-masked]
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

Dennis,

Other comments inline...

[ ... ]

[ ... ]

Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Wed, 15 Oct 2003 16:01:24 -0700

Hi Julian and Stanley,

I have been watching your discussion with interest.  I concur that XML Schema should work just nifty for DAV, and the one case of concern to me can be handled, with a little care.

Let me summarize:

1.	XML Schema is able to assess schema validity for documents that have arbitrary elements (and attributes) from other namespaces.  I see there is an open question on the occurrence of arbitrary attributes on DAV elements and that needs to be looked at separately (e.g., how to use attributes from the global attribute namespace versus ad hoc attributes piled onto the DAV element and the element-local namespace -- and whether the second should be ever allowed).

2.	I agree that XML Schema is not an useful way to specify DAV elements and their content models in the body of the specification. However, having and using XML Schemas for DAV XML documents is still an useful option.

3.	The case of concern for me is that XML Schema assessment can be done without doing anything to the document, as Stanley noticed.  

3.1	There is no requirement to provide an xsi:schemaLocation list, and there is also no requirement that the XML processor use it.  It's all hinting, as I recall.

3.2	However, XML Schemas are targeted to namespaces (which is how the whole document can be assessed for schema conformity even when the document is drawn from many namespaces).  Since DAV does not use immutable namespaces, but uses an extensible namespace with a single namespace URI (DAV:), this means the schema definition has to be updated when new official elements are added.  

3.3	The only wrinkle here is that XML processors that do schema assessment tend to cache the XML Schema definitions by namespace URI.  So when the DAV namespace is given new local names, any processor prepared to accept those and provide appropriate schema assessment must update its cache with the new schema definition.  (This is another reason that namespace versioning is useful, but ... that's life [;<).  [I think my schema/DTD-validating XML editor requires me to flush the whole cache to do that, and that's the price I pay for being so picky.]

The datatype model of XML Schema seems to be gaining popularity for establishing and expressing the (lexical) datatype of simple-element content too.  So there is already a source of data-type tags (and new schema-defined ones) that can be supplied as attributes of property elements conveyed in DAV <prop> listings [;<).

-- Dennis

-----Original Message-----
Sent: Wednesday, October 15, 2003 13:41

Julian,

See my comments inline!

Thx,

-Stanley

----- Original Message -----
From: "Julian Reschke" <[email-masked]>
To: "Stanley Guan" <[email-masked]>; <[email-masked]>
Sent: Wednesday, October 15, 2003 11:32 AM
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

[ ... ]

[ ... ]

[ ... ]

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 16 Oct 2003 09:28:35 +0200

> Sent: Wednesday, October 15, 2003 10:41 PM

How does that help? If a recipient validates a message based on a
RFC2518bis-based XML Schema, and draft-ietf-webdav-redirect-ref protocol
extends a particular element, these messages will not be valid according to
the Schema *without* modifying the schema. The point of extensibility is
that old components continue to work with extended messages *without*
modification.

> allowed on any specific DAV: element?

Because that's allowed now.

Regards, Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 16 Oct 2003 09:31:55 +0200

Dennis,

I think you're still missing a basic fact about WebDAV's extensibility: it
is not centralized, nor linear. There is no common registry. Everybody can
add extension elements anytime. Even if somebody would update that Schema
anytime a new RFC (or Internet Draft) comes out out, there will be a lot of
legal WebDAV messages that won't validate against that schema.

Julian

> Sent: Thursday, October 16, 2003 1:01 AM
> To: Stanley Guan; Julian Reschke; [email-masked]
> Subject: RE: DAV Schema Assessment (was Re: rfc2518bis DAV DTD ...)
>
>
>
> Hi Julian and Stanley,
>
> I have been watching your discussion with interest.  I concur
> that XML Schema should work just nifty for DAV, and the one case
> of concern to me can be handled, with a little care.
>
> Let me summarize:
>
> 1.	XML Schema is able to assess schema validity for documents
> that have arbitrary elements (and attributes) from other
> namespaces.  I see there is an open question on the occurrence of
> arbitrary attributes on DAV elements and that needs to be looked
> at separately (e.g., how to use attributes from the global
> attribute namespace versus ad hoc attributes piled onto the DAV
> element and the element-local namespace -- and whether the second
> should be ever allowed).
>
> 2.	I agree that XML Schema is not an useful way to specify DAV
> elements and their content models in the body of the
> specification. However, having and using XML Schemas for DAV XML
> documents is still an useful option.
>
> 3.	The case of concern for me is that XML Schema assessment
> can be done without doing anything to the document, as Stanley noticed.
>
> 3.1	There is no requirement to provide an xsi:schemaLocation
> list, and there is also no requirement that the XML processor use
> it.  It's all hinting, as I recall.
>
> 3.2	However, XML Schemas are targeted to namespaces (which is
> how the whole document can be assessed for schema conformity even
> when the document is drawn from many namespaces).  Since DAV does
> not use immutable namespaces, but uses an extensible namespace
> with a single namespace URI (DAV:), this means the schema
> definition has to be updated when new official elements are added.
>
> 3.3	The only wrinkle here is that XML processors that do schema
> assessment tend to cache the XML Schema definitions by namespace
> URI.  So when the DAV namespace is given new local names, any
> processor prepared to accept those and provide appropriate schema
> assessment must update its cache with the new schema definition.
> (This is another reason that namespace versioning is useful, but
> ... that's life [;<).  [I think my schema/DTD-validating XML
> editor requires me to flush the whole cache to do that, and
> that's the price I pay for being so picky.]
>
> The datatype model of XML Schema seems to be gaining popularity
> for establishing and expressing the (lexical) datatype of
> simple-element content too.  So there is already a source of
> data-type tags (and new schema-defined ones) that can be supplied
> as attributes of property elements conveyed in DAV <prop> listings [;<).
>
> -- Dennis
>

---

From: "Stanley Guan" <[email-masked]>
Date: Thu, 16 Oct 2003 09:47:07 -0700

Julian,

I am getting more clear on what the true issues are.  Thanks!

However, it seems to me that there are different options to resolve the
extensibility issue and WG seems to choose the following approach:

   1. For client implementations, ignore XML elements they do not
       understand.

       "Older clients will not break when they encounter extensions
       because they will still have the data specified in the original
       schema and will ignore elements they do not understand."

   2. For server implementations, ignore any unknown  XML
       element and all its children encountered.

       "All DAV compliant resources MUST ignore any unknown
        XML element and all its children encountered while processing
        a DAV method that uses XML as its command language."

       As told by you, this rule will be extended to include any unknown
       XML attribute.  Right?

To summarize what I understand so far:

    1. WG is seeking a formal notation to describe the XML components
        contained in any message body that need to be minimally understood
        by all DAV-compliant (including DAV's extensions) implementations.

        Any bogus (or should be called "alien") XML elements (or attributes)
        will be simply ignored without even raising a flag.  So, to avoid
        hackers using this feature to launch denial-of-access attacks is to
limit
        the size of XML data allowed in the request body.

        Additionally, there is no need for any implementation to use any
schema
        to check whether received XML data is valid or not.   What it needs
        to do is just walking through the XML elements and check if it is
what
        the implementation can understand or not.   If yes, take action;
        otherwise, ignore it.

    2. The DAV response header (and new proposed DAV request
        header is just informational and has no constraining power.

Am I right?

Thx,

-Stanley

----- Original Message -----
From: "Julian Reschke" <[email-masked]>
To: "Stanley Guan" <[email-masked]>; "Julian Reschke"
<[email-masked]>; <[email-masked]>
Sent: Thursday, October 16, 2003 12:28 AM
Subject: RE: rfc2518bis DAV DTD (was Re: How to use DTDs, or not ...)

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 16 Oct 2003 21:50:53 +0200

> Sent: Thursday, October 16, 2003 6:47 PM
> Subject: Appropriate XML processing in extensibility consideration (Was
> rfc2518bis DAV DTD)
>
>
>
> Julian,
>

Right.

>        As told by you, this rule will be extended to include any unknown

Actually, that's not obvious. RFC2518 so far hasn't used attributes at all
(except for some wording about xml:lang's role). But I'd assume that yes,
extensibility applies to attributes as well.

>         hackers using this feature to launch denial-of-access attacks is
to limit
>         the size of XML data allowed in the request body.

Right.

Side note: it would be interesting to explore a mechanism for mandatory
extensions (I think RFC2774 can help here).

schema
what
>         the implementation can understand or not.   If yes, take action;

Yes. However, "understanding" is a bit vague. For instance, PROPFIND uses:

<!ELEMENT propfind (allprop | propname | prop) >

So for instance servers SHOULD reject requests such as

<propfind xmlns="DAV:"><prop>...</prop><prop>...</prop></propfind>

>         header is just informational and has no constraining power.

I wouldn't call it "just informational", but it doesn't affect the validaty
of a message body.

Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Thu, 16 Oct 2003 13:02:06 -0700

Julian,

With regard to schema-definition maintenance, I was referring only to extension to the DAV: namespace, not the occurrence of ad hoc elements from other namespaces.  

With regard to schema assessment, I believe XML Schema has ways to allow for occurrence of elements that are from namespaces other than the one covered by a particular schema definition, if you can delimit where such occurrences are allowed.  (I have not checked this as carefully as Stanley may have, however, and the response from Henry Thompson is not clear to me on that point.  I was relying on Stanley's illustration and I did not go check the XML Namespace specs myself, nor review how lax assessment is specified.)  

I think I understand both how the DAV namespace is extensible (by agreement) and how DAV XML elements are extensible (by ad hoc additions having nothing to do with the DAV namespace).

The only case that a reference XML schema definition for DAV would need to be maintained is with regard to the first part.  The second case is up to the people who define/use ad hoc additions. [Because there is no 100%, I see no reason to settle for 0%.  It's not a binary question for me.]

Does that fit your understanding of DAV extensibility?  

If you're telling me that anyone can invent an element and DAV-validly introduce it as if under the DAV namespace, I will quietly pick up my marbles and go figure out how to use SOAP (with attachments) and the Web Services stack on HTTP to do Document Authoring and Versioning on the web. (Such an expression of some level of DAV functionality is probably valuable anyway.) 

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Thursday, October 16, 2003 00:32
To: [email-masked]; Stanley Guan; Julian Reschke;
[email-masked]

Dennis,

Julian

--

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 16 Oct 2003 22:18:52 +0200

> Sent: Thursday, October 16, 2003 10:02 PM
> To: Julian Reschke; Stanley Guan; [email-masked]
> With regard to schema-definition maintenance, I was referring
> only to extension to the DAV: namespace, not the occurrence of ad
> hoc elements from other namespaces.

OK,

yes, you can define a Schema that knows about all presently defined DAV
elements. If you combine that with lax validation for non-DAV elements, and
also take care of all other issues (such as ordering), you may be able to
use that for validation. However, this validation immediately breaks once a
new DAV protocol element is added. Thus, you can't use that Schema to
validate messages in a production system that is meant to accept messages
with not-yet defined extensions.

> ...
>
> The only case that a reference XML schema definition for DAV
> would need to be maintained is with regard to the first part.
> The second case is up to the people who define/use ad hoc
> additions. [Because there is no 100%, I see no reason to settle
> for 0%.  It's not a binary question for me.]

Well, the validation process MUST accept all messages that are legal
according to the spec. If it doesn't, what's the point in using it (unless
just for debugging/education reasons)?

> If you're telling me that anyone can invent an element and
> DAV-validly introduce it as if under the DAV namespace, I will

The WG says it's not allowed, but many companies do it. That's a fact we
can't change in practice.

> quietly pick up my marbles and go figure out how to use SOAP
> (with attachments) and the Web Services stack on HTTP to do
> Document Authoring and Versioning on the web. (Such an expression
> of some level of DAV functionality is probably valuable anyway.)

I'm not sure I follow. The way WebDAV is extensible IMHO doesn't cause any
actual problems. Please be more specific. And before promoting SOAP in a
HTTP-based WG, please make sure that you've read all related
HTTP-with-extensions-vs-SOAP propaganda :-)

Julian

---

From: "Stanley Guan" <[email-masked]>
Date: Thu, 16 Oct 2003 13:57:52 -0700

Hi,

Is there any delta history of the rfc2518bis?  Is it possible to
provide something similar to:
    http://www.w3.org/2001/05/xmlschema-errata.html

regarding editorial efforts to highlight what's new and what's
different from the original rfc2518 in rfc218bis?

Thx,

-Stanley

---

From: Geoffrey M Clemm <[email-masked]>
Date: Thu, 16 Oct 2003 17:38:47 -0400

Julian wrote on 10/16/2003 04:18:52 PM:

> > DAV-validly introduce it as if under the DAV namespace, 
> 
> > I will quietly pick up my marbles and go figure out how to use SOAP

In particular, SOAP is like XML ... it's easy to marshall things with it 
because
it places no semantic constraints on what you marshall ... but the result
of no semantic constraints is no semantic interoperability.

Cheers,
Geoff

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 16 Oct 2003 23:50:23 +0200

OK,

I just spend a few minutes with Jing (Relax NG validator) and Trang (Schema
Language convertor) [1]. Some initial thoughts:

- RelaxNG can express WebDAV's extensibility model
- RelaxNG has both an XML based syntax (à la XML Schema) and a Compact
syntax (à la DTD)
- Trang supports to/from DTD, XML Schema, RelaxG (compact/XML) and (!)
sample XML instance data
- RelaxNG supports the XML schema datatypes

Here's an example for dav:propfind in RNC (Relax NG Compact Syntax):

namespace dav = "DAV:"

start = propfind

allprop = element dav:allprop { empty }

prop =
  element dav:prop {
    element * { empty }+
  }
propname = element dav:propname { empty }
propfind = element dav:propfind { (allprop | prop | propname) & EXT? }

EXT =
  element * - (dav:allprop
               | dav:href
               | dav:multistatus
               | dav:prop
               | dav:propfind
               | dav:propname
               | dav:response
               | dav:status) { empty }

...you get the idea.

Should we pursue this?

Julian

[1] <http://www.relaxng.org/>

---

From: "Eric Sedlar" <[email-masked]>
Date: Thu, 16 Oct 2003 15:33:36 -0700

Dennis--

  The issue is not that "anyone can invent an element", but that new DAV
specs will come out, and would break older clients not updated with the new
schema.  The issue is that XML Schema would constrain the DAV WG's ability
to extend the specification.

--Eric

----- Original Message ----- 
From: "Dennis E. Hamilton" <[email-masked]>
To: "Julian Reschke" <[email-masked]>; "Stanley Guan"
<[email-masked]>; <[email-masked]>
Sent: Thursday, October 16, 2003 1:02 PM
Subject: RE: DAV Schema Assessment (was Re: rfc2518bis DAV DTD ...)

[snip]

If you're telling me that anyone can invent an element and DAV-validly
introduce it as if under the DAV namespace, I will quietly pick up my
marbles and go figure out how to use SOAP (with attachments) and the Web
Services stack on HTTP to do Document Authoring and Versioning on the web.
(Such an expression of some level of DAV functionality is probably valuable
anyway.)

-- Dennis

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Thursday, October 16, 2003 00:32
To: [email-masked]; Stanley Guan; Julian Reschke;
[email-masked]

Dennis,

Julian

--

---

From: "Eric Sedlar" <[email-masked]>
Date: Thu, 16 Oct 2003 15:38:47 -0700

There's not much interest in Relax NG in actual vendor products.  Unless
people are actually going to use validators, I don't think we should push
Relax NG on people.

--Eric

----- Original Message ----- 
From: "Julian Reschke" <[email-masked]>
To: "Julian Reschke" <[email-masked]>; <[email-masked]>;
"Stanley Guan" <[email-masked]>; <[email-masked]>
Sent: Thursday, October 16, 2003 2:50 PM
Subject: RE: DAV Schema Assessment (was Re: rfc2518bis DAV DTD ...)

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Thu, 16 Oct 2003 21:01:38 -0700

I don't understand how we got to this point.

What about this has anything to do with breaking clients?  

I thought the "ignore any element/attribute you don't understand" was a pretty clean rule, and trumped the use of extensions *in*the*DAV*XML* by another participant in the protocol.  And this goes along with knowing that a service may also ignore XML elements and attributes from you that it doesn't understand for the level of DAV that it implements.

The rules to DAV seem pretty clear: new DAV specs that come out can add elements to the DAV namespace.   What an older DAV implementation is to do with any of that sent its way is well defined.  Any use of a schema validator would have to allow for that.  

But I might know that what I am producing (not receiving) is at a particular level of DAV, and I use schema assessment to verify that.  And if there is an update to DAV, I might have to update the schema I am using.  Or not, if I don't intend to use the extension and the rules say it is still legal to ignore the extension. 

Where does breaking come in?  

I apologize Eric, I don't know what the context is any more.  

-- Dennis

-----Original Message-----
From: Eric Sedlar [mailto:[email-masked]]
Sent: Thursday, October 16, 2003 15:34
To: [email-masked]; Julian Reschke; Stanley Guan;
[email-masked]
Subject: Re: DAV Schema Assessment (was Re: rfc2518bis DAV DTD ...)

Dennis--

--Eric

[ ... ]

---

From: "Eric Sedlar" <[email-masked]>
Date: Thu, 16 Oct 2003 21:49:43 -0700

The problem is that there is no good way to say:
"ignore any element/attribute you don't understand"
in XML Schema.

--Eric 

> From: Dennis E. Hamilton [mailto:[email-masked]]
> Sent: Thursday, October 16, 2003 9:02 PM
> To: Eric Sedlar; Julian Reschke; Stanley Guan; [email-masked]
> 
> I thought the "ignore any element/attribute you don't understand" was a
> pretty clean rule, and trumped the use of extensions *in*the*DAV*XML* by
> another participant in the protocol.  And this goes along with knowing
> that a service may also ignore XML elements and attributes from you that
> it doesn't understand for the level of DAV that it implements.
> 
> The rules to DAV seem pretty clear: new DAV specs that come out can add
> elements to the DAV namespace.   What an older DAV implementation is to do
> with any of that sent its way is well defined.  Any use of a schema
> validator would have to allow for that.
> 
> But I might know that what I am producing (not receiving) is at a
> particular level of DAV, and I use schema assessment to verify that.  And
> if there is an update to DAV, I might have to update the schema I am
> using.  Or not, if I don't intend to use the extension and the rules say
> it is still legal to ignore the extension.
>

---

From: "Julian Reschke" <[email-masked]>
Date: Fri, 17 Oct 2003 09:19:57 +0200

Eric,

> people are actually going to use validators, I don't think we should push

I think that's a non-issue. We don't want to push *any* particular language
on people. We were looking for something that would work better than DTDs,
and RelaxNG seems to do that (it can express what we want and it has a
readable syntax suitable for embedding in specs).

If the requirement for the notation actually is that it needs to be a W3C
spec and there have to be implementations by Microft, IBM and Oracle, then
we can stop this discussion right away. The only alternative to DTDs would
be XML Schema, and we know that it doesn't work. In which case we should go
back to the original question about what we want to say normatively about
the DTD fragments we use.

On the other hand, RelaxNG fulfills all technical requirements, has working
open source reference implementations, is as open as a spec as XML Schema
(although Oasis instead of W3C) *and* is on it's way to a real ISO standard
(something you can't say about XML Schema).

Julian

> [mailto:[email-masked]]On Behalf Of Eric Sedlar
> Sent: Friday, October 17, 2003 12:39 AM
> To: Julian Reschke; [email-masked]; Stanley Guan;
> [email-masked]

---

From: "Julian Reschke" <[email-masked]>
Date: Fri, 17 Oct 2003 09:20:40 +0200

Dennis,

> particular level of DAV, and I use schema assessment to verify
> that.  And if there is an update to DAV, I might have to update
> the schema I am using.  Or not, if I don't intend to use the
> extension and the rules say it is still legal to ignore the extension.
> ...

I think both Eric and I are focusing on validating messages upon reception.
If you just want to validate what you produced yourself, then, sure, it's
possible to do that with a Schema that only accepts a subset of all the
possible variations.

Julian

---

From: "Eric Sedlar" <[email-masked]>
Date: Fri, 17 Oct 2003 09:52:30 -0700

There are two possible use cases for a schema language to describe DAV XML
syntax.  Case 1 is as a tool like BNF to formally describe the protocol.  A
second is in the case where people would actually do validation at runtime.
I don't think that most WebDAV implementations today actually run DTD
validation.  For use case 1, XML Schema is fine since there is no backwards
compatibility issue.  If you want to require use case 2, then people would
have to include a validation engine in their WebDAV implementations.  If
WebDAV WG chose RELAX NG for use case 2, then all WebDAV vendors would have
to acquire or develop RELAX implementations, which to me, is pushing RELAX
on people.  If all we care about is case 1, we could always write our
standards using XML Schema, but just note that the schemas are for a formal
description rather than for use in implementations.

As far as your name-calling, goes, yes, Relax is almost a real ISO standard
like X.400 and other such successes from the ISO.  They're much better than
those poorly specified ones coming from IETF and W3C like SMTP and HTTP.  I
don't know why we have to descend into bashing of particular vendors or
standards organizations to have this discussion.  W3C is the organization
that currently owns most of the related standards we deal with in WebDAV,
such as XML and HTTP, so yes, I believe its standards are more relevant than
ISO ones when a conflict occurs (also note the mail address of this list).
Schema was designed by a large committee to meet the needs of a wide variety
of organizations and applications (e.g. structure definition, not just
validation), as opposed to RELAX, which was designed by one guy.  That's why
Schema has more market acceptance and RELAX is more elegant.  In general,
the IETF and the WebDAV WG has tried to keep discussions rooted in
practicality--what's actually out there in the bulk of the users.  As far as
what major vendors are doing, since you mentioned my employer, I would point
out that your employer is also a proponent of XML Schema (quoted from
http://www.w3.org/2001/05/xml-schema-testimonial.html):

SAP is pleased to see that XML Schema has become a W3C Recommendation. XML
Schema is a key integration technology for supporting tightly coupled
business processes through loosely coupled components within and outside of
the company boundary and an essential standard for building and leveraging
shared knowledge about collaborative services and processes. SAP is
committed to embracing XML Schema throughout the mySAP.com e-business
platform by providing XML-based services and leveraging XML Schema to
support business integration within mySAP Technology.
-- Dr. Peter Barth, Director Corporate Marketing mySAP Technology and mySAP
Workplace, SAP AG

--Eric

> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of Julian Reschke
> Sent: Friday, October 17, 2003 12:20 AM
> To: Eric Sedlar; Julian Reschke; [email-masked]
> 
> 
> Eric,
>

---

From: "Julian Reschke" <[email-masked]>
Date: Fri, 17 Oct 2003 19:19:05 +0200

> Sent: Friday, October 17, 2003 6:53 PM
> To: 'Julian Reschke'; [email-masked]

I agree that use case 1 is the one we talk about (at least it's the one that
triggered this discussion). I happen to disagree that XML Schema is a good
choice, because *for this particular use case*, it adds only little to what
we already can do with DTDs, however it has the disadvantage of not having a
compact syntax I'm prepared to put into a spec.

(Side note: unless we use Trang to convert the XML Schema to RelaxNG Compact
Notation)

The DTD fragments we have right now at least offer good readability. I'm not
prepared to throw them away just because DTDs aren't en vogue anymore.

> to acquire or develop RELAX implementations, which to me, is pushing RELAX

Use case 2 is of no interest to me.

> description rather than for use in implementations.

We could, but what would be the advantage compared to DTDs? We would still
have to enumerate all the exceptions caused by WebDAV's extensibility rules.
As far as I understand, we'd only get rid of one single exception (being
namespace-awareness).

> ISO ones when a conflict occurs (also note the mail address of this list).

In fact, WebDAV and HTTP are IETF specs. It's just this mailing list that
happens to be hosted by the W3C.

As for which organization is relevant -- this was not supposed to be an
argument *against* W3C or IETF. In fact, in that case I'd have to ask myself
why I'm putting all this work into IETF specification development.

I just wanted to clarify that RelaxNG isn't just an unimportant niche
product: one of it's creators is Jim Clark (editor of XSLT 1.0 rec) and it's
published by Oasis (members including Oracle, SAP and Microsoft, for that
matter). So as far as I'm concerned it has the same credibility as any
competing W3C spec.

Now I do understand that people who have heavily invested into XML Schema
want to see it used. However, that's really not a convincing element if
there are alternatives which seem to be better suited from a technical point
of view.

BTW here are some more alternatives:

- specify an XSLT transformation that will transform any legal WebDAV
message to a format that *can* be DTD validated, or

- specify constraints by a serious of XPath assertions (a la Schematron),
such as

	count(d:allprop|d:prop|d:propname)=1

...and so on.

> validation), as opposed to RELAX, which was designed by one guy.

Two, as a matter of fact.

So does this make XML Schema better per se?

> That's why

So if all we need is a notation for RFCs, what's more relevant? Elegance or
market acceptance?

> -- Dr. Peter Barth, Director Corporate Marketing mySAP Technology

Actually, greenbytes is my employer.

Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Fri, 17 Oct 2003 11:05:13 -0700

That's correct.

You can't use an XML Schema definition as a pass-fail filter that rejects elements/attributes that might be from an extension and are not comprehended in the schema definition.  Since you also can't reject ad hoc non-DAV elements and attributes either, one must be careful how the schema definition is written and how/whether schema assessment is applied to material that is received.

In my world, that's not a problem. That's just how it is.  It is a limitation on the applicability of XML Schema to DAV XML documents in request and response bodies.

-- Dennis

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Eric Sedlar
Sent: Thursday, October 16, 2003 21:50
To: [email-masked]; 'Julian Reschke'; 'Stanley Guan';
[email-masked]
Subject: RE: DAV Schema Assessment (was Re: rfc2518bis DAV DTD ...)

--Eric 

[ ... ]

---

From: "Lisa Dusseault" <[email-masked]>
Date: Fri, 17 Oct 2003 23:15:28 -0700

> > ...
> > 	(a) the ANY case in an element markup declaration means 
> any element 
> > that is declared in the DTD, not some undeclared or unspecified 
> > element. ...
> 
> Thanks for the reminder. I had forgotten about that. It 
> basically means that by definition you can't express the 
> content model of <prop> using a DTD.

So does this mean we should simply remove the DTD-style definition of 
the 'prop' element (as well as 'owner' and 'resourcetype') from the
spec?  I believe the natural-language specification should be 
sufficient to define these anyway.

The 'owner' element could maybe be defined as MIXED but 'prop' and
'resourcetype' cannot be mixed.

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Fri, 17 Oct 2003 20:35:15 +0200

> Sent: Saturday, October 18, 2003 8:15 AM
> To: [email-masked]; 'Julian Reschke'; 'Geoffrey M Clemm';
> [email-masked]

The drawback would be that this would break the DTD's syntax. An alternative
is to keep ANY, and make sure that the description says what this actually
means.

> 'resourcetype' cannot be mixed.

Yes.

---

From: "Lisa Dusseault" <[email-masked]>
Date: Sat, 18 Oct 2003 00:06:27 -0700

> The drawback would be that this would break the DTD's syntax. 
> An alternative is to keep ANY, and make sure that the 
> description says what this actually means.
> 
It wouldn't break the DTD syntax if we only use DTD fragments to 
formally define those elements that can be formally defined.  I thought
the idea of omitting the full combined DTD appendix was a generally
acceptable idea, as long as the DTD fragments were still there
for most of the elements.

Lisa

---

From: "Lisa Dusseault" <[email-masked]>
Date: Sat, 18 Oct 2003 00:35:09 -0700

I have an ad-hoc document here that might serve your needs:

http://www.sharemation.com/~milele/public/dav/RFC2518%20Changes.doc

It's gotten increasingly inaccurate because it's hard to
track changes to changes to changes to RFC2518.  If you find
any such inaccuracies and point them out to me, I will be 
happy to fix them. 

Lisa

> [mailto:[email-masked]] On Behalf Of Stanley Guan
> Sent: Thursday, October 16, 2003 1:58 PM
> To: [email-masked]
> Subject: Easy-to-read deltas for rfc2518bis?
> 
> 
> 
> Hi,
> 
> Is there any delta history of the rfc2518bis?  Is it possible 
> to provide something similar to:
> regarding editorial efforts to highlight what's new and 
> what's different from the original rfc2518 in rfc218bis?
> 
> Thx,
> 
> -Stanley
> 
>

---

From: "Stanley Guan" <[email-masked]>
Date: Fri, 17 Oct 2003 13:06:46 -0700

Lisa,

This is a great help!  Will let you know if I've
found out any inaccuracies in it!

Thx,

-Stanley

---

From: "Julian Reschke" <[email-masked]>
Date: Fri, 17 Oct 2003 22:44:33 +0200

OK,

here's an attempt to summarize... Almost all participants in this discussion
have been trying to "solve" the issue in different ways.

1) Remove the DTD fragments and replace them by english language
descriptions. This actually triggered this thread (see title) because in
rfc2528bis, some of the information present in rfc2518 was lost.

2) Extend/fix the DTD: we can do better with DTD notation but not much
better. However there's a readability cost associated with these changes,
and it's unclear wether that cost justifies the small increase in precision.

3) Move to XML Schema: only solves few of the problems that we have with the
DTD notation, yet has a big problem with readability in the spec.

4) Move to something else: I was proposing RelaxNG (Compact Notation)
because it solves all technical issues that we have. Drawback: some think
that the fact that it's not as popular as DTD or XML Schema makes it
unsuited for usage in a RFC.

So it seems there's no support at all for a major change. Let's fix just the
issues that we have with the DTD fragments. I'd suggest to re-read what the
ordering spec currently says and, if necessary, to clarify it. Then we can
use that in all other specs.

Julian

---

From: "Dennis E. Hamilton" <[email-masked]>
Date: Fri, 17 Oct 2003 16:32:56 -0700

Good summary.

I looked at the ordering specification.  

1.	I would suggest that some other form than ANY be used to mean that there is an open content model with nothing said about it.  Maybe an invented form like "(...)"   [or even %:... oddly enough]

2.	With regard to the statement about ordering, it may be necessary to say more.  For example, in

I would read this, in terms of order being unimportant, as providing for at most one <ordering-type> element, none or more <order-member> and extension elements, all in any order.

I assume that if there are multiple <ordering-type> elements, it is the ones after the first that must be ignored if that usage is not understood.  On the other hand, an example in rfc2518 section 23 (and bis-04 section 24) would indicate that a second occurrence of <ordering-type> is illegal and must be rejected with a 400 response.  (See (4) below).

3.	A lax DTD fragment for all of that is something like

	<!ELEMENT orderpatch (ordering-type | order-member %:...)*>

where other conditions on multiplicity are stated in the text [and, in this example, %:... can be defaulted to the empty string or some hypothetical sequence beginning with "| ".  

3.1	Does appearance of mixed content qualify as an extended use?  Then it would be

	<!ELEMENT orderpatch (#PCDATA | ordering-type | order-member %:... )* >

If you really mean that ANY DAV-defined element can potentially be here as well as any other (i.e., <error>, <make-collection>, and so on), that is the best that can be done in a DTD that does not exclude any possibly-valid element, leaving the rest up to the DAV processor.

3.2	In a narrower notion of what an admissible extension could be, it might be more useful to consider

	<!ELEMENT orderpatch (ordering-type | order-member %orderpatch:... )* >

with the side stipulation that there be at most one <ordering-type> element.  I would give this the reading that there may be any number of potentially-acceptable <orderpatch>-extension content elements other than those explicitly identified.  That is, the conditions on admissible ordering-type and order-member occurrences are not modifiable by extension to the <orderpatch> element.

4.	Is there any more-precise definition of extension than the one in this part of the ordered-collection specification and in rfc2518 section 23?  (rfc2518bis-04 section 24?)  

4.1	I am thinking there is a difference between an extension and an application involving ad hoc additions, as in the content of a <proc> element, but perhaps it is all regarded as extension.  

4.2	It also would seem, based on the example of a mandatory 400 response, that extensions cannot contradict the multiplicity given in 2518 or any extension specification (as honored in 3.2).  

It's a mystery.

-- Dennis

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Julian Reschke
Sent: Friday, October 17, 2003 13:45
To: [email-masked]
Subject: RE: How to use DTDs, or not to (was: RE: ACL and lockdiscovery)

[ ... ]

Julian

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 18 Oct 2003 02:17:09 +0200

> Sent: Saturday, October 18, 2003 1:33 AM
> Subject: RE: DTD-Fragment Approach (was Re: How to use DTDs, or not to
> ...)
>
>
>
> Good summary.
>
> 1.	I would suggest that some other form than ANY be used to
> mean that there is an open content model with nothing said about
> it.  Maybe an invented form like "(...)"   [or even %:... oddly enough]

Yes, I agree that if we don't mean "ANY", we shouldn't say so. Or if we
continue to say so, that ANY comes with a comment line saying what it means.
See, for instance:

<http://greenbytes.de/tech/webdav/rfc3253.html#PROPERTY_supported-live-prope
rty-set>

(side note: the HTML versions of the RFCs on our web page contain DCMI
metadata encoded in the format specified in RFC2774)

> I would read this, in terms of order being unimportant, as
> providing for at most one <ordering-type> element, none or more
> <order-member> and extension elements, all in any order.

Correct.

> I assume that if there are multiple <ordering-type> elements, it
> is the ones after the first that must be ignored if that usage is

Actually, multiple instances of ordering-type are illegal. That's why the
DTD says "?".

> not understood.  On the other hand, an example in rfc2518 section
> 23 (and bis-04 section 24) would indicate that a second
> occurrence of <ordering-type> is illegal and must be rejected
> with a 400 response.  (See (4) below).

Right.

> where other conditions on multiplicity are stated in the text
> [and, in this example, %:... can be defaulted to the empty string
> or some hypothetical sequence beginning with "| ".

in Relax NG Compact Syntax, this would be:

	namespace dav = "DAV:"
	orderpatch = element dav:oderpatch (ordering-type? & order-member* & EXT*)

	ordering-type = element dav:ordering-type ...
	(and so on...)

So in theory we could just borrow the "&" notation (which is the same as ","
except that ordering isn't significant), and use EXT (or any other
identifier) to denote elemements not defined by the spec (and other base
specs). So I'd prefer:

 	<!ELEMENT orderpatch (ordering-type? & order-member* & EXT*)>

(of course this is also a demonstration about why the RelaxNG Compact
Notation is very popular around former DTD users).

> If you really mean that ANY DAV-defined element can potentially
> be here as well as any other (i.e., <error>, <make-collection>,
> and so on), that is the best that can be done in a DTD that does
> not exclude any possibly-valid element, leaving the rest up to
> the DAV processor.

Actually, RFC2518 is vague about how mixed content and extensibility work
together. Technically it says that anything can be extended, however I
assume that almost all implementations get very confused when I start
extending elements with text content, such as

	<d:getcontentlenght>123<bytes/></d:getcontentlength>

although I think this conforms to the spec. So a conservative approach would
be to limit extensibility to elements that do not have text content, and not
to allow text content as extension.

> with the side stipulation that there be at most one
> <ordering-type> element.  I would give this the reading that
> there may be any number of potentially-acceptable
> <orderpatch>-extension content elements other than those
> explicitly identified.  That is, the conditions on admissible
> ordering-type and order-member occurrences are not modifiable by
> extension to the <orderpatch> element.

%orderpatch would need to represent all elements not defined in the relevant
specs, in RNC:

	EXT = * - (dav:allprop | dav:href | dav:multistatus | dav:prop |
dav:propfind | dav:propname )
		{ any }

(of course the list is incomplete...).

> 4.	Is there any more-precise definition of extension than the
> one in this part of the ordered-collection specification and in
> rfc2518 section 23?  (rfc2518bis-04 section 24?)

No.

> 4.1	I am thinking there is a difference between an extension
> and an application involving ad hoc additions, as in the content
> of a <proc> element, but perhaps it is all regarded as extension.

We discussed this just a few days ago. I wouldn't call the content model of
<prop> "extensible". Whatever you put there is defined by the spec (as
identifier of a property). I'd like to use the term "extensibility" when we
speak about things *not* defined by the spec (such as adding new child
elements to DAV:response).

> 4.2	It also would seem, based on the example of a mandatory 400
> response, that extensions cannot contradict the multiplicity
> given in 2518 or any extension specification (as honored in 3.2).

I didn't get that. Could you please explain?

> It's a mystery.

I don't think it's that bad. Except for one particular client, it doesn't
seem to have caused actual interoperability problems so far.

Julian