Subject: RE: Tx of text type docs and CR/LF, CR, LF translation

From: "Lisa Dusseault" <[email-masked]>
Date: Wed, 3 Jul 2002 16:18:49 -0400

The problem you describe is not particularly a WebDAV problem. IMAP and
FTP are subject to the same problems.  MIME also does not attempt to fix
the problem.

The HTTP 1.1 specification does make some attempt to improve the
situation, and it works *when viewing in the browser*.  It's not clear
how browsers should save files though. The browser could: 
 - save the file the same way it was downloaded
 - save the transformed file, replacing single CR or LF with CRLF

Either approach leads to problems.  The current common approach, where
the browser saves the file the way it was downloaded, means the file may
be unusable when viewed in various text editors.  However, if the
browser transforms the file, it may be altering the file
inappropriately.  The transformation assumes that wherever CR or LF is
used CRLF is appropriate, and that may not be true for all uses of
'text/plain'.  

I see why you wish attention to be brought to this subject, but the real
offenders are not WebDAV clients.  It's applications like NotePad that
can't display reasonable file formats that are problematic. Those
implementers are unlikely to read this list or the WebDAV specification.

You might try reporting this behavior as a bug to the manufacturers of
the offending software.

Lisa 

-----Original Message-----
From: ME [mailto:[email-masked]] 
Sent: Wednesday, July 03, 2002 12:17 PM
To: [email-masked]
Subject: Q?: Tx of text type docs and CR/LF, CR, LF translation

Hello,

I have searched with google, read afew dav FAQs, and looked through some
RFC before posting this message here. This is my first post to any
W3/IETF
list, so please bear with me for any mistakes I may make. :-)

Problem: When WebDAV enabled clients (tested with "MS Web Folders",
"Cadaver", and "Goliath") request a text file (text/plain, text/html)
from
a DavEnabled Webserver (in this case Apache 1.3.26 with mod-dav-1.0.3
running over SSL) the file is transmitted from the server with the same
line termination the file had when examined on the server from a shell.

Files previously edited with SimpleText (MacOS), or notepad (Windows)
and
stored on the server with netatalk-asun or samba contain CR (^M) or CRLF
(^M^J) respectively. Files created from Linux bash with emacs/vi have
the
LF (^J) as the line termination char.

Files created strictly from a Mac are not in "human readable format"
when
the windows user opens then with notepad.

Files created strictly from a Windows box are not in "human readable
format" when the mac user opens then with SimpleText.

Forcing users to use a web authoring tool, or "BBEdit" (Mac) or
something
like Codewarrior Editor (Windows) that can deal with the 3 common line
terminations strings for files and leave the file in "human readable
format" are known, but not desired.

Web browsers (Netscape, MSIE, Lynx) read these files from the same web
server and perform proper translation such that the browser
subsitutes the client OS's own line break for the served content's body
(or file's contents.)

This seems to echo rfc2616 , section: 3.7.1 
    "Canonicalization and Text Defaults:"
( http://asg.web.cmu.edu/rfc/rfc2616.html )

"When in canonical form, media subtypes of the "text" type use CRLF as
the
text line break. HTTP relaxes this requirement and allows the transport
of
text media with plain CR or LF alone representing a line break when it
is
done consistently for an entire entity-body. HTTP applications MUST
accept
CRLF, bare CR, and bare LF as being representative of a line break in
text
media received via HTTP."

So, here is my thought:

If RFC for HTTP/1.1 states "HTTP applications must accept CRLF, bare CR,
and bare LF as being representative of a line break in text media
received
via HTTP." and WebDAV clients are considered HTTP Applications, and the
specs for WebDAV include use of HTTP for transmission of text type docs,
then shouldn't WebDAV clients also "inheirit" this requirement from
HTTP/1.1 RFC?

If so, could this be included to be more explicit in the present RFC for
WebDAV?

Presently, I am using mod_rewrite and server side processing with
directory forward inheiritance to process through DAV based requests for
files and perform translation to the end of line termination string of
the
DAV requests files based on the client string submitted in the request.
This is clunky and is not the "right solution" and I am looking for
somehting better such as getting vendors of WebDAV based clients to
change.

Thanks!
-ME

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/CM$/IT$/LS$/S/O$ !d--(++) !s !a+++(-----) C++$(++++) U++++$(+$)
P+$>+++ 
L+++$(++) E W+++$(+) N+ o K w+$>++>+++ O-@ M+$ V-$>- !PS !PE Y+ !PGP
t@-(++) 5+@ X@ R- tv- b++ DI+++ D+ G--@ e+>++>++++ h(++)>+ r*>? z?
------END GEEK CODE BLOCK------
decode: http://www.ebb.org/ungeek/ about:
http://www.geekcode.com/geek.html

---

From: ME <[email-masked]>
Date: Mon, 8 Jul 2002 10:12:50 -0700 (PDT)

> FTP are subject to the same problems.  MIME also does not attempt to fix

FTP allows for use of ASCII (A) mode transfers as well as binary/image
(I), and as for MIME attached docs to e-mail messages, CR/LF and line
termination for text files is done by most modern mail clients to allow
the user of the local OS to view their text/* files in their local OS's
line termination format. (I know this means some mail clients do not
follow the RFC for mail, but they offer the user something they want and 
are able to use.)

> implementers are unlikely to read this list or the WebDAV specification.

I can understand this side and point of view, and see it would be easier
to not enforce client side conversion for local OS line break format with
text files. Inclusion of an OS specific line break as a requirement ties
"OS" to the RFC, and can make it less employable in unforseen
environments. Users who happen to use WebDAV to move CGI files or perl
scripts and edit with a text editor that does not understand multi-OS line
break formats will wonder why the script stops working on the web server,
but they will need to be told it is not WebDAV, but the editor that came
with their OS. They can download and use BBEdit for MacOS 9.x and earlier
or the new one with MacOSX, etc.

> You might try reporting this behavior as a bug to the manufacturers of

Alas, there are at least 30 different text based editors for various OS
that recognize the local OS line break formats exclusively, though many
are used by few, outdated, no longer available, and not open
source. However, there are about 4 major WebDAV clients out there in
production where a modification from server formatted line breaks to local
OS line breaks with little modification to code required. I chose to try
the least-work path and have it addressed here.

Based on the responses in this list, I can see enough support exists
against this line break feature being incorporated into the WebDAV RFC
that I don't feel any need to push it.

Insetad, I have found a way to make my web server provide on-the-fly line
break conversion of files pushed through WebDAV such that each client gets
their local OS's line breaks for specific text/* files and then upon
being returned to the server get converted to the Server's expected line
break format. This works for my users, and only applies to the WebDAV
work. If WebDAV clients should ever add this as a local feature, the
"fix" (hack) allows for addresing client specific information to determine
how the conversion should work - if at all.

Thanks for your time in reading the original request, WebDAV is still
much, much better than other file transfer protocols for the needs of my
web users.

-ME

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/CM$/IT$/LS$/S/O$ !d--(++) !s !a+++(-----) C++$(++++) U++++$(+$) P+$>+++ 
L+++$(++) E W+++$(+) N+ o K w+$>++>+++ O-@ M+$ V-$>- !PS !PE Y+ !PGP
t@-(++) 5+@ X@ R- tv- b++ DI+++ D+ G--@ e+>++>++++ h(++)>+ r*>? z?
------END GEEK CODE BLOCK------
decode: http://www.ebb.org/ungeek/ about: http://www.geekcode.com/geek.html