Subject: RE: Interest in standardizing Batch methods?

From: "Clemm, Geoff" <[email-masked]>
Date: Mon, 7 Jan 2002 22:57:45 -0500

Like Greg, I'd like to see some evidence that this is not
a problem that is more generally solved by pipelining requests.

Cheers,
Geoff

-----Original Message-----
From: Greg Stein [mailto:[email-masked]]
Sent: Monday, January 07, 2002 9:33 PM
To: Jim Whitehead
Cc: WebDAV
Subject: Re: Interest in standardizing Batch methods?

On Fri, Jan 04, 2002 at 01:11:03PM -0800, Jim Whitehead wrote:
>...
> To address these performance issues, several "Batch" methods were
developed
> as relatively simple extensions to existing WebDAV methods. Switching
> Outlook Web Access to use these methods resulted in approximately an order
> of magnitude performance increase (obviously, the performance benefit of
> going from N round-trips to 1 round trip depends on N). From the user
> perspective, the observed elapsed time for executing an operation went
from
> multiple seconds down to close to a second (depending on latency, of
> course). It was a signficant performance improvement. The batch methods
are:

I wonder whether the performance suffered because the requests were
performed in a request/response fashion, rather than as a series of
pipelined requests. If you can pipeline requests, not waiting for an answer,
then a series of DELETE operations is simply a "larger request" and then you
handle a "larger response". Yes, each request/response has more overhead
than a batched operation.

Personally, I'm going to guess they didn't pipeline requests, so a batch
mechanism was a must to get around deficiencies in their protocol stack.

That said, it is important to recognize the overhead in a sequence of, say,
DELETE requests and their responses, relative to a potential batch
operation. Specifically, you're going to have a lot of duplicate headers on
the requests and responses (there are no bodies in this case). How much does
this pose over a batch delete with a list of URLs? Maybe 3x or 4x in the
number of bytes? Maybe 10x? When you're talking over a modem (which is
typically the case for MSFT's Hotmail servers), then that 10x can be rather
significant.

Ah, it's all a numbers game. Personally, I'm not interested in batch
operations. I would guess that most of their benefit is obviating by
pipelining requests.

Cheers,
-g

ps. yes, this is mostly supposition; I'm not about to sit down and start
measuring byte counts and network traffic; I don't know whether they were or
were not pipelining; but my intuition tells me "no" and that pipelining is
the answer...

-- 
Greg Stein, http://www.lyra.org/

---

From: "Clemm, Geoff" <[email-masked]>
Date: Wed, 9 Jan 2002 12:16:59 -0500

"Idempotent" doesn't mean "returns the same response",
it means "produces equivalent state on the server".
RFC 2516 makes this clear by explicitly identifying PUT
and DELETE as idempotent operations.

Cheers,
Geoff

-----Original Message-----
From: Stefan Eissing [mailto:[email-masked]]
Sent: Wednesday, January 09, 2002 10:47 AM
To: Clemm, Geoff; WebDAV
Subject: RE: Interest in standardizing Batch methods?

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Clemm, Geoff
> Sent: Wednesday, January 09, 2002 4:17 PM
> To: WebDAV
> Subject: RE: Interest in standardizing Batch methods?
>
>
> A single MOVE, DELETE, or PROPPATCH request is idempotent
> (repeating the same request multiple times produces the
> same result as just doing it once).

Not quite: A second MOVE/DELETE/PROPPATCH will return a
different result to the client. So a second DELETE will
answer 404 instead of 204.

The server "state" will stay the same. I think that's
what you had in mind.

> A sequence of DELETE's is always idempotent.  A sequence of PROPPATCH's
> is always idempotent if the same property isn't updated by different
> PROPPATCH requests in that sequence.  A sequence of MOVE's is always
> idempotent if none of the Destination URLs overlap with any of the
> request URL's.
>
> Cheers,
> Geoff
>
>
> From: Julian Reschke [mailto:[email-masked]]
> Sent: Wednesday, January 09, 2002 3:34 AM
> To: Lisa Dusseault; Greg Stein; Jim Whitehead
> Cc: WebDAV
> Subject: RE: Interest in standardizing Batch methods?
>
>
> Looking at RFC2616:
>
> "8.1.2.2 Pipelining
> Clients SHOULD NOT pipeline requests using non-idempotent methods or
> non-idempotent sequences of methods
> (see section 9.1.2).
>
> So it seems that pipelining wouldn't be allowed for anything
> except PROPFIND
> (MOVE/DELETE/PROPPATCH aren't idempotent), right?
>
> > From: [email-masked]
> > [mailto:[email-masked]]On Behalf Of Lisa Dusseault
> > Sent: Wednesday, January 09, 2002 6:30 AM
> > To: Greg Stein; Jim Whitehead
> > Cc: WebDAV
> > Subject: RE: Interest in standardizing Batch methods?
> >
> >
> >
> > There's potentially a little more to it than that.
> > (1) Imagine a client selects a bunch of resources and drags to
> > move them all
> > to a different folder.  A batch MOVE operation can do those in one
> > transaction, so that the whole request fails if not all can be
> > moved.  This
> > becomes rather more important if the client is actually using an API
> > (MSDAIPP??) that offers large-scope operations, yet how can it guarantee
> > that operation will work or won't work if it can only send it
> piecemeal to
> > the server?
> >
> > (2) See Yaron's email
> > (http://lists.w3.org/Archives/Public/w3c-dist-auth/1998OctDec/0303.html)
> > about why pipelining doesn't always work (can't always be used even when
> > available).  I don't know to what extent pipelining is realistically
> > unavailable/unworkable.
> >
> > That said, it's still not clear batch methods are so necessary they'd
> > preempt other work we've got to do.
> >
> > Lisa
> >
>
>
>

---

From: "Clemm, Geoff" <[email-masked]>
Date: Tue, 15 Jan 2002 12:22:57 -0500

From: Lisa Dusseault [mailto:[email-masked]]

   > Personally, I'm going to guess they didn't pipeline requests, so
   > a batch mechanism was a must to get around deficiencies in their
   > protocol stack.

   There's potentially a little more to it than that.
   (1) Imagine a client selects a bunch of resources and drags to move
   them all to a different folder.  A batch MOVE operation can do
   those in one transaction, so that the whole request fails if not
   all can be moved.  This becomes rather more important if the client
   is actually using an API (MSDAIPP??) that offers large-scope
   operations, yet how can it guarantee that operation will work or
   won't work if it can only send it piecemeal to the server?

We should carefully distinguish between BATCH for "pipelining" and
BATCH for "transactions". Every server should be able to support a
pipelined request, but only servers whose repository is inherently
transactioned-based are likely to be able to support transactions.

Note that the Bxxx methods supported by the Microsoft Exchange 2000
server are pipelining batch requests, not transactioning batch
requests.

Cheers,
Geoff