Subject: RE: [Simple] MSRP: Flow Control

From: Adam Roach <[email-masked]>
Date: Mon, 28 Apr 2003 15:51:15 -0500

The Head of Line Blocking thread has drifted to flow
control. The problem basically stems from relay
behavior when an incoming link is faster than an
outgoing link.

As Paul pointed out earlier, the solution to this problem
is to have a window of max number of {messages,bytes}
outstanding.

The difficult, reinvent-the-transport-layer solution
is a dynamic window with bytes as a metric. The easy
way is a fixed window with messages as a metric. Writing
up the easy way is a trivial endeavor ("Clients MUST NOT
have more than three outstanding SEND transactions for
each session at any one time"). Easy to specify, easy
to implement.

If you add this, Head-of-line blocking goes away, since
relays can be engineered to buffer incoming messages,
with assurances that each session can take a well-known,
finite amount of memory. (You really want the application
to read information off the wire as it arrives, instead of
allowing the incoming TCP buffer to fill up, causing
retransmissions and backoff at the transport layer, anyway).

To make this work properly, we should also allow messages to
pend for long enough for the window of messages to clear the
smallest expected pipe. (Note that this is true even without
relays). For example, if we take "three messages"
to be the window size, 64k to be the max message size, and
20kbps to be the smallest expected pipe, each SEND might take
as long as 64k @ 20 kbps = 27 seconds (assuming relays start
sending messages to their next hop before the entire message
is received). If our window size is three, we will need to
allow messages to pend for at least (27 * 3 = ) 81 seconds
before timeout, or we'll lose the ability to send chunked
transfer over a skinny pipe. (This is true even without a
bounded window, except that you need to substitute "infinity"
for "81"). We'll also need to state 20 kbps (or whatever we
decide to use) as an engineering parameter, and note that
chunked transfers over slower links will behave suboptimally
(i.e. timeout).

By adding such a window, one can at least engineer a relay by
saying that internal buffering of messages shall be
(3 * 64k * max_sessions). A bound, even if large, is better
than requiring relays to have infinite memory.

The metrics "three messages" and "20 kbps" are just gut-feeling
estimates. If someone has reason to believe that they should
be different, speak up.

/a
_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Ben Campbell <[email-masked]>
Date: 28 Apr 2003 16:09:56 -0500

> each session at any one time"). Easy to specify, easy

I recall a big push-back to the idea of not allowing overlap on IMs back
in the page-mode spec days. It is not clear to me that allowing a small
fixed number of pipelined messages is significantly better than complete
non-overlapping for the situations that people seem to care about--that
is, large file tranfers. (Maybe we should just specificy how to setup
ftp sessions...) Specifically, this would seem to add a lot of
inneffeciency to Cullen's 5gb transfer. Of course, that also keeps
Cullen from hogging the relays.

> retransmissions and backoff at the transport layer, anyway).

I assume you mean it, plus a  message size limit, makes HoLB go away,
right?

---

From: "Dean Willis" <[email-masked]>
Date: Mon, 28 Apr 2003 16:25:00 -0500

I think flow control is a completely seperable issue from HOLB, although the
two can intersect.

The fundamental flow control problem is that if there is more stuff flowing
into a relay node than there is flowing out of it, then the buffer space
available on that node gets used up. This is an application issue, not
really a transport issue, and it potentially applies to ANYTHING that
operates in a store-and-forward manner.

I believe the flow control problem for MSRP, assuming messages are not
infinitely long, is easily addressed by the allowing the intermediate server
to respond to a message request with a "Wait! I'm Full! Stop Sending Me
Stuff" response, perhaps along the lines of the "Retry After" semantic in
HTTP and SIP. Oddly enough, this is reminiscent of frame relay's BECN or
TCP's ECN functions, RS-232's XON and XOFF (or RTS/CTS), and so on. It's
also very similar to the way that SMTP works. Not rocket science, and I
don't think there's any need to develop a standardized mathematical formula
describing what "full" means to a server  . . . 

--
Dean