Subject: RE: [Simple] Re: Draft-campbell-simple-im-session-01: Framing

From: Adam Roach <[email-masked]>
Date: Mon, 21 Apr 2003 14:59:36 -0500

> -----Original Message-----
> From: Cullen Jennings [mailto:[email-masked]]
> 
> There more or less two way to do framing. Put in the size at 
> the begging or
> put a unique marker at the end. Both have pros and cons. 
> MSRP looks like it
> has the worst of both.

How? Learning from the lessons of SIP, MSRP puts the
length field in a fixed location near the top of the
message. It seems that this is the first of two methods
that you point out as being acceptable.

The only real improvements that I could propose would be
(a) putting the length as the *first* field in the message,
instead of the *second*, (b) including an offset into
the message that points to the start of the body, and
(c) making the numbers fixed length. [Use of fixed length
numbers has drawbacks, of course...]

That said, you call out delimiter-framing as your favored
choice:

> Given we are taking about large 
> message that may be
> generated by things that start sending them before the 
> generation of the
> message is complete, I think the unique marker is the best 
> way to go. 

I would tend to disagree. While this makes real-time
composition and streaming of content on the source size
slightly easier for these relatively rare cases in which
content is produced on the fly (albeit requiring escaping
of the end marker), it imposes a substantial penalty on
the receiving end. For example, if you want to have a
lower layer splitting individual messages out based on
framing, an end marker requires this layer to scan every
received character for both the end marker and for escaped
versions of the end marker.

In addition to requiring somewhat more code, this is
going to be truckloads slower, as it requires every
octet to be touched one additional time on receipt,
*and*, if the receiver wants to pull the body together
into contiguous memory (which will often be the case),
it will require a second copy of the message to be
built as it is unescaped.

For example, minus error checking, here's a dirt-simple,
fast way to get a single MSRP message off a stream
(minus error checking):

int read_msrp_message (int sock_desc, char *buffer, int bufflen)
{
  int length = 0;
  int chars_read = 0;

  /* Read "MSRP " string */

  read (sock_desc, buffer, 5);
  buffer += 5;
  chars_read += 5;

  /* Read the total message length */

  read (sock_desc, buffer++, 1);
  chars_read++;
  while (*buffer & 0xF0 == 0x30)
  {
    length *= 10;
    length += (*buffer & 0x0F);
    read (sock_desc, buffer++, 1);
    chars_read++;
  }

  /* Read the remainder of this MSRP message. */
  /* Potentially very fast, if network card interface is DMA. */

  read (sock_desc, buffer, length - chars_read);

  return length;
}

Can you show me a similar routine for marker-based
delimitation? Would it be reasonable to just drop it
in an e-mail, or would it be too large?

/a
_______________________________________________
Simple mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/simple

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 21 Apr 2003 22:43:40 -0700

What I don't want to do is have a counter stopping me from streaming, then
say I am going to secure it with S/MIME forcing me to watch the bits too.

> in an e-mail, or would it be too large?

Ouch - well you know my code, bit of STL, 5 templates, some C++ and my
object code to implement the same thing as your code would certainly be too
large to IM to you :-)

I was not assuming any escaping - that way lays madness. I know this
introduces some issues like having to stop a message, change the marker and
start writing out again when streaming and it requiring larger makers. A 64
bit random marker has a surprisingly low rate of message corruption.

I think it all depends on the architecture we are talking about here. I am
ignoring the CPU power required to do it on a cell phone because I don't
know as much about those. I was thinking more about what was happening at
the relays where I was assuming a SUN or Intel type architecture. The PC I
am working on can transfer stuff over the PCI bus at a bit over 100MB/s
while I can get upward of 3GB/s from memory to the CPU. If the marker
started with 8 "-" I could just check for 4 aligned bytes that were equal to
"----". Even ignoring MMX, this is blazingly fast on any RISC machine. I
will be able to do it a whatever the CPU/Memory transfer rate is - given the
bus with the Ethernet card to memory rate is a small faction of this, I
don't think this is a big deal to look for a marker even in the cases where
you can directly DMA from the card to user space memory (which as you know
generally takes more than 10 lines of code :-). The point is, if bus
bandwidth is the bottle neck, you can use some extra CPU for free. (unless
of course you CPU is busy computing RSA stuff to set up TLS connections :-)

My example picked a 5G file because it is about the size of a DVD and larger
than 32 bits. 

If we do use markers, I think we should say they must start with some fixed
sequence such as 4 or 8 dashes to speed up finding them.

> that you point out as being acceptable.

Yes - big improvement.

> instead of the *second*,

I like it how it is so that things can classify the traffic before hitting
any variable fields.

> (b) including an offset into

I can live with either (size or delimiter based) but would prefer not to
have to do both on every message.

> built as it is unescaped.

Depending on the escaping you can usually build the new message on top of
the memory used by the old message.

---

From: Ben Campbell <[email-masked]>
Date: 22 Apr 2003 14:43:41 -0500

Some comments in line.

> > that you point out as being acceptable.

I am a bit confused--bit improvement over what? SIP? A perception that
MSRP did _not_ put the size at the beginning?

> I like it how it is so that things can classify the traffic before hitting

Agreed. Further, putting the size first presents an opportunity to
interpret certain forms of garbage as a real size.

> have to do both on every message.

Can you clarify what you mean by doing both? Using a delimiter to find
the body? It seems to me that these two concepts (length of message and
position of body) are logically distinct. For example, a relay does not
care where the the body starts. An endpoint is likely to frame the whole
message first, then leave body parsing for later.

If have a strong desire to add a start line field containing body offset
or length, I don't have a big problem with it. Thoughts from others?

[...]

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 23 Apr 2003 20:40:25 -0700

Few comments deep inline - sorry.

> MSRP did _not_ put the size at the beginning?
Sorry - yes I meant current MSRP draft is an improvement over SIP.

> message first, then leave body parsing for later.

I was not really worried about finding the start of the body - I was talking
about finding the end of the body. I find it highly likely that the current
MRSP draft would make me put in a size before I could send the message and
that when I received the message I would still have to search the message
for an end marker to do the S/MIME stuff.