Subject: RE: GSE IDs [Re: IETF multihoming powder: just add IPv6 and stir]

From: "Bound, Jim" <[email-masked]>
Date: Fri, 9 May 2003 08:05:09 -0400

By storing the end nodes in a routing header all other location can
change enroute.
This is completely stateless.
/jim

> -----Original Message-----
> From: Iljitsch van Beijnum [mailto:[email-masked]] 
> Sent: Friday, May 09, 2003 4:49 AM
> To: Tony Li
> Cc: [email-masked]
> Subject: Re: GSE IDs [Re: IETF multihoming powder: just add 
> IPv6 and stir]
> 
> 
> On vrijdag, mei 9, 2003, at 01:08 Europe/Amsterdam, Tony Li wrote:
> 
> > I would prefer that we avoided having a stateful mapping mechanism.
> 
> It all flows from having more than one address with different 
> reachability properties. At some point, someone has to decide which 
> address to use for a particular purpose, and it had better be the 
> "right" address most of the time or we're worse off than being 
> single-homed.
> 
> > It's unnecessary and certainly more complication than we need.
> 
> It doesn't have to be complicated: something as simple as publishing 
> all the addresses in the DNS and/or in an option in the first packet 
> along with a simple algorithm that figures out which address seems to 
> be working best is all we need. (But it may not be all we want.)
> 
> What kind of stateless mechanism do you have in mind? Obviously a 
> stateless solution would be preferable over a stateful one.
> 
> > I would also prefer that we not proclaim something to be GSE that 
> > isn't, regardless of congruence or continuation of ideas.
> 
> I did use the term "GSE" a bit loosely a week or so ago but rest 
> assured that this won't happen in the final version of the draft.
> 
> 
>

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 9 May 2003 14:40:24 +0200

> This is completely stateless.

Not really. Someone must know to put in the routing header at some 
point.

And this doesn't provide the functionality we need. Assume ISP A can 
reach destination X but ISP B can't. In this case, the source must 
transmit the packet over A. Transmitting it over B isn't going to yield 
useful results, regardless of the presence of routing headers.

Also, we can't assume that users will universally prefer a stateless 
solution with per-packet overhead over a stateful one that doesn't have 
extra overhead.

I think state in the endpoints is perfectly acceptable. If people want 
central control over their multihoming and/or they want to 
multihome-enable unmodified hosts by implementing the multihoming 
functionality in a middlebox, having state in this middlebox would also 
be acceptable, especially if it's "soft state" that can be recreated 
without breaking sessions if the middlebox fails. NAT shows that it is 
possible to build middleboxes that keep lots of state.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 09 May 2003 15:08:55 +0200

Iljitsch van Beijnum wrote:
> ... NAT shows that it is

and the failure modes associated with NATs show why
this is a really bad idea. But even NATs keep that state
reasonably local; it doesn't have to be known by magic
to an anti-NAT at the other end.

Iljitsch, the big hole in your -00 draft is that is doesn't
at all discuss the magic needed to distribute mapping state
and keep it fresh. As I think I already said, this is why we
never pursued the map-and-encap proposal some years ago- it's
not a side issue, it's *the* issue in this class of solutions,
and it is the fundamental difference from 8+8/GSE.

I believe MHAP does discuss this question. And as Jim said, there
are stateless solutions, but that means adding bits to the packet.

  Brian

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 9 May 2003 15:34:24 +0200

On vrijdag, mei 9, 2003, at 15:08 Europe/Amsterdam, Brian E Carpenter 
wrote:

> and keep it fresh.

Ok, we'll revisit this as soon as I have this part in the draft or when 
it's finished. In the mean time...

> And as Jim said, there are stateless solutions, but that means adding 
> bits to the packet.

And as I replied to Jim, I don't feel what he mentioned is truly 
stateless. Sure, it avoids state in the core of the network and at the 
receiver. Nobody was suggesting we should have state in the core, so 
the problem must be with mapping back at the receiving end. I don't see 
how this can be considered problematic as  doing this for the 
destination address is entirely trivial (just replace PA prefix A or B 
with PI prefix X) and for the source address it may either be 
unnecessary if we transmit the source address unmodified or the state 
necessary for this must be present in order to be able to send packets 
back anyway. Similar state must be present in order to add the routing 
header that Jim mentioned.

---

From: "Michael H. Lambert" <[email-masked]>
Date: Fri, 09 May 2003 09:36:53 -0400

>--On Friday, 9 May 2003 15:08 +0200 Brian E Carpenter 

> are stateless solutions, but that means adding bits to the packet.

And which is better (is some meaningful sense)?  We don't know.  And 
without implementations and operational experience, we aren't going to 
know.  The crux of the issue is that sites are reluctant to adopt IPv6 
because the "multihoming problem" has not been solved, but we can't solve 
the problem without that operational experience.

Michael

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Tue, 13 May 2003 10:11:09 +0200

On fredag, maj 9, 2003, at 14:40 Europe/Stockholm, Iljitsch van Beijnum 
wrote:

> solution with per-packet overhead over a stateful one that doesn't 
> have extra overhead.

I would like to argue that per-packet overhead is cheaper than stateful 
mappning.

- kurtis -

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 13 May 2003 11:10:01 +0200

On dinsdag, mei 13, 2003, at 10:11 Europe/Amsterdam, Kurt Erik 
Lindqvist wrote:

> What worries me as ex-operator is that state in a network comes at a 
> cost. Normally state is actually among the most costly components of 
> network design. So, I agree that I would like to avoid any stateful 
> mapping. Now, that said I agree that a stateless mechanism is going to 
> harder to design and implement.

I think we should more carefully define the word "state". A BGP routing 
table is state. An IPsec security association is state. An ARP table 
entry is state. A TCP control block is state. A socket bound to a UDP 
port is state. We use all of this (maybe with the exception of IPsec) 
every day, so apparently keeping all this state is worth it.

Assume we have A and B that want to communicate, and they have C and D 
sitting between them passing along the packets. There are also E - Z 
who are connected to A, B, C and D but they aren't involved in this 
particular communication. My position is:

State in A and B: perfectly acceptable. Applications and transport 
layer protocols keep state anyway.

State in C and D: not good, but not entirely inconceivable.

State in E - Z: unacceptable.

Which part were you talking about?

> I would like to argue that per-packet overhead is cheaper than 
> stateful mappning.

I'm sure this is true for you. However, I regularly connect to the net 
with my laptop over GSM at 960 bytes per second costing me 11 cents per 
minute, and when I do that every byte counts. If you're sitting behind 
gigabit ethernet where you can do 100000 packets per second regardless 
of how big they are, you'd rather throw away 10% of the usable data in 
each packet than add extra processing that halves the number of packets 
per second you can do. What I'm saying is that it is impossible to 
determine that globally, EVERYONE will always prefer either state or 
overhead.

Another important point is that the additional information can't be 
trusted so it must be authenticated. This means crypto, which is harder 
to do fast than state. Obviously some people will implement very fast 
crypto but for others even a moderate amount of crypto is extremely 
undesireable because either they simply don't have the CPU power or 
they don't want to use it (CPU costs energy, not nice when running on 
the battery).

But I think we should revisit this when there is an actual proposal 
that adds extra headers to packets because only then we'll be able to 
see how this helps us and how it hurts us.

Finally, I want to warn everyone that there is a very dangerous "who 
cares about the overhead" attitude in some circles. (Not just IETF, but 
also IEEE and others.) I did some calculations on VoIP over IPsec and 
it turns out that even with a factor 4 compression of the data, the 
total number of bits used by this is larger than regular TDM for voice. 
Don't forget we have more than 5 layers, if every one of those adds 3% 
overhead, that's 16% total. If they all do 5% it's 28% and if they all 
do 10% it's no less than 61%. It adds up very quickly. A good example 
of what can happen hee is 802.11g that has a 54 Mbps bitrate but 
delivers little more than 20 Mbps. (But that's partially because of 
some radio stuff obviously.)

---

From: marcelo bagnulo <[email-masked]>
Date: 13 May 2003 13:40:08 +0200

Hi,

> see how this helps us and how it hurts us.

Well, using MIPv6 could be one of such proposals (there are some issues
to solve) but still can be useful to quantify the amount of overhead
involved.
Regards, marcelo

-- 
marcelo bagnulo <[email-masked]>
uc3m

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 13 May 2003 14:53:53 +0200

On dinsdag, mei 13, 2003, at 13:40 Europe/Amsterdam, marcelo bagnulo 

> to solve) but still can be useful to quantify the amount of overhead

Mobile IP in IPv6 uses a 24 byte header to carry the original source 
address, right?

If we assume ethernet with a 1500 byte MTU and include the ethernet 
encapsulation (MAC addresses, ethertype and frame check sequence) but 
not the lower layer stuff (this is a good compromise between looking 
only at IP, which is slightly worse or also including the extra framing 
bits/gap, which is another 20 bytes per packet) and assume full sized 
packets we arrive at:

               IPv4   IPv6   MIPv6
Data          1460   1440   1416
Overhead        58     78    112
Efficiency1   96.2%  94.9%  93.3%
Efficiency2   94.4%  92.5%  90.3%

In the efficiency1 figure we drop 2.9% when moving from IPv4 to MIPv6 
but this figure doesn't include the TCP ACKs that nearly double in 
size. For a full duplex link this adds up to another 3% in overhead 
while this is only 1.8% for IPv4 so the overall drop in efficiency is 
4.1%.

Now if you're going to do high speed data transfers you'll obviously 
want to enable the RFC 1323 high performance extensions which means 
another 12 bytes per packet are used for timestamp landing you at 89.1% 
efficiency (93.2% in IPv4). And then there is IPsec...

In the real world all of this is a lot worse because the average packet 
size is significantly smaller than the maximum possible packet size.

Now all of this may not seem like a huge deal but what if the bank 
decides they want to take 4% of the money in your account, or look at 
the trouble people go through to lose 4% of their weight. For some 
people this may be a reason to stick with IPv4. On the other hand, if 
we can keep it at 4% it might be ok. But if the application protocols 
(which usually have HUGE overhead) and all the lower layers add 4% too 
then it's no fun anymore.

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Wed, 14 May 2003 09:23:39 +0200

On tisdag, maj 13, 2003, at 11:10 Europe/Stockholm, Iljitsch van 
Beijnum wrote:

>> mapping. Now, that said I agree that a stateless mechanism is going 
>> to harder to design and implement.
>
> I think we should more carefully define the word "state". A BGP 
> routing table is state. An IPsec security association is state. An ARP 
> table entry is state. A TCP control block is state. A socket bound to 
> a UDP port is state. We use all of this (maybe with the exception of 
> IPsec) every day, so apparently keeping all this state is worth it.

Absolutely, we do! But some of those examples cost more than others. 
And the total costs quite a lot. No need to add to this.

> Which part were you talking about?

Just to be clear in the example above, is A, B, C, D, E-Z sites, or 
nodes?
"
My answer would say that it depends. As (I think it was) Brian pointed 
out, you have a problem to "distribute the state" as well. And if that 
state should sit in in the sending/receiving nodes or the edge nodes, 
or in all of the nodes of the sending/receiving sites - is a good 
question. And one of the questions we need to solve.

My point was merely that I want to minimize state at all. We most 
likely will have to allow some state, but the cost of that state will 
depend on the protocol (type of state) and where that state is.

> with my laptop over GSM at 960 bytes per second costing me 11 cents 
> per minute, and when I do that every byte counts. If you're sitting 
> behind gigabit ethernet where you can do 100000 packets per second 
> regardless of how big they are, you'd rather throw away 10% of the 
> usable data in each packet than add extra processing that halves the 
> number of packets per second you can do. What I'm saying is that it is 
> impossible to determine that globally, EVERYONE will always prefer 
> either state or overhead.

True. I made a very general comment. And it is true that there are 
cases where conservation of bit's would be preferred, I my self use 
GPRS quite a lot and then I would also like to have the cheapest option.

However, this actually comes down to another issue. This is seen from 
the POV of the user. From the POV of the network operator, the cost of 
state will be higher. That cost will be taken out of the users, but it 
will not be as visible to the users as more bits would.

> cares about the overhead" attitude in some circles.

Agreed. But it's a trade-off.

- kurtis -