Subject: Re: comments on draft-py-multi6-gapi-00.txt

From: "J. Noel Chiappa" <[email-masked]>
Date: Wed, 5 Feb 2003 11:02:15 -0500

> From: Iljitsch van Beijnum <[email-masked]>

    > there is only one other objective way of pre-allocating address space:
    > on geography. 

Pre-allocating routing-names in a global-scale network without controlled
connectivity is an idea so broken I won't even bother to say anything more
about it.

If you want to pre-allocate some other kind of name (say an identifier).
that's reasonable.

	Noel

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 5 Feb 2003 17:39:48 +0100 (CET)

> connectivity is an idea so broken I won't even bother to say anything more

Without pre-allocation there can't be any successful aggregation. We can
see this in IPv4 today: the RIRs (pre-) allocate blocks that are too
small to ISPs so these ISPs end up with lots of relatively small blocks.

---

From: marcelo bagnulo <[email-masked]>
Date: 05 Feb 2003 17:44:36 +0100

> If you want to pre-allocate some other kind of name (say an identifier).

I guess this is how MHAP uses GAPI. Geographic Identifiers.

Regards, marcelo

---

From: Joe Abley <[email-masked]>
Date: Wed, 5 Feb 2003 13:08:42 -0500

On Wednesday, Feb 5, 2003, at 11:39 Canada/Eastern, Iljitsch van 

> small to ISPs so these ISPs end up with lots of relatively small 

Organisations move, merge and split apart. Physical and layer-2 
networks can change radically with no impact on layer 3. There are 
operators today selling wide-area layer-2 transport services in which 
single subnets span continents. Layer-3 topologies (both intra-AS and 
inter-AS) change on a daily basis.

Given such a turbulent soup of connectedness, what criteria for 
pre-allocation of routing names stands a chance of not being 
out-of-date as soon as it is published?

[There was no pre-allocation with IPv4, and there is certainly *some* 
successful aggregation today, despite what you say, and even given the 
allocation issues you mention.]

Joe

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 5 Feb 2003 21:53:16 +0100 (CET)

On Wed, 5 Feb 2003, Joe Abley wrote:

> > Without pre-allocation there can't be any successful aggregation. We
> > can see this in IPv4 today: the RIRs (pre-) allocate blocks that are too

> inter-AS) change on a daily basis.

People keep themselves busy. So?

> out-of-date as soon as it is published?

Let's take the speed of light for this. You can change many things about
your network, but not the fact that making packets take a 10000 km
detour adds a 50 ms delay. Also, if you're going to physically move your
network renumbering is only a minor extra task in addition to everything
else that's going on.

> allocation issues you mention.]

ISPs get /19s, /20s and /16s because it is assumed they'll need more in
the future than the single /24 they're requesting today. This is what I
mean by pre-allocation.

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 06 Feb 2003 11:59:51 +0100

below...

> the future than the single /24 they're requesting today. This is what I

Yes, but that is in a CIDR context with *topological* aggregation.
You seem to imagine that geographical allocation will somehow
match up with topology and therefore cause aggregation. All
historical evidence suggests otherwise.

If we could pass a law that every local government in the world
had to set up a monopoly IXP for its area, we might be able to
obtain enough congruence between geography and topology. But I
don't expect that any time soon.

Can't we just accept that only a two-layer solution that separates
identfier-addresses from locator-addresses can solve this puzzle,
and move on to figure out the two-layer solution?

   Brian

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 6 Feb 2003 14:05:46 +0100 (CET)

> historical evidence suggests otherwise.

Historical evidence shows a trend of interconnection getting denser (but
slowly). I don't see why we shouldn't take advantage of that where
possible. If we use geographic addressing for multihomers there is the
potential that we can later aggregate on geography if there is some
level (not necessarily a very high level) of interconnection within
regions. If we use another mechanism to allocate address space for
multihomers we don't have this option. As there doesn't seem to be any
other way to allocate address space for multihomers that has other
benefits (other than being slightly simpler) I can't see any
justification to NOT allocate them geographically.

> don't expect that any time soon.

100% interconnection within regions is not required: if there is
interconnection, fine, we can aggregate. If there isn't, also fine, but
no aggregation for multihomers in that region. This is a numbers game:
most of the large networks interconnect in enough locations to be able
to aggregate to a usable degree. And presumably, this will get better
rather than worse over time. At some point it may even make business
sense to interconnect more rather than buy bigger hardware.

> and move on to figure out the two-layer solution?

I'm all for that but I think we need something short-term as well.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 6 Feb 2003 16:02:26 +0200 (EET)

I'd like to expand on one particular point:

> don't expect that any time soon.

.. IXP does not have to have a monopoly AFAICS, more specifics can be 
advertised in every IXP.

What is more troublesome is that the IXP business model must be so that it
has upstream connectivity, and advertises the aggregate to the Internet.  
And that the IXP has sufficient redundancy so people would not have to
work around its back..

"Neutral" IXP's without even AS numbers are not sufficient.  I'm not sure
how it is out there, but here in Finland that's the only model for IXP's.

The only way to make it "work" without IXP's would be building a billing
model like the telephone transit (AFAIK) for these PI aggregate
advertisements (paid by those who connect the organizations and have the
more specifics).. and we know how well billing models work in the
Internet.....

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Joe Abley <[email-masked]>
Date: Thu, 6 Feb 2003 09:36:04 -0500

> .. IXP does not have to have a monopoly AFAICS, more specifics can be

This assumes there will always be IXPs. Today, the "IXP business model" 
is a concept of the developed world, and even there it is not relevant 
when we consider the largest ISPs, for whom peering is most definitely 
a business decision rather than a technical one.

 From Kathmandu, Nepal, traffic to India travels to Singapore by 
satellite, across the pacific to the US, across the US, across the 
Atlantic, up to another satellite, and down to India. Packets go round 
the world once and into space twice, despite the fact that Nepal and 
India share a geopolitical border.

Interconnecting countries in South Asia and other developing regions is 
most definitely a governmental policy issue, and not a technical issue 
(the technical case for direct interconnection is trivial, and yet 
there is no direct interconnection). It might take governments twenty 
years to decide that fibre across the land border is a good idea, or 
they might not decide to do it at all.

Any routing system that relies on (geopolitically) local 
interconnection better be able to accommodate frequent departures from 
that model. There are a lot of people in South Asia, and the 
infrastructure is growing a lot faster than governments are able to 
think, never mind legislate.

Joe

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 6 Feb 2003 16:48:27 +0100 (CET)

> has upstream connectivity, and advertises the aggregate to the Internet.

What are you talking about???

This statement is incorrect both when applied to the current situation
(at least in the parts of the world I'm familiar with) and also when
applied to my draft.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 6 Feb 2003 17:15:30 +0100 (CET)

> they might not decide to do it at all.

Both goverments and business make bad decisions at times. Only
governments usually need more time to make them.

However, there may not be a technical case for interconnection. Here in
the Netherlands around 45% of all traffic is to the US (this includes
stuff that is only reachable through the US), 45% is domestic and 10% to
the rest of Europe. Obviously these aren't very hard figures, but each
time I checked it was pretty close. Under these circumstances it doesn't
make much sense for non-huge networks to have direct connections to
neighboring countries: reserving some extra capacity on the US link
makes much more sense as it is virtually free, no extra effort and
better burstability. Now recompute with much higher costs for links to
neighboring countries and lots of government intervention.

> think, never mind legislate.

In the US there isn't much local interconnection as not much traffic
stays local. In most other countries, much more traffic stays local
because the countries are usually simply smaller and because of language
barriers. So now we have examples with little local interconnection and
examples with little regional interconnection. I'm still waiting for
examples of significant internet use with no local OR regional
interconnection.

Not that any of this is of practical importance with regard to the
decisions we are facing.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 6 Feb 2003 22:27:44 +0200 (EET)

> (at least in the parts of the world I'm familiar with) and also when

Who will advertise the aggregate for e.g. a country if not the IXP?  Why 
would any ISP do it an attract traffic it gets zero payment for?  The only 
possibility seems to be leaking all the more specifics or the IXP having 
upstream connectivity?

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 6 Feb 2003 22:18:27 +0100 (CET)

> would any ISP do it an attract traffic it gets zero payment for?

Excellent point. From the draft: "Note that this
behavior is completely hidden from the peer: the aggregates are only
used within the local network, they are not announced to peers. To avoid
confusion with regular provider aggregatable routes, the term "pilot
routes" will be used for this type of private aggregates."

Simply reading the draft will clear up most (all?) of this confusion.
It's 10 pages, so this should be doable:

http://www.muada.com/drafts/draft-van-beijnum-multi6-isp-int-aggr-00.txt

(Note that the term IX doesn't even appear in the draft.)

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 07 Feb 2003 14:03:07 +0100

> Not that any of this is of practical importance with regard to the

On the contrary, it all illustrates that there are very limited
economic forces driving the topology towards geographically-based
interconnects. In fact, we basically don't understand what caused
the various kinks in the bgp.potaroo.net curve, except for the one
at 20k announcements (i.e. the BGP4 switch on).

I can't disagree with the assertion that *if* we start handing
out PI prefixes, a loosely geographical or metro-based solution
is probably better than random numbers. But until we actually
know how PI prefixes are going to become routeable without
creating a swamp, I'd rather sit on my hands.

   Brian

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Feb 2003 20:12:35 -0500

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "Pekka" == Pekka Savola <[email-masked]> writes:
    Pekka> I'd like to expand on one particular point:

    Pekka> On Thu, 6 Feb 2003, Brian E Carpenter wrote:
    >> If we could pass a law that every local government in the world had to
    >> set up a monopoly IXP for its area, we might be able to obtain enough
    >> congruence between geography and topology. But I don't expect that any
    >> time soon.

    Pekka> .. IXP does not have to have a monopoly AFAICS, more specifics can
    Pekka> be advertised in every IXP.

    Pekka> What is more troublesome is that the IXP business model must be so
    Pekka> that it has upstream connectivity, and advertises the aggregate to
    Pekka> the Internet.  And that the IXP has sufficient redundancy so
    Pekka> people would not have to work around its back..

  Yes. 
  The key is that the amount of bandwidth can be limited if the traffic has
to move to a PA address.

  The ideal way is that every ISP that peers at such an IXP advertises this
aggregate. 

  The potential to abuse such an advertisement must be dealt with. Maybe
by making the MTU very low... 90 bytes maybe, and rate limiting by 
destination address. 

]       ON HUMILITY: to err is human. To moo, bovine.           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] [email-masked] http://www.sandelman.ottawa.on.ca/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [

---

From: Pekka Savola <[email-masked]>
Date: Mon, 10 Feb 2003 09:27:45 +0200 (EET)

> http://www.muada.com/drafts/draft-van-beijnum-multi6-isp-int-aggr-00.txt

I'd read it before, but I did not understand certain unstated assumptions.

I've glanced through it again.

This seems to solve a non-problem, that is, how PI addresses are handled
inside an ISP or between its peers.

The real problem this does not seem to take a stance on is (but there is
some text in section 10, not sure if related), who is advertising the PI
routes to the default free zone (or upstreams, in any case)?  If these are
not aggregates, would explode.  If these are aggregates, the upstream and
the ISP will have handle traffic that does not belong to it, because by
the definition of geoPI addressing, it is not completely aggregatable --
else it would not be useful at all for multihoming etc. purposes.

I guess the goal(s) of the draft should be stated as clearly as possible,
in the abstract or the introduction section.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 10 Feb 2003 09:31:20 +0100 (CET)

> not aggregates, would explode.

I think this point was made clear enough but apparently not. In
principle everyone announces all their customer routes to everyone.
From a router perspective it is more efficient to decide at the source
which routes are announced where, but this assumes that a network always
knows where a peer needs this information. This means that if you
monitor the global routing table using some kind of route collector
you'll see a huge routing table. This can work because networks simply
filter out routing information they have no use for at this location. So
a router in London ignores routing information for US destinations.
Packet for those destinations are sent to a router in the US that in
turn has no use for European routes. (Note that this other router
belongs to the same AS or a transit ISP.) In other words: because there
are no aggregates announced to other networks and more specific routes
are ignored to keep the routing tables manageable, networks must
transport the traffic to a place in their network where more specific
information is known.

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Mon, 10 Feb 2003 11:57:51 +0100

> network renumbering is only a minor extra task in addition to 

I was part of doing one of the largest network mergers ever. I am glad 
we didn't have to worry more about renumbering that what we already had.

You don't want to renumber as well as merge routing.

- kurtis -

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Mon, 10 Feb 2003 12:41:48 +0100

> the rest of Europe. Obviously these aren't very hard figures, but each

 From KPNQwest we saw traffic that was 50% national, 30% regional and 
the rest US based. At least for the larger language regions, CH, AT & 
DE + NO, SE & DK. BE + FR showed similar patterns.

> neighboring countries and lots of government intervention.

Actually this is most likely the largest network mistakes made. KQ was 
one of few operators who actually could move traffic on the same DWDM 
systems inside a language region in Europe. This helped us a lot with 
designing our IP network. I know other carriers that had to send 
traffic from Stockholm to Oslo via Amsterdam. That does not scale.

> stays local. In most other countries, much more traffic stays local

Huh?

> interconnection

See Joes India/Nepal example.

> Not that any of this is of practical importance with regard to the

It will/might impact routing models.

- kurtis -

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Feb 2003 20:08:39 -0500

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "Brian" == Brian E Carpenter <[email-masked]> writes:
    >> ISPs get /19s, /20s and /16s because it is assumed they'll need more
    >> in the future than the single /24 they're requesting today. This is
    >> what I mean by pre-allocation.

    Brian> Yes, but that is in a CIDR context with *topological* aggregation.
    Brian> You seem to imagine that geographical allocation will somehow
    Brian> match up with topology and therefore cause aggregation. All
    Brian> historical evidence suggests otherwise.

    Brian> If we could pass a law that every local government in the world
    Brian> had to set up a monopoly IXP for its area, we might be able to

  For it to be useful, it suffices that *some* local governments will setup 
such an IXs. 

    Brian> Can't we just accept that only a two-layer solution that separates
    Brian> identfier-addresses from locator-addresses can solve this puzzle,
    Brian> and move on to figure out the two-layer solution?

  I strongly agree with you.
  I am in favour of a two-layer solution, and am pretty sure that the outer
layer will be v6-in-v6 or ESP.

  GAPI becomes simply *one* way to get a locally controlled number space for
the identifier-addresses.  

  MIPv6 can provide a way to map identifiers to locators. 
  If you don't want that, you can use DNS reverse map.  (Hey, you can get
the nodes' public key along the way and encrypt by default)
  If not that, then find a new database.