Subject: [lisp] ETR connectivity to site - question on EID reachability (section 6.4)

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 09:05:57 -0700

Folks,

As Dino asked in his e-mail, I am starting a new threat on the topic.

>From the latest LISP spec:

 6.4.  EID Reachability within a LISP Site

   A site may be multihomed using two or more ETRs.  The hosts and
   infrastructure within a site will be addressed using one or more EID
   prefixes that are mapped to the RLOCs of the relevant ETRs in the
   mapping system.  One possible failure mode is for an ETR to lose
   reachability to one or more of the EID prefixes within its own site.
   When this occurs when the ETR sends Map-Replies, it can clear the
   R-bit associated with its own locator.  And when the ETR is also an
   ITR, it can clear its locator-status-bit in the encapsulation data
   header.

Let's consider a scenario where an ETR lost connectivity to its
site (note that the ETR may be either CE or PE). As a result, the
ETR lost reachability to all the EID prefixes within the site. In
this case the ITRs that are communicating with that site via that
ETR need to find out that the ETR lost connectivity to that site.

One possible solution proposed in the above is for the ETR to clear
the R-bit associated with its own locator in the Map-Replies. But
why would the ETR be sending Map-Replies in the absence of Map-Requests ?
And if the ETR sends Map-Replies in response to (received)
Map-Requests, then why would the ITR originate such Map-Requests
when the ETR loses its reachability to the site ?

Another solution proposed in the above is for the ETR to clear its
locator-status-bit in the encapsulation data header. But it applies
only when the ETR is also an ITR for the traffic going in the
opposite direction, which may, or may not be the case. What are the
procedures for indicating the loss of connectivity when this is not
the case. Moreover, even for the case where the ETR is also an ITR
for the traffic going in the opposite direction, wouldn't that
solution require the ETR to have some traffic going in the opposite
direction ? If yes, then how would the ETR get this traffic, given
that its connection to the site is down ?

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 10:46:50 -0700

> As Dino asked in his e-mail, I am starting a new threat on the topic.

Thanks for doing this Yakov, appreciate it.

>   ITR, it can clear its locator-status-bit in the encapsulation data

So note it says the site has the xTRs, so the context is respect to a  
CPE router doing LISP functionality.

> ETR lost reachability to all the EID prefixes within the site. In

Right, okay.

> ETR need to find out that the ETR lost connectivity to that site.

Right, with you so far.

> the R-bit associated with its own locator in the Map-Replies. But

Yes, for new ITRs that are requesting a mapping. For the ones that  
already have the EID-prefix cached, this would not do any good.

> why would the ETR be sending Map-Replies in the absence of Map- 

It would not for the ITRs that already had the EID-prefixed cached.  
But for these ITRs, when RLOC-probing is turned on, will probe the  
ETR's RLOC directly. The RLOC-probe is a Map-Request and the RLOC- 
probe reply is a Map-Reply. So for the ITRs that have the map-cache  
entry can see the R-bit set to 0 in the RLOC-probe reply and set the  
RLOC to be unreachable. Thereby, the ITRs can encapsulate to the other  
RLOCs in the locator-set (i.e. the ones that claim they are up and are  
reachable from the ITR's point of view).

> when the ETR loses its reachability to the site ?

RLOC-probing, TTL timeout, and responding to an SMR.

> locator-status-bit in the encapsulation data header. But it applies

The R-bit the LSBs convey the same information. The former is in the  
control-plane and the latter is in the data-plane.

> opposite direction, which may, or may not be the case. What are the

Right.

> that its connection to the site is down ?

By now, you can see I am being repetitive.  ;-)   RLOC-probing in the  
control-plane tells you the reachability.

Dino

>
> Yakov.
> _______________________________________________
> lisp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lisp

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 11:04:58 -0700

Dino,

> Thanks for doing this Yakov, appreciate it.

Welcome :-)

> CPE router doing LISP functionality.

Are you saying that a PE can not act as an xTR for a site connected
to that PE ?

> > ETR lost reachability to all the EID prefixes within the site. In

Good.

> > ETR need to find out that the ETR lost connectivity to that site.

Good.

> already have the EID-prefix cached, this would not do any good.

And since my e-mail is about only the ITRs that already have
the EID-prefix cached, for the scenario described in my e-mail
use of R-bit in the Map-Replies, as you said, "would not do any good".

> reachable from the ITR's point of view).

So, to cover the failure scenario described in my e-mail the ITRs
have to rely on the RLOC-probe mechanism.

> control-plane tells you the reachability.

The conclusion from this e-mail exchange is that for the failure
mode I described in my e-mail one can't rely on the use of
locator-status-bit in the encapsulation data header, as either (a)
the ETR may not be an ITR for the traffic going in the opposite
direction, or (b) the ETR may have no traffic to send in the opposite
direction. And the only thing which is left is the RLOC-probing.

With this in mind please add the following after the first paragraph
in 6.4:

  Another possible failure mode is for an ETR to lose connectivity
  to the site (e.g., due to the link failure between the ETR and 
  the site). The only mechanism specified in this document to
  detect such failure is RLOC-probing.

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 11:21:24 -0700

> Are you saying that a PE can not act as an xTR for a site connected

I am not saying that. I am indicating what the context of the above  
paragraph is.

> have to rely on the RLOC-probe mechanism.

That is one mechanism. As we told John last year, this is work in  
progress.

> direction. And the only thing which is left is the RLOC-probing.

That is your conclusion and you are not being creative enough.  ;-)   
Joking.

>  detect such failure is RLOC-probing.

We don't know that for sure and again, we are speculating.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 11:28:22 -0700

Dino,

> I am not saying that. I am indicating what the context of the above  

Please point me to the specific text in 6.4 that spells out that
the context of the above paragraph is the scenario where an ETR is
not a PE.

> That is one mechanism. As we told John last year, this is work in  

What is exactly "work in progress" ?

> That is your conclusion and you are not being creative enough.  ;-)   

Jokes are fine.

> We don't know that for sure and again, we are speculating.

What is that exactly that you "don't know for sure" ?

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 11:33:43 -0700

I am going to terminate this thread, we are going in circles.

Dino

---

From: Alia Atlas <[email-masked]>
Date: Mon, 23 May 2011 14:46:38 -0400

Dino,

A real problem is different assumptions that are not spelled out in the drafts.
Of course you've been in the guts of this for years and have those assumptions
as part of your model.  I, at least, do not - though I find
assumptions (right or wrong)
creeping in to try and explain bits that aren't fully specified in the drafts.

At the end of the day, a reasonably skilled practitioner should be
able to understand
and implement LISP correctly and to inter-operate.

If a fully specified mechanism, other than RLOC-probing, exists in the
drafts that will
handle failure of connectivity from an ETR to its LISP site, please
explain what that is??

I don't think we're circling yet.  From my perspective, you said
"RLOC-probing is what
deals with this", Yakov suggested text saying that, and then you said
"We don't know
that for sure and again, we are speculating."

A mechanism either is or isn't in the draft.  Of course there can and
will be future work
around LISP - but this is the set of drafts being discussed and last-called now.

Thanks,
Alia

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 11:56:51 -0700

> that for sure and again, we are speculating."

I am not hiding anything. The spec says to use RLOC-probing with both  
its advantages and disadvantages.

> around LISP - but this is the set of drafts being discussed and last- 

And maybe there is future work. I DON'T KNOW. Let's work on it so the  
future can become reality.

Dino

>>>>>> RLOCs in the locator-set (i.e. the ones that claim they are up  
>>>>>> and
>>>>>> are

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 12:12:43 -0700

Dino,

> I am going to terminate this thread, we are going in circles.

We are not going in circles. You are refusing to address legitimate
issues.

You still did not answer my question about the specific text in 6.4 
that spells out that the context of the paragraph in 6.4 is the scenario 
where an ETR is not a PE.

And in response to my request to document that at the present
moment the only mechanism specified in the LISP document to cover 
the failure mode where an ETR loses connectivity to its site is 
RLOC-probing, you said

If you think that the LISP document specifies some other mechanism
that *does* work, please spell this out. Otherwise, you just
speculating that some other mechanisms could be possible.

So, this thread must continue.

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 12:28:50 -0700

> We are not going in circles. You are refusing to address legitimate

I have been on email with you all day. I think I am cooperating.

> where an ETR is not a PE.

Can you ask a specific question please?

> the failure mode where an ETR loses connectivity to its site is

When the CPE box is an ETR-only. When it is an ITR-and-ETR, it can  
also use locator-status-bits.

> So, this thread must continue.

I know of no other mechanisms at this point in time.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 13:06:34 -0700

Dino,

> I have been on email with you all day. I think I am cooperating.

Same for me.

> > that spells out that the context of the paragraph in 6.4 is the  
> > scenario where an ETR is not a PE.
> 

Where is the specific text in 6.4 that spells out that the context 
of the first paragraph in 6.4 is the scenario where an ETR is CPE
(and not a PE) ?

> also use locator-status-bits.

And as I explained in my e-mails, even if it is an ITR-and-ETR,
there are cases where it can *not* use locator-status-bits.

> I know of no other mechanisms at this point in time.

With this in mind let me propose the following modification
to the text I suggested earlier (to include discussion on the use
of locator-status-bit):

   Another possible failure mode is for an ETR to lose connectivity
   to the site (e.g., due to the link failure between the ETR and
   the site). Use of locator-status-bit in the encapsulation data
   header to indicate this failure is applicable only when the ETR is
   also an ITR for the traffic going in the opposite direction,
   which may, or may not be the case.  Moreover, even if the ETR
   is also an ITR for the traffic going in the opposite direction,
   use of locator-status-bit in the encapsulation data header
   requires the ETR to have some traffic from the site going in the
   opposite direction.  However, due to the nature of failure, the
   ETR may not have such traffic (as the ETR lost connectivity to
   the site). Thus one can not rely solely on the use of
   locator-status-bit in the encapsulation data header to indicate
   this type of failure. The only mechanism specified in this
   document to detect such failure that is not subject to the
   limitations of the locator-status-bit mechanism is RLOC-probing.

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 13:27:10 -0700

>   limitations of the locator-status-bit mechanism is RLOC-probing.

You suggested something like this last summer and the authors and  
chairs decided to put in what is in section 6.4 today.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 13:37:50 -0700

Dino,

As you trimmed my original e-mail, you also trimmed the question
I asked. Namely, where is the specific text in 6.4 that spells out 
that the context of the first paragraph in 6.4 is the scenario where 
an ETR is CPE (and not a PE) ?

I am still waiting for your answer. So, please do not trim this again.

> chairs decided to put in what is in section 6.4 today.

Whatever "the authors and chairs decided" is not sufficient,
as the current text in 6.4 (a) does not describe the limitations
of relying on the locator-status-bit, and (b) does not spell out
that RLOC-probing is the only mechanism currently specified in the
document that does not have such limitations.

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 13:48:09 -0700

> an ETR is CPE (and not a PE) ?

8.2.  Border/Edge Tunnel Routers

    Using customer-edge (CE) routers for tunnel endpoints allows the EID
    space associated with a site to be reachable via a small set of  
RLOCs
    assigned to the CE routers for that site.  This is the default
    behavior envisioned in the rest of this specification.

...

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 13:53:04 -0700

Dino,

>     space associated with a site to be reachable via a small set of RLOCs

Does this mean that whenever draft-ietf-lisp-12 mentioned the term
"ITR" or "ETR", then unless stated otherwise, these terms imply
that ITR/ETR is a CPE router ?

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 14:02:33 -0700

> that ITR/ETR is a CPE router ?

We believe the sweet-spot for LISP is at the CPE.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 14:06:16 -0700

Dino,

> We believe the sweet-spot for LISP is at the CPE.

What you believe is "the sweet-spot" does not answer my question.

Yakov.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 14:22:40 -0700

> What you believe is "the sweet-spot" does not answer my question.

And therefore the spec emphasizes CPE.

Dino

---

From: Yakov Rekhter <[email-masked]>
Date: Mon, 23 May 2011 14:27:10 -0700

Dino,

> And therefore the spec emphasizes CPE.

You still avoid answering my question. Let me repeat my question:

Whenever draft-ietf-lisp-12 mentioned the term "ITR" or "ETR", then
unless stated otherwise, do these terms imply that ITR/ETR is a CPE
router ?

The *only* possible answers are "yes" or "no".

Yakov.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 23 May 2011 19:14:52 -0400

Let me try phrasing things a little differently, to see if it makes 
sense, or if we can determine that there is soemthing more that needs to 
be said, and if so, what.

The draft has a number of mechanisms for dealing with liveness / 
reachability detection.
These were added progressively, as folks in the WG noticed and worked on 
the problem.  (Dave Meyere wrote a draft on part of the problem just to 
help the WG understand the pieces of the problem.)

At this point, we have several mechanisms which cover different cases.
In the worst case, a client which uses RLOC probes has coverage at 
whatever rate he chooses to probe.
It is clear that the solution would not have necessary coverage for the 
experiment if we did not have RLOC probing.
But we do not have enough experience to mandate specific rates for such 
probing.
We also do not have enough experience to know what proportion of the 
time we will need to use it.
All of which in turn affects how well it will scale.

I would expect that the experiment will tell us some of that.

With all of that said, what more do you think it is useful to add?

Yours,
Joel

On 5/23/2011 2:46 PM, Alia Atlas wrote:
> On Mon, May 23, 2011 at 2:33 PM, Dino Farinacci<[email-masked]>  wrote:

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 16:42:01 -0700

Alia, I have enclosed the presentations I gave at the LISP working  
group meeting in Stockholm. Maybe this will help understanding of the  
design tradeoffs and decisions made for locator reachability algorithms.

You will find 3 of them here. One called "TCP-counts" was not put into  
the draft because feedback from the working group indicated it was not  
general enough. Turns out, it was the cheapest to implement. RLOC- 
probing is the most general and solves more problems but has scaling  
challenges.

Paul Francis, Scott Brim and Iljitsch van Beijnum, as well as many  
others were figuring out how to scale probing. Paul was thinking of a  
rendezvous point where probabilistically if an ITR could reach that  
point if could reach most of its ETRs so probes could be reduced from  
n to k, where n was 1 or 2 orders of magnitude larger than k.

Have a look at them in the order below. Because the first feeds the  
need for the second.

Dino

On May 23, 2011, at 4:14 PM, Joel M. Halpern wrote:

> sense, or if we can determine that there is soemthing more that  
> needs to be said, and if so, what.
>
> These were added progressively, as folks in the WG noticed and  
> worked on the problem.  (Dave Meyere wrote a draft on part of the  
> problem just to help the WG understand the pieces of the problem.)
>
> It is clear that the solution would not have necessary coverage for  
> the experiment if we did not have RLOC probing.
> But we do not have enough experience to mandate specific rates for  
> such probing.

---

From: Alia Atlas <[email-masked]>
Date: Mon, 23 May 2011 19:55:39 -0400

Dino,

Thanks - I will take a look at those presentations & reply tomorrow.

Joel & Dino,

What would be useful to say is a mixture of two things:
   a) These are the core mechanisms that are needed (and possibly
these are those that
       are expected to change/evolve during the experiment).  Even
saying these are the skeleton
       of mechanisms, but they'll be further defined during
experimentation would be helpful.

   b) Here are common questions and these are the mechanisms to address them.

The LISP drafts describe a wide range of techniques and mechanisms -
but it is hard to
tell what an expected set is or what piece of the problem-space each
is solving.  The drafts
are largely written from an individual mechanistic perspective, which
makes it harder to
tell the functionality that a particular box/role has to have or to
tell how the common questions
have been addressed.

In general, I can tell that you've been dealing with the same set of
questions over and over,
but that means the reasoning and trade-offs should be captured.
Honestly, I'd love to see a
framework draft - even if that happens after these documents progress.

Alia

On Mon, May 23, 2011 at 7:42 PM, Dino Farinacci <[email-masked]> wrote:
> Alia, I have enclosed the presentations I gave at the LISP working group
> meeting in Stockholm. Maybe this will help understanding of the design
> tradeoffs and decisions made for locator reachability algorithms.
>
> You will find 3 of them here. One called "TCP-counts" was not put into the
> draft because feedback from the working group indicated it was not general
> enough. Turns out, it was the cheapest to implement. RLOC-probing is the
> most general and solves more problems but has scaling challenges.
>
> Paul Francis, Scott Brim and Iljitsch van Beijnum, as well as many others
> were figuring out how to scale probing. Paul was thinking of a rendezvous
> point where probabilistically if an ITR could reach that point if could
> reach most of its ETRs so probes could be reduced from n to k, where n was 1
> or 2 orders of magnitude larger than k.
>
> Have a look at them in the order below. Because the first feeds the need for
> the second.
>
> Dino
>
>
>
>
>
>
>> Let me try phrasing things a little differently, to see if it makes sense,
>> or if we can determine that there is soemthing more that needs to be said,
>> and if so, what.
>>
>> the problem.  (Dave Meyere wrote a draft on part of the problem just to help
>> the WG understand the pieces of the problem.)
>>
>> We also do not have enough experience to know what proportion of the time
>> we will need to use it.

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 23 May 2011 17:34:57 -0700

I think we have done that already:

 > egrep -c experiment draft-ietf-lisp-13.txt
8
 > egrep -c "further study" draft-ietf-lisp-13.txt
2
 > egrep -c future draft-ietf-lisp-13.txt
7
 > egrep -c research draft-ietf-lisp-13.txt
4

Dino