Subject: RE: [Forces-protocol] PL/TML Interface

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Thu, 22 Apr 2004 11:30:52 -0700

Weiming,

I am not sure I completely understood your comment below. Are you saying
you prefer the implicit flag approach over all other approaches ? What
do you think about the 'Handwaving' approach proposed by David ?
I think this will enable us to make fastest progress on PL issues rather
than getting diverted on TML details. What do you think ? 

Jamal, any comments ?

Thanks
Hormuzd

-----Original Message-----

It also seems to me a little abnormal  to define a unit in PDU that is
for local
use. If we need to choose between explicit flags and implicit fields,
implicit
is also more aceeptable to me.

Cheers,
Weiming
----- Original Message -----
From: "Putzolu, David" <[email-masked]>
To: <[email-masked]>; <[email-masked]>

Hi all.

Still getting caught up on the discussions!

The protocol design team could define a PL/TML
API if desired.  However, since APIs are not
currently in the charter, we would have to make
a case to the ADs for adding it.  This might be
tough, because an intra-endpoint API is a
different problem than the once ForCES protocol
tackles.

ForCES protocol tackles FE/CE interoperability.
A PL/TML API would be tackling intra-FE and intra-CE
interop between an implementer of the ForCES PL and
the implementer of the interconnect.   This isn't a
bad idea at all - but the challenge would be convincing
the ADs that a standard API is needed here.
I'm not ruling it out - just letting you know that
we would need to make a good case for it to the ADs.

Using a implicit/PL header flags approach is also
feasible, although I'm a bit concerned in that I'm
not familiar with prior examples of such an approach.
Are there other cases where a protocol had header
fields to signal the protocol implementation on the
same endpoint, rather than to the other end of the
connection?

Finally, there is the option of simply handwaving
and saying that implementers will implement the
system such that the TML is signaled appropriately.
So for example:

The TML abstract definition section of the doc might say:
  "A multicast, unreliable message-oriented service
   is provided.  Each message delivered to this
   service includes an indication of which recipients
   should receive the message.  The service does not
   guarantee reliable delivery but does guarantee
   messages will not be corrupted and will be delivered
   in order.  This service is intended to be used
   for timely delivery of information."

The PL configuration section of the doc might say:
  "when adding an IP route to multiple FEs, the
   TML's multicast,  unreliable message oriented service
   is used to deliver the following message to each
   desired FE:
     ------------------------------
     | (list if FEid/LFBid pairs) |
     ------------------------------
     | new route                  |
     ------------------------------
   Each FE scans the list of FEids and LFBid pairs
   to see which of its LFBs should be given the
   new route structure"

And the Ethernet TML doc might say:
  "The multicast, unreliable message oriented service
   is implemented using Ethernet broadcast as follows:
   the PL indicates the list of FEs and passes
   down a PDU...the TML uses Ethernet broadcast
   and the following header listing which FEs should
   parse the message and includes a CRC and message
   number for ensuring non-corruption and ordering:
     ---------------------------------
     | FEid,                         |
     | ...                           |
     | FEid                          |
     ---------------------------------
     | CRC-32                        |
     ---------------------------------
     | Message #                     |
     ---------------------------------
     | Payload to deliver to each FE |
     ---------------------------------

Finally, a PL implementer might implement it as a
generic (but proprietary) API that they define and
share with customers, e.g.
  "FooCorp's implementation of ForCES includes a
   TML abstraction API allowing customers to use
   any interconnect.  The API that an interconnect
   must support for FooCorps' implementation to work
   for multicast unreliable connections is
   void FooSendMulticast(addressee_type *addrAray,
        messageType m)"
(In this case m would correspond to the PL-defined
PDU shown in the PL config section above).

Sorry to jump in to the technical discussion -
my original intention was to just clarify that we would
need to go to the ADs to define an API, and to ask if there
was precedent for using protocol header bits to signal
within an end point (as opposed to other endpoints).
I just can't resist a good technical discussion :)

-David

_______________________________________________
Forces-protocol mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/forces-protocol

---

From: "Wang,Weiming" <[email-masked]>
Date: Fri, 23 Apr 2004 12:00:38 +0800

Hormuzd,

My understandind of handwaving approach David presented below using multicast
add route as an example is that the FEid(s) and the message type is used as a
waving signal to TML to implicitly ask for the defined service in the TML.  This
is opposite to the explicit approach in that the latter may use an explicit flag
to indicate such service requirement. Do I misunderstand something here?

Actually my another thought (as I posted before) is to let the protocol layer
more flexible to use TML by allowing some configurations from Pl to TML based on
the TML capabilities . The configuration may be some thing like "ple use #xx
service template for message type xx when the message asks for multicast". This
approach is actually consistent with the handwaving approach, only with more
flexibility in that protocol customers may have more choices other than that
defined by TML requirements. I think this is also useful for more flexible
vendor supports.  But I just don't want to push this too hard now because I
agree with you that we now need to move forward much more quickly.

Anyway, I have no doubt  to use the handwaving approach. I'm also anxious to
hear what Jamal thinks.

Best regards,
Weiming

----- Original Message -----
From: "Khosravi, Hormuzd M" <[email-masked]>
Weiming,

Jamal, any comments ?

Thanks
Hormuzd

-----Original Message-----

Cheers,
Weiming
----- Original Message -----
From: "Putzolu, David" <[email-masked]>
To: <[email-masked]>; <[email-masked]>

Hi all.

-David

---

From: Jamal Hadi Salim <[email-masked]>
Date: 23 Apr 2004 08:42:41 -0400

Weimig/Hormuzd,
Apologies for disappearing - important customer fire.

> to indicate such service requirement. Do I misunderstand something here?

I think it is implicit unless i misread it. I dont see the PL worrying
about sending a list of FEids though. I think that should be the TMLs
problem. PL sends a single address (CE/FEid), the TML may translate or
rewrite it.
To take an example of a PL sending a broad/multicast FEid:
-if TML is capable of broad/multicast, the single message is sent.
- if TML is not capable of broad/multicast, then it will take that FEid,
do an internal lookup, come up with a list of unicast FEids and
unicast-send to each. 

> service template for message type xx when the message asks for multicast". This

I think that message type alone is insufficient. Since message type is a
static item, we need flags to override default behaviors. 
To borrow an example from netlink2, we used the ACK flag to indicate
that a message/transaction was to be reliably delivered.
Even on an example of an ADD ROUTE, if i choose not to have a specific
ADD ROUTE reliably delivered for whatever reason, i should be able to
override the message type decision by NOT setting the ACK flag.
Corrollary: In a message type where reliability is not needed, i should
still be able to request for a specific message to be reliably delivered
by setting a flag such as the ACK.

TMLs should be defining something of the nature:
1) This TML is capable of both unicast and multicast
2) When a message has to be delivered reliably, operation is:
send on multicast UDP, if no response send on TCP and wait for upto y
seconds for a response. Notify PL on success if it asked for ACK. Notify
PL on failure always.
3) etc

> Anyway, I have no doubt  to use the handwaving approach. I'm also anxious to

I agree we need to move past this IO stall but lets verify that the
implicit approach will work. Heres an attempt at listing the
requirements:

1) Reliability:
A ACK flag on the header should be sufficient on the header.
2) Congestion control:
I think this is a behavior of the TML that PL should have no control
over, no?
3) HA:
We may decide we want the PL to make decisions. Nothing in the protocol
header is necessary; events from TML<->PL are necessary though

4) Security
Like congestion control, PL has no say.

5) Uni/multi/broadcast addressing/delivery
The FEid/CEid is all the PL cares about. The TML takes care of the rest.
Somehow the PL needs to be notified of the semantics of the IDs.
Or we could come up with a scheme like the one we started discussing
much earlier on to have specific IDs allocated for different things.

6) Timeliness:
Not sure how at this point. But one should be able to say
"obsolete the message if it doesnt make it in 200ms".

7) Capability exchange
I think this is again like CC and security the responsiblity of the
TML??

8)The FEM/CEM interface
TML responsibility

9) Batching:
Responsibility of PL and PL alone?

10) transactional operations (2pc etc)
Responsibility of PL and PL alone?

I may have missed others.

Overall, it seems to me like the implicit approach should work. 
There are still some holes for example when i wanna ask for "just a
little more from the TML" like #6 above. Another "just a little more"
example would to say offload something like a LFB heartbeat from the PL
and tell the TML:
"I want you to send this msg unreliably, if you dont receive a response
two times, i want you to attempt to deliver it reliably with two more
retransmits and then tell me when that fails, otherwise i am not
interested but i will ask you later for all failure stats"
OK, that was long - but you get my message.

Thoughts?

cheers,
jamal

---

From: "Wang,Weiming" <[email-masked]>
Date: Sun, 25 Apr 2004 14:05:31 +0800

Jamal, Hormuzd,

----- Original Message -----
From: "Jamal Hadi Salim" <[email-masked]>

> do an internal lookup, come up with a list of unicast FEids and

[Weiming] Agreed.
The only remained question to me is how and when the multicast FEid/CEid is
configured. We know it is not a problem to the broadcast FEid/CEid, which can be
predefined by CEM/FEM as for the actual FEs/CEs address mapping table, but my
original thought on FE multicast is it is a dynamical one, that is, the mapping
to actual groups of CEs/FEs may be setup or deleted according to system need. If
so, then I still can not catch how and when this can be done? Shall we say that
the multicast can only be preset, just the same as for broadcast case?  Or,
shall we say the CEM/FEM will always be running all the time, even during the
post-assiciation phase, so that the multicast mapping table can be set on the
fly?
[/Weiming]

> static item, we need flags to override default behaviors.

[Weiming]Agreed. Actually I think we now have had three items considered as
static implicit signals: the message type, the FEid/CEid, the priority flag(I
suppose).

> by setting a flag such as the ACK.

[Weiming]Although I agree adding ACK with TML meaning is of help to more
flexible TML use, I just a little worry if such will add complexity for
customers, that is, they should be very familiar with and always remember the
ACK TML meaning when they use the flag. I wonder if in the end they may tend to
only remember original meaning of asking for ACK feadback. I also wonder if
there is a scenario where they only want to have the ACK meaning for the flag.
Anyway, we can have this in mind and move forward.
[/Weiming]

> seconds for a response. Notify PL on success if it asked for ACK. Notify
[Weiming]Agreed to 1), and partially to 2). I think we may be able to define
more than one service template like 2). When reading 2), I also think of another
issue. That is, if we allow a service template to have parameters like the 'y'
in your 2). This may add more flexibility but will eventally make some implicit
waving approach unable to do so.

> A ACK flag on the header should be sufficient on the header.
[Weiming]I also agree Hormuzd's idea that message type can surely show some
reliability requirements to TML.

> I think this is a behavior of the TML that PL should have no control
[Weiming]If CEM/FEM have the ability to setup some congestion control policies,
I think this is ok for PL not to do more.

> header is necessary; events from TML<->PL are necessary though
[Weiming]Shall PL takes some actions when it receives events from TML regarding
HA? Or can we use a HA policy attribute to reconfigure the HA in TML based on
the events?

> Like congestion control, PL has no say.

[Weiming]Agreed.

> much earlier on to have specific IDs allocated for different things.

[Weiming]I more incline to the schem the FEid/CEID care about everything on the
U/M/B in PL. If we can ask the CEM to notify CE, and FEM to FE,  of the IDs
semantics?

> "obsolete the message if it doesnt make it in 200ms".

[Weiming]If the TML policy scheme (as FE attribute that can be set by PL) can be
used, I think this can be resolved.

> I think this is again like CC and security the responsiblity of the

[Weiming]The capability exchange between TMLs should not be seen by PL. But if
you also mean capability change, I think PL should know something.

> Responsibility of PL and PL alone?
[Weiming]Agreed to above all.

> OK, that was long - but you get my message.

[Weiming] as my comments on #6, if TML policy is used, may help in some way for
this.

> Thoughts?
[Weiming]Seems we have made some progress.

Cheers,
Weiming