Subject: Re: ForCES Framework submission for AD Review

From: Alex Zinin <[email-masked]>
Date: Tue, 11 Mar 2003 17:44:30 -0800

David,

  I'm taking this thread to the rtg-dir list so I don't function
  as a relay. I'll be sending my comments in a separate e-mail.
  Regards.

-- 
Alex

Wednesday, March 5, 2003, 4:48:15 PM, Putzolu, David wrote:
> Hi Alex.

> My compliments to Chris Hopps - it is clear that he
> really did a thorough job reading this stuff and grasped
> the essentials quite well.

> My responses inline.

>> > - NIT: 8-bit set characters in draft.
>> > 
>> > - How does the architecture deal with multiple distinct CEs sharing
>> > a single FE. The framework mentions multiple CEs with an Fr link
>> > but these CEs actually implement a single logical CE.  Does this
>> > mean that the physical FE must present multiple virtual FEs if it
>> > is to be shared amongst multiple real CEs?  Perhaps this restriction
>> > should be pointed out.

> My understanding of having multiple CEs sharing
> a single FE is that it falls into two cases:
> * Either the FE is a physical FE that is logically
>   partitioned into multiple virtual FEs (GSMP wg has
>   a draft that does this). If this is the case, each
>   virtual FE only sees a single CE, so the issue becomes
>   one of defining a robust virtualization (not something
>   we are trying to do).  I remember discussions about this
>   in the WG, but can't find anything in the framework, so
>   this may be a missing item.
> * The other case is that the FE actually does have
>   multiple CEs controlling it.  In this case the approach
>   I understand they have taken is that they require that
>   the CEs must cooperate (via the Fr reference point) to
>   ensure that the FE receives consistent information via
>   ForCES.  This becomes in effect a multi-master control
>   problem, which is too difficult for FEs to be asked to
>   solve (as it would involve some sort of capabilities
>   or role-based scheme or other similar complexity). See
>   the 3rd paragraph of section 3.1.

>> > - The architecture does not (currently) define Fi 
>> (forwarding interface
>> > between FEs); however, if the goal is promote interchangeability,
>> > is this not critical to standardize at some level?

> Yes. Fi (and several of the other reference points defined
> by the framework) all are necessary to achieve complete
> plug and play.  ForCES is solving one part of the puzzle.
> When ForCES was chartered there was an awareness of these
> other problems but it was felt that a single WG was not
> the right place to address them all.

> Some of the necessary technologies are already happening
> in other places.  PCIsig is defining hardware and interconnect
> standards that can be used to interoperably put blades
> together.  NPF is defining a way to enable state sharing
> between blades of per-packet information.

> I would actually like your opinion and advice on this topic.
> In particular,  I believe it would be valuable for a real
> standards body like IETF (as opposed to a forum/sig) to
> define the entire standard.  At the same time, while some
> of the reference points (e.g. Fp, Fc, Ff) are likely to be 
> run across networks, some others (Fi) will probably remain
> "within" boxen for a long time, due to bandwidth/performance
> requirements (imho).  Are purely within-boxen technologies
> the right thing to do @ IETF?

>> > - The arch. seems to require single managers (CE and FE). Is there a
>> > need for an arbitration mechanism?  I'm imagining multiple 
>> blades (FEs)
>> > each with a "bus master" capability. I.e., N of M blades in an NE
>> > might be capable of being the FE manager.

> The FE and CE managers are logical entities, and their
> function is one of initial configuration of FEs and CEs
> to be able to find each other (section 1, bottom of p3).  
> So I'm not really sure I understand the bus master analogy.
> If there is a need for multiple managers they
> would probably need to arbitrate among themselves to decide
> who gets to give the FE or CE its initial config info.

>> > - in 5.4 there appears to be no mechanism mentioned to 
>> allow for the CE
>> > to block delivery of exceptional packets from the FE. Is 
>> this therefore
>> > susceptible to DoS attack?

> A DoS attack is a real possibility and the filtering 
> mentioned in para 2 and 3 of 5.4 would be used to drop 
> such packets when an attack is detected. In addition 
> to this, the requirements say that a priority mechanism 
> is necessary - this is needed to deal with a DoS attack 
> when it first happens (before filters are installed) 
> to allow prioritization of delivery of the packets 
> that install the filter on the FE over delivery of
> the packets being "aimed at" the CE. See section 7 requirement
> 5 in the requirements.

> I was unable to find any mention of this in the framework,
> but the framework is more about identifying architectural
> entities and functions than protocol details.  Should this 
> be included in the framework as well as the 
> requirements/protocol proposals?

>> > - allowing ospf hello and other portions of CE functionality to be
>> > implemented by FEs appears to open the architecture so wide it would
>> > be impossible to actually achieve the goal of interchangeable operation.

> I think a generic framework for offloading anything
> would indeed be too open ended (and would seem to me
> to be akin to defining some sort of virtual machine
> on the FE).  My belief is that it would be necessary
> to tackle such things on a case-by-case basis, i.e.
> define a standard way to do OSPF hello offload for
> instance.  If the problem is broken into manageable
> pieces then I think it is not too wide open.

>> > - does the architecture allow for non-stop forwarding? Perhaps this
>> > should be directly addressed somewhere.

> Good catch. Requirements doc section 5 (arch requirements) item 7
> mentions this.  Framework probably should as well.

>> > More general:
>> > 
>> > - The document mentions in a few places that ForCES will 
>> focus on the
>> > ForCES protocol only (the CE/FE communication channel). It 
>> does point out
>> > the other communication channels (and therefore possibly protocols)
>> > may exist or be required.
>> > 
>> > If the goal of ForCES is to allow for interchangeable 
>> operation of CEs
>> > and FEs (if it isn't why standardize it?), how can we not 
>> also plan on
>> > standardizing the other required protocols (at minimum the 
>> CE Mgr - FE Mgr,
>> > CE Mgr - CE, FE Mgr - FE, and FE - FE).

> Are these two questions the same as the Fi one up above?
> If so see my response above.  If not I'm having trouble
> understanding and could use some elaboration :)

> Note: I have responded to all of the above based on my
> personal understanding in an effort to get a prompt answer
> out.  Is it ok for me to share the feedback directly with
> the authors and/or WG (and if so, should it be anonymized?)
> I don't really know what IETF procedure is on this.

> Thanks,
> David

---

From: Alex Zinin <[email-masked]>
Date: Tue, 11 Mar 2003 18:03:16 -0800

David,

inline below, pls.

>   this may be a missing item.

I think this is what Chris meant.
I have no strong opinion on this topic, and would not
have a problem with ForCES addressing the simpler problem
first.

> the right thing to do @ IETF?

If we didn't have ForCES, I would say 'no' :)

Seriously, I think defining only the Fp protocol at this point is
fine. When/if we see a lot of interest in the community to define
others, we'll discuss this. The framework seems to correctly identify
the interface points, it's just that we choose the first step.

> who gets to give the FE or CE its initial config info.

Chris will correct me if I'm wrong, but I think what is meant
here is if, for example, the FE manager is collocated with
an FE and there are many of such FEs in an NE, how do they
choose whose the one... I would think this would be part of
the Fi interface.

> requirements/protocol proposals?

I think this could be included as a function within the general discussion
on sending packets from FEs up to CEs.

> pieces then I think it is not too wide open.

I share Chris'es concern here. I'd rather us not go try
to specify this, not in the first ver anyways...

The same comment applicable to sections 5, 5.2, 5.6.

> mentions this.  Framework probably should as well.

Section 4.3 talks a bit about this--

>         The example in Figure 5 is used to illustrate what happens when the
>         association is broken and later re-established again.  Section 3.2 
>         already explains what happens if CE1 fails and how CE2 can take 
>         over.  If no CE redundancy is provided, at the association 
>         establishment phase FEs need to be told what to do in the case of CE 
>         failure.  FEs may be told to stop packet processing all together if 
>         its CE fails.  Or, FEs may be told to continue forwarding packets 
>         for a limited time even in the face of CE failure.  No matter what 
>         the instruction is, it needs to be part of the configuration when 
>         the association is established.  

--but spelling this out may be a good idea.
Let's call it... hmmm... "graceful restart" for consistency :)

> I don't really know what IETF procedure is on this.

My recommendation would be to distill the comments on this list
and then bring an agreed upon set back to the WG.

Here's some more comments from me:

> 4.2.3. Steady-state Communication

>                      |(My port is down, with port #.)
>                     1|---------------------->| 
>                      |                       | 
>                      |(Route to this port instead...)                 
>                     2|<----------------------| 
>                      |                       | 

"route to this port instead" seems wrong, the CE would need to Ack and
then come back with updated FIB entries.

>      4.3. Association Re-establishment
[...]
>         In the same example in Figure 5, assuming CE1 is the working CE for
>         the moment, what would happen if one of the FEs, say FE1, leaves the 
>         NE temporarily?  FE1 may voluntarily decide to leave the 
>         association.  Or, FE1 may stop functioning simply due to unexpected 
>         failure.  In former case, CE1 receives a "leave-association request" 
>         from FE1.  In the latter, CE1 detects the failure of FE1 by some 
>         other mean.  In both cases, CE1 keeps a note of such event from FE1 
>         while continue commanding FE2.

not just "note", but also inform the routing protocols that will need
to recalculate their route which will result in FIB updates.

>         When FE1 decides to rejoin again, or
>         when it is back up again from the failure, FE1 needs to re-discover 
>         its master (CE).  This can be achieved by several means.  It may re-
>         enter the pre-association phase and get that information from its FE 
>         manager.  It may retrieve the previous CE information from its 
>         cache, if it can validate the information freshness. Once it
>         discovers its CE, it starts message exchange with CE to re-establish 
>         the association just as outlined in Figure 9, with the possible 
>         exception that it might be able to bypass the transport of the 
>         complete initialization information.

What about the security hand-shake though?

> Suppose that FE1 still has its 
>         routing table and other state information from the last association, 
>         instead of sending all the information again from scratch, it can 
>         choose to use more efficient mechanism to re-sync up the state with 
>         its CE.  For example, a checksum of the state might give a quick 
>         indication of whether or not the state is in-sync with its CE.  By 
>         comparing its state with CE first, it sends information update only 
>         if it is needed. 

This would require corresponding mechanisms in the protocol itself,
right?

>      9. Security Considerations

Ed: Move before refs?

Thanks,
Alex