Subject: RE: Re: draft-ietf-forces-requirements-08.txt

From: "Putzolu, David" <[email-masked]>
Date: Tue, 8 Apr 2003 10:07:57 -0700

(Apologies for the previous empty email. Outlook has
decided that when I use the "offline" mode that certain
emails should go out w/out any body text.  I am resending
this in online mode, hopefully it will go through.)

---

Thanks for the feedback - responses inlined...

> Comments on forces-reqs-08.  In summary, I'd like to express 
> a very grave
> concern of trying to address every possible problem, a path that seems
> doomed to failure.  Note a list of dozen different forwarding 
> functionalities that still fail to capture all of them -- and the 
> associated complexity of handling all the different nuances 
> and orderings 
> of these functions.
> 
> Seems way, WAY too complex to make interoperable and useful.  
> I wonder how
> far we have to go with this effort to see it as a dead-end.  Or
> alternatively, some other architectural/approach revamp might 
> be able to
> salvage some of the mechanisms (e.g. concentrate on certain core tasks
> only, but then again, one might question how useful the mechanism is).

I agree that trying to define all possible features is
infeasible, as the list is ever-evolving.  I also agree
that a reduction to a single feature (e.g. only address
IPv4 forwarding and nothing else), while feasible, is
useless.

The initial goal of forces is to try to capture that 20%
of features/configurations that applies to 80% of the 
applicable scenarios.

> 1) it is not clear how different efforts (most of which 
> referenced in the 
> draft) interconnect: e.g. midcom (similar protocol between hosts, 
> typically), gsmp, ccamp.

This seems like a framework question, not a requirements
question. That being said, I will answer as best I can.
Midcom is primarily focused on NAT and ALGs. One could
construct a NAT box using CE/FE separation. If that is the
case, ForCES might be present.  Midcom would deliver information
to the CE (e.g. "open this port for a RTP stream as part of
this H.323 call"), the CE would then configure the FE appropriately
("add ACL entry: pass ports 3310/3311 between hosts A and B").

GSMP controls switches. The GSMP authors are following ForCES
pretty closely (one is a co-author on the requirements).  Thus
far ForCES has focused on CE/FE separation routers (not switches).
Could GSMP be used to control FE routing functions?  Yes, but the
GSMP authors have so far not indicated a desire to extend GSMP for
that.  Could ForCES be used to control FE switching functions?
Yes, but that would be extending the scope of ForCES significantly.
Is routing enough to satisfy the 80/20 rule, or do you think that
switching must be thrown in as well?  Sounds like a scope/charter-
level change, so I'm hesitant to jump into that debate.

CCAMP is everything from GMPLS to LMP to OSPF extensions.  A few
of the things CCAMP is relevant to might show up in routing FEs,
seems to me more would fall into the GSMP lap.

> 2) I find the justification a bit questionable:
> 
>                       A standard set of mechanisms for connecting
>    these components provides increased scalability and allows the
>    control and forwarding planes to evolve independently, thus
>    promoting faster innovation.
> 
> .. it seems to me that in practice, such independent evolution and 
> innovations typically require changes in both elements: I 
> doubt one can 
> create such a generic architecture which would accommodate anything 
> happening to the other in the future.

Anything talking to anything is impossible.
A CE talking to successively faster generations of FEs
with IPv4 forwarding and DS support, using the same
standard ForCES protocol, is possible.

> Again in:
> 
>    By allowing the control and
>    forwarding planes to evolve independently, different types of FEs
>    can be developed - some general purpose and others more 
>    specialized.
> 
> .. this seems a bit questionable: different types of FEs can 
> already be
> developed, and have been developed for a long time now.  What you're
> aiming for is mixing and matching a CE one or more FE's with 
> a standard protocol with different characteristics and possibly 
> different vendors.

Within the context of the 80/20 rule mentioned above, yes.

> 3) the applicability of ForCES seems to be unclear, for example:
> 
>    Addressable Entity (AE) - A physical device that is directly
>    addressable given some interconnect technology.  For example, on IP
>    networks, it is a device to which we can communicate using an IP
>    address; and on a switch fabric, it is a device to which we can
>    communicate using a switch fabric port number.
> 
> .. the latter would seem to indicate some support for non-IP 
> devices too, 
> or should this use some rewording to clarify which kind of 
> switch fabric was the question was all about?

ForCES will define a model that applies to FEs that are
network-connected and backplane/fabric connected.  
ForCES protocol will then run over IP, ForCES model will be
likely reused for non-IP CE<->FE link scenarios (analogous to 
GSMP over Ethernet/GSMP over ATM/GSMP over IP).   AE is an
effort to get a term which works equally well in both cases.

> 4) I haven't delved into ForCES, but the last two sentences 
> look a bit 
> scary:
> 
>    Forwarding Element (FE) - A logical entity that implements the
>    ForCES protocol.  FEs use the underlying hardware to provide per-
>    packet processing and handling as directed by a CE via the ForCES
>    protocol.  FEs may use PFE partitions, whole PFEs, or 
>    multiple PFEs.
> 
> .. this seems to imply that CE's instruct logical FE's to 
> perform some 
> tasks, while there exists some protocol between FE->PFE.
> 
> - what is used between FE and PFE's?
> - are there interoperability reqs between FE / PFE's (can 
> they be from 
> different vendor)?
> - is this useful at all?  Seems like very dangerous to me -- 
> because might 
> require an additional abstraction layer at FE's -- which 
> might make the 
> CE-FE control even more useless (too generic) and less powerful.

I agree the wording is bad and it should be fixed.  What should
be coming across here is that a FE is what a CE controls.  If
under the covers the FE happens to be a single blade, a partition
of a blade, of two physical blades that through proprietary magic
appear as a single very reliable blade to the CE,  is beyond the
scope of concern of forces.

> 5) Is the partitioning concept really useful?  Could you list some 
> applicability scenarios?

I will provide some scenarios because you ask, but I personally
am not comfortable arguing for it.  The main point I'd like to 
make is that ForCES should be agnostic about partitioning issues,
and forcing ForCES to define/care about/create partitioning would
be an undesirable complexity.

Potential applicability scenario: Service provider wants a
capability to offer hard guarantees to customers that they
get access, performance, and control of what looks like their
own virtual router.  SP can either buy a physical box per
customer (expensive) or partition a single physical box into
logical partitions.

Again - this isn't and shouldn't be for ForCES to spec out.
Main point here is that ForCES shouldn't care if somebody
does partitioning under the covers ahead of time - a FE is a FE.

> 6) Is it possible to have multiple CE's manage one FE?  This probably 
> would have tradeoffs and drawbacks, but is integral when you 
> consider the applicability of the protocol (ie. is it a network management

> protocol or control-dataplane protocol).

Yes - multiple CEs may control one FE.  When this happens the CEs
had better make sure to coordinate among each other so that they
don't give contradictory orders to the FE.

Simple case: 1 CE talking to FE. 
Possible case: 2 CEs, from same vendor, speaking proprietary
 CE<->CE protocol to coordinate with each other, control parts
 of the same FE.
Theoretical case: Somebody defines that CE<->CE protocol and
 heterogeneous CEs used.  Hard problem and beyond the scope of
 ForCES.

> 7) Is it necessary/useful to keep the ForCES protocol 
> open-ended even now?  
> I fail to see valid reasons for multiple protocols..:
> 
>    ForCES Post-Association Phase Protocol - [...]
>                      This protocol could be a single protocol or
>    could consist of multiple protocols working together.

Looking at router functions, there can be a fairly large
difference between the proportion of CE<->FE "control" traffic 
(that is: CE giving new FIB entries to FE, FE reporting alarms 
to CE,  etc.) and the CE<->FE "data" traffic (that is: OSPF
HELLO and other packets, the SNMP SET/GETs, the TCP SYN
arriving from a peer router setting up a BGP connection).

Also: The CE<->FE data traffic has quite different reliability
requirements.  That is: If a SNMP packet, which arrived over
UDP, gets dropped in transport between FE and CE, it is much 
less of a problem than if a CE packet adding an ACL entry 
gets dropped going from CE to FE.

Also: In some architectures there is a physically, or at least
logically (different queue etc) distinct control channel from
data channel.

Given these different requirements and characterizations,
the ForCES architecture might specify that some traffic
("data" traffic) gets transported over a lightweight,
unreliable protocol, and other traffic ("control" traffic)
gets transported over a reliable, perhaps even secure,
protocol.

Mandating in the requirements that all types of CE<->FE
traffic all run over a single protocol/connection just
doesn't seem something a requirements doc should do.

> 8) Again, the ForCES model seems quite open-ended:
> 
>    FE Manager - A logical entity that operates in the pre-association
>    phase and is responsible for determining to which CE(s) a FE should
>    communicate.  This process is called CE discovery and may involve
>    the FE manager learning the capabilities of available CEs.  A FE
>    manager may use anything from a static configuration to a pre-
>    association phase protocol (see below) to determine which CE(s) to
>    use.  Being a logical entity, a FE manager might be physically
>    combined with any of the other logical entities mentioned in this
>    section.
> 
> ... I fail to see the usability of FE's discovering CE(s).  Being a 
> master-slave association where CE's are masters, I would 
> think it would be 
> natural to assume CE's are the masters and thus _they_ discover their 
> slave FE's.

FEM/CEM discovering each other is outside the ForCES scope.
The FEM/CEM concept is included to be able to say basically
"look, we know CEs and FEs need to find each other - and
we don't define how it is done, because we wanted to limit
our scope".

For the near future I'd expect that a CEM/FEM might be little
more than a human being typing in a CE IP address via the
serial port of the FE and v.v. on the FE.

> 9) In the snippet above and some others: it seems to be 
> typical in the 
> requirements to overload the terminology with rather long 
> architectural 
> descriptions.  This has both good and bad sides, but in any 
> case it should 
> be considered whether some other way to achieve the big 
> picture would be 
> more usable.
> 
> Also, there are some terms in the terminology which are not 
> used in the
> document.  If this is also meant as a terminology document, 
> this might be
> ok, but otherwise keeping to the minimum might be useful.

This is actually something I've never gotten a good answer
to.  In dealing with terminology, what is the BKM? Is it:
A) Put all terms we know we will need in the first doc
   (in this case reqs), adding deltas to later docs once
   reqs is published, or
B) Put in reqs only terms used in reqs - each doc defines
   all the terms it uses, or
C) Put a separate terminology doc.

I think C doesn't work because it would be a normative
ref that would hold up other docs from progressing.
Not sure whether to do A or B though - do you think B
is the best approach?

> 10) The list of FE functions below is extensive:
> 
>    Some functions that FEs could perform include layer 3 forwarding,
>    metering, shaping, firewall, NAT, encapsulation (e.g., tunneling),
>    decapsulation, encryption, accounting, etc.  Nearly all 
> combinations
>    of these functions may be present in practical FEs.
> 
> .. but in reality, gives little hope, as the list is far from 
> complete,
> and completely evolving.  It seems apparent that if one tries to
> standardize all the possible functions, with *a lot* of 
> abstractization,
> this is a source of a LOT of work, and work that's never going to end.
> 
> .. so, it might be useful to consider the scope: deal with 
> the most basic form of forwarding, and that only, for now.

The section that appears with is just giving an architectural
overview.  The model document is chartered with specifying exactly
what functions will be tackled by ForCES v1.  Most likely it will
be a shorter list as you suggest.

> 11) The last sentence here implies a possible disconnect in the 
> architecture:
> 
>    1) CEs and FEs MUST be able to connect by a variety of interconnect
>    technologies.  Examples of interconnect technologies used 
>    in current
>    architectures include Ethernet,bus backplanes, and ATM (cell)
>    fabrics.  FEs MAY be connected to each other via a different
>    technology than that used for CE/FE communication.
> 
> ==> Is it intended to connect FE's to CE or provide a FE-FE 
> protocol?  
> Having some interconnection and/or protocol between FE's 
> seems useful in a 
> sense, but also potentially out of scope of this architecture.

The goal of this section is to explain that the CE<->FE wire
(which is what forces cares about) may or may not be the same
as the FE<->FE wire.

There are some efforts going on to look at FE<->FE protocol.
However, the expertise involved (fast path data plane issues,
state sharing between forwarding devices) is somewhat less
likely to be found in IETF than the expertise for CE<->FE
protocol issues.  That, in combination to keep ForCES at a
manageable scale, excluded FE<->FE as a deliverable in the
ForCES protocol.

> 12) This sounds very challenging:
> 
>    The variety of FE functionality that the ForCES architecture allows
>    poses a potential problem for CEs.  In order for a CE to 
> effectively
>    control a FE, the CE must understand how the FE processes packets.
>    We therefore REQUIRE that a FE model be created that can 
> express the
>    logical packet processing capabilities of a FE.
> 
> .. I'd say that most functionalities differ *a lot* from 
> vendor to vendor.  
> Are you really, really sure that you should (or can) express 
> all of these 
> in completely independent way *AND* that subsequent use if 
> it's understood 
> would be *effective*.  Also see other subsections here, like 6.2.

The functions do sometimes differ from vendor to vendor.
There is an opposing trend going on with the emergence of
merchant silicon and network processors that is leading
to some convergence of (at least the basic) FE 
functionalities around one or a few basic models.

> 13) Another architectural problem..:
> 
> 6.3. Ordering of Logical Functions
>    The model MUST be capable of describing the order in which these
>    logical functions are applied in a FE. The ordering of logical
>    functions is important in many cases.  For example, a NAT function
>    may change a packet's source or destination IP address.  Any number
>    of other logical functions (e.g., layer 3 forwarding, 
> ingress/egress
>    firewall, shaping, accounting) may make use of the source or
>    destination IP address when making decisions.  The CE needs to know
>    whether to configure these logical functions with the pre-NAT or
>    post-NAT IP address. [...]
> 
> .. so are we talking about N! possible orderings (with N 
> possible logical
> functions).  Seems like a horror ("pre-nat post-qos-meter
> pre-qos-classification pre-forward-do-something foo bar boo 
> baf"-state).

The difference between English and nonsense is that one
can combine letters (and words) in any order, but one
can only make an English word or sentence if one follows
certain rules of grammar.

Same for routers.  Given a repertoire of the most well known
functions (words), it is likely that a few particular
configurations (sentences) will be common. 

That being said: I expect initially that ForCES will only
work for the most basic configs, enabling interoperability
between simple devices, or devices that have been pre-tested
with each other.   That is a good step forward from today,
even if it isn't the ultimate goal of plug-n-play any CE 
to FE.  This is not uncommon in IETF.  VoIP (RTP/SIP/etc)
is a good example of where a complex problem required starting
simple, doing interop tests, interop profiles, plugfests, etc., 
and still doesn't have true off-the-shelf 100% interop, but 
has nevertheless provided significant value.

> 14) I'm having difficulties parsing the actual requirements in:
> 
>    8)Off-loaded Functions
> 
> .. do you require that basically any piece of code CE wants 
> to offload to
> FE's must be supported as long it's expressed as a finite 
> state machine..  
> or what?  That that code is to be executed on all (or a 
> specified portion
> of)  the packets?

Others felt the same way, this is being removed as we speak.

> 15) I don't disagree that security must be supported (btw. 
> nit: s/make private/encrypt/), but..:

Disagree on the nit: Making private might be a secure Ethernet
cable, but encrypt is a mathematic function.  What we want is
privacy, the means arrived at doesn't matter (for the reqs at
least).

>    2)Support for Secure Communication
>    a) FE configuration will contain information critical to the
>       functioning of a network (e.g. IP Forwarding Tables). 
> As such, it
>       MUST be possible to ensure the authenticity and integrity of
>       ForCES protocol messages.
>    b) FE configuration information may also contain 
> information derived
>       from business relationships (e.g. service level agreements).
>       Therefore, it MUST be possible to secure (make private) ForCES
>       protocol messages.
> 
> .. I'm not sure how applicable this is in in most/some 
> scenarios.  For 
> example, if the interconnect between CE and FE's is a 
> shared/dedicated 
> bus, e.g. an Ethernet link, I'm not sure how applicable these are.   
> Basically if the control messages traverse over the Internet, 
> they should 
> be securable.  I'm not sure how one considers the possible deployment 
> scenarios, but I don't think over-the-Internet case is too common 
> (typically hinting at CE/FE located in a different NE).

See above & I agree.  A shared/dedicated bus is private, but not
encrypted. The privacy requirement is there, just met by other means
(physically secure wire). Requirements are being updated to capture this.

> 16) on the use of reliable protocols:
> 
>    4)Multihop
>    When the CEs and FEs are separated beyond a single hop, the ForCES
>    protocol will make use of an existing RFC2914 compliant L4 protocol
>    with adequate reliability, security and congestion control (e.g. 
>    TCP, SCTP) for transport purposes.
> 
> ==> I fail to see the added value of a separate protocol for 
> singlehop links; there, the latency is low and reliable protocols 
> should be quite usable.

Running TCP over a tightly coupled backplane e.g. PCI seems to some
like overkill.  I think it might be ok even there but the requirements
isn't the place to say that - requirements is instead being slightly
rewritten to focus on what the actual reliability/retransmit requirements
are.  The framework and protocol drafts will then specify how they answer
those requirements, and one valid answer might be "we do this by using TCP
in all cases".

I personally am a fan of assuming a reliable transport, however, at
least one of the proposed forces protocol builds in reliability
itself as needed.  Excluding it via requirements seems an improper
approach.

> 17) security considerations should be properly written out, a 
> blind referral insufficient:
> 
> 9. Security Considerations
> 
>    See architecture requirement #5 and protocol requirement #2.

Will be done.

> 
> 
> Semi-editorial:
> ---------------
> 
> CEs may consist of PCE partitions or whole
>    PCEs.
> 
> ==> what about 'multiple PCEs' (compare to the text of FE's) ?

Ok.

>    Post-association Phase - The period of time during which a FE does
>    know which CE is to control it and vice versa,
> 
> ==> is "FE controlling CE" (through vice versa) ? I think not?

Good catch. Will be fixed.

>    2) FEs MUST support a minimal set of capabilities necessary for
>    establishing network connectivity (e.g., interface discovery, port
>    up/down functions).  Beyond this minimal set, the ForCES
>    architecture MUST NOT restrict the types or numbers of capabilities
>    that FEs may contain.
> 
> ==> it is not unambiguous what the last sentence tries to 
> say; does it 
> take a stance that at least a minimum set of capability types must be 
> supported, that ForCES must work in the presence of any 
> possible set of 
> capabilities, or what..

It says the first thing you posited, that being said, will have
text clarified.

>    9) Any proposed ForCES architectures MUST explain how that
>    architecture supports all of the router functions as defined in
>    [RFC1812].
> 
> ==> this must be elaborated.  RFC1812 is a long RFC, and no proposal 
> should be expected to go through all of these functions!  
> Clearly there are certain specific issues which need to be addressed
here...

Framework document actually does go through 1812 pretty well.

>    10) In a ForCES NE, the FEs MUST be able to provide their topology
>    information (topology by which the FEs in the NE are connected) to
>    the CE(s).
> 
> ==> the wording in ()'s doesn't really clarify much and should be 
> rewritten; also, it's rather difficult to understand.  Same 
> in the second version of this req.

OK

> 
>    We therefore REQUIRE that a FE model be created that can 
> express the
> 
> ==> REQUIRE is not a proper RFC2119 keyword.

Will fix.

>    2)Forwarding Functions
>    The FE model MUST be capable of expressing the data that 
> can be used
>    by the forwarding function to make a forwarding decision. Support
>    for IPv4 and IPv6 unicast and multicast forwarding 
> functions MUST be
>    provided by the model.
> 
> ==> just as a comment, I fear multicast functions may be 
> significantly more difficult to provide than unicast.

I agree, I know of at least two quite different ways multicast
has been internally supported in routers.  Will be one of
the more difficult parts of forces imho.

>    The FE model SHOULD be extensible so that new, currently 
> unknown  FE
>    functionality can be expressed. The FE Model SHOULD NOT be extended
>    to express standard/common functions in a proprietary manner. This
>    would NOT be ForCES compliant.
> 
> ==> NOT is not a proper RFC2119 keyword.

Ok.

>    5)Message Priority
>    The ForCES protocol MUST provide a means to express the protocol
>    message priorities.
> 
> ==> should this be elaborated?  Ie., do you mean in the case protocol 
> messages get queued and some messages are more important than 
> others so 
> they'll be able to overtake the queue?  (and not e.g. in 
> TOS/traffic class 
> context.)  Note that this may have quite interesting implications for 
> multipart or sequenced commands, so prioritizing should be 
> used with care.

Will be elaborated.  Example of what is meant here is that the 
message to install a filter to drop all SYNs from a certain attacker
should get priority treatment over nearly all the other traffic
going across the backplane (including those SYNs).

>    9)Packet Redirection/Mirroring
>    a) The ForCES protocol MUST define a way to redirect 
> packets from the
>       FE to the CE and vice-versa. Packet redirection terminates any
>       further processing of the redirected packet at the FE.
> 
> ==> when redirected from CE to FE, the latter sentence does not parse 
> properly.

True - redirected from FE to CE, and originated by CE and sent via
FE to another network element. Will get fixed.

> 
> 
> Editorial:
> ----------
> 
>   Status of this Memo
> 
>    This document is an Internet-Draft and is in full conformance with
>    all provisions of Section 10 of RFC2026.  Internet-Drafts are
>    working documents of the Internet Engineering Task Force (IETF),
>    its areas, and its working groups.  Note that other groups may
>    also distribute working documents as Internet-Drafts. 
> [...]
> 
> ==> the boilerplate is not as-is.

Ok, will fix.

>   Conventions used in this document
> 
> ==> I'd place this under a numbered section, at the introduction or 
> something, but I'm not sure if there is a strict policy on that.

Will check nits.

> 1. Abstract 
> 
> ==> abstract must not be numbered, per ID-nits etc.

Hmm. Damn sneaky nits always get through. Been over this
doc twice looking for nits already!

> Khosravi, Anderson, et. al.   Expires July 2003               [Page 1]
> 
> ==> this should be either "Khosravi, et al." or "Khosravi & Anderson"
> (rfc-ed policy)

Didn't know this one, will get fixed.  Does this apply even
when there are two editors?

>    8. 
> References.....................................................13
>  8.1. Normative 
> References...........................................13
>  8.2. Informative 
> References.........................................13
>    9. Security 
> Considerations........................................13
>    10. Authors' Addresses & 
> Acknowledgments..........................14
>    11. Editors' Contact 
> Information..................................15
> 
> ==> personally I'd like sec considerations before references 
> (that's the more typical way), but that's not a strict rule.

Seems reasonable.

> ==> note that IPR & copyright statements are also missing, 
> but those can be added later on too.  

Assumed those would be added by RFC-editor, can add it now.

>    entities that cooperate to provide a given functionality (such as a
>    routing or IP switching) and yet appear as a normal integrated
> 
> ==> s/a routing/routing/
> 
>    For the purpose of illustration, let us consider the 
> architecture of
>    a router to illustrate the concept of separate control and
> 
> ==> please try not to use the word "illustrate" twice.
> 
>    architectures include Ethernet,bus backplanes, and ATM (cell)
> 
> ==> s/,bus/, bus/
> 
>    ForCES protocol elements from joining a NE.(For more protocol
> 
> ==> s/(/ (/
> 
> requirement #1)
>    details, refer to section 7 requirement# 2) 
> 
> ==> choose the format how to refer to requirements and be 
> consistent; I
> prefer " #x" rather than "# x" but YMMV.

Will make it so.

Thanks for the detailed technical and editorial feedback!

-David

---

From: Pekka Savola <[email-masked]>
Date: Fri, 11 Apr 2003 17:36:06 +0300 (EEST)

Hi,

Sorry for the delay in responding.

On Tue, 8 Apr 2003, Putzolu, David wrote:
> > Comments on forces-reqs-08.  In summary, I'd like to express a very
> > grave concern of trying to address every possible problem, a path that
> > seems doomed to failure.  Note a list of dozen different forwarding
> > associated complexity of handling all the different nuances and
> > orderings of these functions.
> > 
> > Seems way, WAY too complex to make interoperable and useful.  I wonder
> > how far we have to go with this effort to see it as a dead-end.  Or
> > alternatively, some other architectural/approach revamp might be able
> > to salvage some of the mechanisms (e.g. concentrate on certain core
> > tasks only, but then again, one might question how useful the
> > mechanism is).
> 

Ok, I guess it remains to be seen later whether this is the right 
approach.  My worry is that if we don't define the "applicable scenarios" 
properly (or define wrong scenarios), we might end up specifying the 
wrong 20%.
 
> > 1) it is not clear how different efforts (most of which referenced in
> > the draft) interconnect: e.g. midcom (similar protocol between hosts,
> > typically), gsmp, ccamp.
> 

Yes, that was my impression.
 
> level change, so I'm hesitant to jump into that debate.

I got the impression that Ethernet switches (or at least the quite typical 
switch/routers) were in scope, but I may have misread.
 
> > innovations typically require changes in both elements: I doubt one
> > can create such a generic architecture which would accommodate
> > anything happening to the other in the future.
> 
> Anything talking to anything is impossible. A CE talking to successively
> faster generations of FEs with IPv4 forwarding and DS support, using the
> same standard ForCES protocol, is possible.

My experience indicates that typically CE's develop faster than FE's --
due to simplicity of generic programming, rather than implementing
hardware functions.  But CE's supporting more features than FE's 
doesn't seem all too useful.
  
> > .. the latter would seem to indicate some support for non-IP devices
> > too, or should this use some rewording to clarify which kind of switch
> > fabric was the question was all about?
> 

Ok.
 
> > 4) I haven't delved into ForCES, but the last two sentences look a bit
> > scary:
> > 
> > .. this seems to imply that CE's instruct logical FE's to perform some

Yep, clarification will be helpful.
 
> be an undesirable complexity.

I totally agree here.
 
> > would have tradeoffs and drawbacks, but is integral when you consider
> > the applicability of the protocol (ie. is it a network management

Perhaps this could be spelled out in the requirements, to some degree; 
like: 

  Multiple CE's controlling one FE MAY be supported.  However, 
  coordination between CE's is out of scope.
 
(or even a bit more, or even more harsh wording.)

In addition to listing requirements, it's IMO also useful to list 
_non-requirements_.  But YMMV.

> doesn't seem something a requirements doc should do.

Ok, sounds reasonable.

I was thinking of specifying multiple protocols for achieving about the 
same function, which would seem to be wasteful.

> > master-slave association where CE's are masters, I would think it
> > would be natural to assume CE's are the masters and thus _they_
> > discover their slave FE's.
> 

Yep -- could be clarified slightly.
 
> > 9) In the snippet above and some others: it seems to be typical in the
> > architectural descriptions.  This has both good and bad sides, but in
> > any case it should be considered whether some other way to achieve the
> > big picture would be more usable.
> > 
> > Also, there are some terms in the terminology which are not used in
> > the document.  If this is also meant as a terminology document, this
> > might be ok, but otherwise keeping to the minimum might be useful.
> 

C is not all that useful here.  I don't have a strong opinion on A) vs B).  
Personally, I'd probably try to keep the list as simple as possible, to 
the extent of understanding the requirements .. and put the rest in 
framework, usage model or other documents where the model is expanded.
> > complete, and completely evolving.  It seems apparent that if one
> > tries to standardize all the possible functions, with *a lot* of
> > abstractization, this is a source of a LOT of work, and work that's
> > never going to end.
> > 

Then my suggestion would be to shorten the list of functions a bit.  E.g. 
encapsulation, NAT, decapsulation, and encryption seem something that seem 
the most redundant of those listed.  Or cut it even more dramatically.

> > ==> Is it intended to connect FE's to CE or provide a FE-FE protocol?  
> > Having some interconnection and/or protocol between FE's seems useful
> > in a sense, but also potentially out of scope of this architecture.
> 
> The goal of this section is to explain that the CE<->FE wire (which is
> what forces cares about) may or may not be the same as the FE<->FE wire.

Please state that the latter is out of scope ?
 
> manageable scale, excluded FE<->FE as a deliverable in the

Good.

> >    We therefore REQUIRE that a FE model be created that can express
> > the
> > .. I'd say that most functionalities differ *a lot* from vendor to
> > vendor.  Are you really, really sure that you should (or can) express
> > all of these in completely independent way *AND* that subsequent use
> > if it's understood would be *effective*.  Also see other subsections
> > here, like 6.2.
> 

If you say so.
 
> configurations (sentences) will be common. 

I seem to have different kind of routers in mind.  Of course, one could 
try to do some "pattern recognition" or whatever, but I'm not sure how 
useful such would be.
 
> has nevertheless provided significant value.

There is a difference of "interoperability issues due to differently
implemented standards" and "interoperability issues due to different ways 
non-standard, purposely implementation-specific features are implemented".

In ForCES, the problem space is of the latter, which is much more 
difficult.  It can be attacked e.g. by abstraction layers, but a danger is 
that by too much abstraction, the operation becomes very complex and you 
may lose the original data the abstractization is based on.

> > .. do you require that basically any piece of code CE wants to offload
> > to FE's must be supported as long it's expressed as a finite state
> > machine..  or what?  That that code is to be executed on all (or a
> > specified portion of)  the packets?
> 

Ok, good.

> privacy, the means arrived at doesn't matter (for the reqs at

Ok, I can agree with the distinction but I'd still argue on rewording it 
otherwise; "make private" doesn't sound too good.  "Ensure privacy"?
(privacy is also a bit dubious word..).

> > .. I'm not sure how applicable this is in in most/some scenarios.  
> > For example, if the interconnect between CE and FE's is a
> > shared/dedicated bus, e.g. an Ethernet link, I'm not sure how
> > applicable these are.  Basically if the control messages traverse over
> > the Internet, they should be securable.  I'm not sure how one
> > considers the possible deployment scenarios, but I don't think
> > over-the-Internet case is too common (typically hinting at CE/FE
> > located in a different NE).
> 

Ok.

> like overkill.  

I agree -- but then again, such backplanes are not places for an IP 
protocols, probably.

> I think it might be ok even there but the requirements

Ok.

[snip]

> Framework document actually does go through 1812 pretty well.

Good .. but here, the requirement above will need to be clarified too, 
though.

[snip]

> when there are two editors?

For one author:

Doe

for two authors, it can be:

Doe & Dee

for more:

Doe et al.
 
(or so I've understood it.)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Patrick Droz <[email-masked]>
Date: Wed, 07 May 2003 14:16:40 +0200

Hi Pekka,

sorry for the late response I thought my co-chair had
already answered.

> doesn't seem all too useful.

I think here we have a misunderstanding. FE's are supposed to
mainly address the "fast pass" of the "data path" which is a fraction
of what the "data path" as part of the CE can do (example: handling
IP options). It is actually better that CE's are developing
faster which makes it always possible that the CE's can make
best use of the offerings of the FE's. It would be bad to have
FE's that can offer a lot of features which the CE's can not
configure and manage.

> Yep, clarification will be helpful.

I will bring it to the DT to clarify.

>>be an undesirable complexity.

I have not seen scenarios by David but let me try to step
in. The GSMP did the full exercise in terms of partitioning.
Partitioning sometimes also called virtualization seems to
be a rather hot topic. The main argument for it is that a
single user will not be able to make use the total available
resources so you want to share it among many different users
but preventing that they get on each others turf. I completely
agree it is difficult to achieve but it is doable. Another
example would be to assign certain resources to certain VPN.
I will poll the DT on the subject.

>   coordination between CE's is out of scope.

I think this what reflects the consensus of the group.
So I will ask the DT to spell it out.

> Yep -- could be clarified slightly.

ok

> framework, usage model or other documents where the model is expanded.

I will ask the DT to go with your suggestion only putting the
relevant terminology in the document.

> the most redundant of those listed.  Or cut it even more dramatically.

ok I ask the DT to come up with a shorter list as an initial
ForCES v1

> Please state that the latter is out of scope ?

I agree with my co-charis statement. At the moment we have to
leave this open. There are certain things that have to be
exchanged between an FE and CE as well as among FE. A good
example of this is ARP cache synchronization. What we want
to avoid in ForCES to come up with things that concern only
FE <-> FE relevant things.

> try to do some "pattern recognition" or whatever, but I'm not sure how 

I agree with my co-chair on this. Let me try to explain.
Take a traditional centralized router with all the
functionality a router can do in software. When you
look at the data plane code you can draw a clear graph
(nodes representing the functionality) a packet can
travel through a router. This different paths through
the graph represent the set of possible sequences. Such
a graph has a single entry point and a single exit point,
and some node a probability assigned that the packet will
be dropped or lost.

>>Others felt the same way, this is being removed as we speak.

here I think I have to kick in against my co-chair. This was
heavily discussed on the official mailing list and it was put
in there because a significant amount of people had advocated
for it. It was never the intend to off-load any piece of code
to an FE. Offloading was standing to protocol off-load e.g.
TCP / SCTP off-load. As part of the ForCES we have capability
discovery during which a FE a signal to the CE "hey I can do
TCP offload". Another example would be OSPF hello offload. There
are FEs out there that are not doing anything else than TCP off-
loading combined with IP forwarding.
I completely disagree that something that was put into the document
by consensus of the official mailing list ought to be dropped
just like this!

> (privacy is also a bit dubious word..).

I will ask the DT to come up with a better explanation.

> I agree -- but then again, such backplanes are not places for an IP 

yes, but the same ForCES message format / data structures
can be used without being inside an IP packet.

> Good .. but here, the requirement above will need to be clarified too, 

ok I will ask the DT to do so.

> for two authors, it can be:

I will ask the editor to do so.

> (or so I've understood it.)

Thanks,
Patrick

-- 
   Dr. Patrick Droz 			| [email-masked]
   IBM Zurich Research Laboratory 	| http://www.zurich.ibm.com/~dro
   Saumerstrasse 4 			| Tel. +41-1-724-85-25 CH-8803
   Rueschlikon/Switzerland 		| Fax. +41-1-724-85-78

---

From: Pekka Savola <[email-masked]>
Date: Thu, 8 May 2003 15:13:21 +0300 (EEST)

> FE's that can offer a lot of features which the CE's can not

I can agree with this, but I don't think it has bene stated that FE's do 
not have to implement all the functionality, but that control plane can 
also handle forwarding plane functionality, if the specific FE element 
doesn't know how to do them.

With some vendors, this "CE overload" may be a typical practice, but not
so for all of them.

Perhaps trying to insert some discussion on the features of FE/CE like 
these could clarify the sentence.

Ok.

Ok.

> ok I ask the DT to come up with a shorter list as an initial

Thanks.

> FE <-> FE relevant things.

Umm.  I don't know the details so bear with me: is ARP cache 
synchronization to be specified or already specified in ForCES, or is this 
just something implementations will/may have to do?

In the latter case, the last sentence:

  FEs MAY be connected to each other via a different
  technology than that used for CE/FE communication.

should be rephrased slightly, like:

  FEs MAY be connected to each other via a different
  technology than that used for CE/FE communication;
  however, FE-FE interconnection is out of scope.

(obviously this is not an optimal wording, but you get the picture what I 
meant..)

> the graph represent the set of possible sequences. 

Yes, you can do that, but my argument is that I fear that the graph could 
be very dense, interconnected.

But I'm having trouble figuring out what you actually mean with this 
"ordering of logical functions".

Do you have for example some specific examples?  That might help in trying 
to see the intent here.

> by consensus of the official mailing list ought to be dropped

Well, in any case I think the section was not clear exactly what kind of
features were to be offloaded.  The title says "functions" ("NAT",
"filtering"), but what it seems to me are just details of the functions, 
to make them as lightweigh to the CE as possible.

> can be used without being inside an IP packet.

Is exchanging ForCES messages in PCI backplanes (or where you don't use IP 
protocols) in scope?