Subject: Deadline of Aug. 5th (was RE: Proposed changes to RSIP evaluation (was RE: [midcom] Comments on protocol evaluation draft

From: "Mary Barnes" <[email-masked]>
Date: Fri, 2 Aug 2002 20:02:08 -0500

Hi Jim,

Just one final "process" note; I missed putting a deadline for your
feedback.  If you plan on proposing additional new text which would be
required for some of your proposed changes, I would need that text by COB
Monday, August 5th, so that we have a day or so to discuss on the list prior
to me making the final edits.  

If you could post each individual requirement in separate emails to the
list, that will facilitate my tracking of resolution of the discussion.  The
Subject line should be similar to the following: 
   Protocol eval: Section x.y.z - <Short description of change/categoy>  

Thanks,
Mary. 

-----Original Message-----
From: Barnes, Mary [NGC:B602:EXCH] 
Sent: Friday, August 02, 2002 9:48 AM
To: Taylor, Tom-PT [CAR:B602:EXCH]; [email-masked]'
Cc: 'midcom'
Subject: Proposed changes to RSIP evaluation (was RE: [midcom] Comments
on protocol evaluation draft

Jim et al, 

I've not thoroughly reviewed all the proposed changes in detail, but will do
so shortly, but I also did want to respond to the Category 4 to which Tom
also responded wrt security.  We have already discussed and resolved this
concern per email discussions based on the -01 version.  Refer to:
http://www.ietf.org/mail-archive/working-groups/midcom/current/msg02397.html

Now, you're also proposing that RSIP be "upgraded" for the security ones
that you had scored as Failing to meet compliancy in your analysis.  Based
upon the text currently in section 2.1.8, 2.3.1, which also applies to
2.3.2-2.3.4, I don't have a problem changing those to Ps.  The P+ category
would only apply if these changes could be made to RSIP as a natural
extension of the protocol with no impact to current users of that protocol
(i.e. current users don't make use of the information, and wouldn't be
bothered by it being in the messages).  

There are a many other RSIP scores, many of which were derived from your
original analysis, that you're also now proposing changes for:
2.1.2 - The text for this one clearly states that an extension is required.
2.2.1 - The text currently supports the P+ score. 
2.2.2 - I think a P+ needs additional text to substantiate that this would
be a compatible extension of the protocol (OR we need some text upfront in
the RSIP overview describing the extensibility mechanisms defined by RSIP)
2.2.3 - ditto 
2.2.7 - I can't quite see how this goes from F to P+; you'll need to provide
more text or clarification.
2.2.8 - I think a P+ needs additional text to substantiate that this would
be a compatible extension of the protocol 
2.2.9 - I don't have a problem with changing this to P.  Again, P+ would
require additional text to substantiate that this would be a compatible
extension of the protocol.
2.2.11 -ditto 
  
Upon another quick glance, I think many of the proposed changes for the
other protocols are bringing forth issues with other "scores" that we have
already discussed (e.g. 2.2.2 for Megaco)  Certainly, if there is
overwhelming support for your changes, then they can be done, but we've got
to have more than one person supporting the change for something that we
have previously discussed and I certainly thought had reached concensus on.
I won't be responding to each of these that we've already discussed, you can
search the email archives, where each has been addressed in a separate email
thread. 

Again, I haven't looked at each in detail, and you may indeed have found
some changes which are valid, so I will post separately each of those which
I would agree to change provided there is WG concensus (i.e. no negative
responses to my proposals).

Regards,
Mary.

-----Original Message-----
From: Taylor, Tom-PT [CAR:B602:EXCH] 
Sent: Friday, August 02, 2002 9:01 AM
To: [email-masked]'; Barnes, Mary [NGC:B602:EXCH]
Cc: 'midcom'
Subject: RE: [midcom] Comments on protocol evaluation draft

From my point of view you raise two issues I cannot leave unanswered.  The
first has to do with P+ extensions and your first scoring inconsistency, the
second with security.

I confess that the need for protocol extensions poses a logical issue: how
to treat this need when it affects multiple categories of response.  On the
one hand, I think it is over-stating things to simply score 1 against each
category that depends on the extension.  On the other, it is certainly true
that, whether you are defining a new MIB, a new DIAMETER application, or a
new Megaco package, you imply the need for extra code in both the Agent and
the Middlebox above that supplied by the base protocol implementation.  I
myself would propose adding an evaluation category that specifies the
increment to the base protocol implied by its use for MIDCOM.  Leaving that
aside, any protocol that fails to meet one of the MIDCOM requirements is
probably beyond our consideration.

On the security issue: you dismiss too quickly those protocols that rely on
lower layers for their necessary degree of security.  The RFCs specifying
these protocols have security sections.  The protocol security architecture
in each case was arrived at by considering the specific threats that had to
be countered in the protocol's domain of application.  An architecture that
relies on lower layer security is perfectly valid if that meets the
requirements for that protocol.

What we do have to think about is whether the threats are different for the
MIDCOM application from those considered when the candidate protocol was
being designed.  If so, the security architecture for that protocol may have
to be modified to fit MIDCOM application.  But the security section of the
MIDCOM requirements document reflects the results of the MIDCOM threat
analysis.  Thus if our candidate protocols, including their specified
security architectures, can meet these requirements in a practical way, they
must be scored as passing those requirements.  

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Friday, August 02, 2002 12:45 AM
To: Barnes, Mary [NGC:B602:EXCH]
Cc: 'midcom'
Subject: RE: [midcom] Comments on protocol evaluation draft

Mary (and other middlebox experts),

Thank you much for your reply.

The clarification of the grading system is especially useful. I had a
lot of trouble understanding it before, but I think I got it now.

Here's what I now understand the grades to mean:
T  - Compliance to the requirement with the protocol as is
P+ - Compliance to the requirement with an extension to the protocol
P  - Compliance to the requirement with a change, or an extension that
     impacts existing users of the protocol
F  - No compliance (even with extension of the protocol???)

Some observations on this:

- Since all the protocols are extensible and extensions can generally be
made, one way or another, to be backward compatible, I'm surprised that
we have any P's of F's. :-)

- We seem to have lost the concept of how large an extension needs to be
made to add a "thing" to the protocol. Is the protocol close to the
requirement and just needs a little extension? Or does the protocol have
nothing to offer and the thing has to be added completely? I originally
thought that was the idea behind the the T, P and F grades ("How much do
ya have to change the protocol to satify the requirement? None, a little,
and a lot, respectively"), but I guess we evolved from that, assuming
that was the original intention.

As you suggested, I went back and looked for what I thought were incon-
sistencies in P+ and P grades. Once I got started digging, what I wound
up doing was quickly reviewing all the scores for consistency with the
grading system as I now understand it and the summary information for
each protocol's compliance with each requirement as given in the
protocol evaluation I-D (BTW, I used the -02 version of the I-D, without
any of the changes that have been discussed since it was published.).

The grading inconsistencies that I saw fell into 4 categories:

1. The requirement compliance text for the protocol basically said "This
can be done with an extension to this protocol" and the grade was a T,
instead of a P+ as I believe it should be. I included the closely related
"The protocol does not preclude this extension from being made." in this
category.

2. The requirement compliance text for the protocol says the protocol
has something here but extension is required and can be made with no
impact, but graded it as a P instead of a P+.

3. The requirement compliance text for the protocol says the protocol has
nothing here but the necessary extension can be made without impact, and
graded it as an F instead of a P+.

4. The requirement compliance text for the protocol says the protocol has
nothing here but if ya use TLS or IPsec or whatever for security ya get
everything, and graded it as a T.

In the last case, I'm not sure what the right score is, but I would
suggest at most P+, to distinguish these from protocols that have the
right thing here without resorting to TLS or IPsec or whatever for
security, which I think should get a T grade. (For consistency, I
considered grades of F (Honestly given, in my opinion, but I could be
wrong.) but the requirement could be fulfilled with TLS or IPsec or
whatever for security, hence now graded P+ for consistency, as instances
of this case also.)

I also found a few other what I believe to be inconsistencies, at least
based on the scoring system as I now understand it. Call this category 5.

I list the inconsistencies I found at the end of this message, but
here's some statistics. Take 'em for whatever ya think they're worth.

Out of a total of 135 scores (27 requirements times 5 protocols) I found
what I thought were 31 inconsistencies. That's just less than 23%.

The number of inconsistencies by category by protocol are:

Category        SNMP    RSIP    MEGACO   DIAMETER        COPS    Total
--------        ----    ----    ------   --------        ----    -----
1        1       0       3       3        1       8
2        0       2       0       1        0       3
3        0       4       0       0        0       4
4        0       5       4       5        0      14
5        0       2       0       0        0       2
        --      --      --      --       --      --
Total    1      13       7       9        1      31

The resulting adjustments in protocols scores are:

Protocol        T       P+      P        F
--------        ---     ---     ---      ---
SNMP    - 1     + 1
RSIP    + 2     +10     - 3     - 9
MEGACO  - 7     + 7
DIAMETER        - 8     + 9     - 1
COPS    - 1     + 1
        ---     ---     ---     ---
Total   -15     +28     - 4     - 9

And the resulting protocol scores are:

Protocol        T       P+      P        F       Total
--------        ---     ---     ---      ---     -----
SNMP     18       9       0       0       27
RSIP     14      13       0       0       27
MEGACO   13      11       2       1       27
DIAMETER         13      13       1        0      27
COPS     19       6       1       1       27
        ---     ---     ---     ---      ---
Total    77      52       4       2      135

This was done real quick, so there's probably still stuff not right, but
now, at least to me, the numbers better match the general feeling I get
when reading the I-D.

Comments expected and welcome.

Jim

================

Inconsistencies I found in the MIDCOM protocol evaluation grades
(This was composed with tab stops every 8 characters.)

                       Score
Require-                -------------------              Inconsistency
ment    Protocol        Was     Should be                Category
--------        --------        ---      ---------               --------
2.1.2   RSIP    P       T                5
2.1.8   RSIP    F       P+               4
        MEGACO  T       P+               4
        DIAMETER        T       P+               4
2.2.1   RSIP    P+      T                5
2.2.2   SNMP    T       P+               1
        RSIP    P       P+               2
        MEGACO  T       P+               1
        DIAMETER        T       P+               1
2.2.3   RSIP    P       P+               3
2.2.4   MEGACO  T       P+               1
        DIAMETER        T       P+               1
2,2,6   DIAMETER        T       P+               1
        COPS    T       P+               1
2.2.7   RSIP    F       P+               3
        DIAMETER        T       P+               1
2.2.8   RSIP    P       P+               2
        MEGACO  T       P+               1
2.2.9   RSIP    F       P+               3
2.2.11  RSIP    F       P+               3
2.3.1   RSIP    F       P+               4
        MEGACO  T       P+               4
        DIAMETER        T       P+               4
2.3.2   RSIP    F       P+               4
        MEGACO  T       P+               4
        DIAMETER        T       P+               4
2.3.3   RSIP    F       P+               4
        MEGACO  T       P+               4
        DIAMETER        T       P+               4
2.3.4   RSIP    F       P+               4
        DIAMETER        T       P+               4

_______________________________________________
midcom mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/midcom

_______________________________________________
midcom mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/midcom

_______________________________________________
midcom mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/midcom

---

From: [email-masked]
Date: Sun, 4 Aug 2002 22:51:02 -0500

Mary,

Before I saw your last two messages, I sent a single e-mail responding
to your first e-mail.

I'm travelling tomorrow, but I expect to arrive early enough at my
destination that I can respond to the last two e-mails by, say, midnight
tomorrow.

Jim

"Mary Barnes" <[email-masked]> on 08/02/2002 08:02:08 PM

Sent by:  "Mary Barnes" <[email-masked]>

To:   James Renkel/MW/US/3Com
cc:   "'midcom'" <[email-masked]>
Subject:  Deadline of Aug. 5th (was RE: Proposed changes to RSIP evaluation
       (was RE: [midcom] Comments on protocol evaluation draft

Hi Jim,

Monday, August 5th, so that we have a day or so to discuss on the list
prior

If you could post each individual requirement in separate emails to the
list, that will facilitate my tracking of resolution of the discussion.
The
Subject line should be similar to the following:
   Protocol eval: Section x.y.z - <Short description of change/categoy>

Thanks,
Mary.

-----Original Message-----
From: Barnes, Mary [NGC:B602:EXCH]
Sent: Friday, August 02, 2002 9:48 AM
To: Taylor, Tom-PT [CAR:B602:EXCH]; [email-masked]'
Cc: 'midcom'
Subject: Proposed changes to RSIP evaluation (was RE: [midcom] Comments

Jim et al,

I've not thoroughly reviewed all the proposed changes in detail, but will
do

2.2.7 - I can't quite see how this goes from F to P+; you'll need to
provide

Upon another quick glance, I think many of the proposed changes for the
other protocols are bringing forth issues with other "scores" that we have
already discussed (e.g. 2.2.2 for Megaco)  Certainly, if there is
overwhelming support for your changes, then they can be done, but we've got
to have more than one person supporting the change for something that we
have previously discussed and I certainly thought had reached concensus on.
I won't be responding to each of these that we've already discussed, you
can
search the email archives, where each has been addressed in a separate
email
thread.

Regards,
Mary.

-----Original Message-----
From: Taylor, Tom-PT [CAR:B602:EXCH]
Sent: Friday, August 02, 2002 9:01 AM
To: [email-masked]'; Barnes, Mary [NGC:B602:EXCH]
Cc: 'midcom'
Subject: RE: [midcom] Comments on protocol evaluation draft

From my point of view you raise two issues I cannot leave unanswered.  The
first has to do with P+ extensions and your first scoring inconsistency,
the
second with security.

What we do have to think about is whether the threats are different for the
MIDCOM application from those considered when the candidate protocol was
being designed.  If so, the security architecture for that protocol may
have
to be modified to fit MIDCOM application.  But the security section of the
MIDCOM requirements document reflects the results of the MIDCOM threat
analysis.  Thus if our candidate protocols, including their specified
security architectures, can meet these requirements in a practical way,
they
must be scored as passing those requirements.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Friday, August 02, 2002 12:45 AM
To: Barnes, Mary [NGC:B602:EXCH]
Cc: 'midcom'
Subject: RE: [midcom] Comments on protocol evaluation draft

Protocol        T       P+      P        F
--------        ---     ---     ---      ---
SNMP    - 1     + 1
RSIP    + 2     +10     - 3     - 9
MEGACO  - 7     + 7
DIAMETER        - 8     + 9     - 1
COPS    - 1     + 1
        ---     ---     ---     ---
Total   -15     +28     - 4     - 9

Protocol        T       P+      P        F       Total

Jim

================

                       Score