Subject: [midcom] Comments on protocol evaluation process

From: [email-masked]
Date: Sun, 4 Aug 2002 23:32:56 -0500

Middlebox experts,

As intimated in several of my recent e-mails to this list, I have a
general concern about the protocol evaluation process that we're just
finishing. This is beyond any specific issues of what score was given
to a protocol for a particular requirement.

This concern can perhaps be best expressed as a question.

In evaluating these protocols, are we:
- trying to get the best insight into and information about the
  protocols as they exist? or are we
- trying to maximize the scores that the protocols receive?

I certainly hope that we are trying to do the former, but I fear we are
doing the latter.

There are two things that cause my fear.

The first is the grading system that we are using. After some discussion
on the list, the scoring systme that we are now using, as I understand
it, is:
T  - the requirement is satisfied by the protocol as is
P+ - the requirement is satisfied when the protocol is extended with a
     backward compatible extension
P  - the requirement is satisfied only with a non-backward compatible
     extension to the protocol
F  - the requirement can not be met by the protocol, even with an
     extension (perhaps because the protocol is not extensible)

We have a requirement for protocol extensibility, and all the protocols
satisfy it. So I don't see how any of these protocols can receive an F
score for any other requirement. I mean it's not like middleboxes are
made of matter and the protocols of anti-matter so that if ya put them
together they explode or something.

Since all the protocols were presumably pretty well designed, and
extensions can generally be made to be backward compatible, I would be
surprised if any scores of P are given.

So what we are left with is T and P+. One says the protocol already does
something, the other says it can be extended to do so. While this is
interesting, this doesn't give the insight to and information about the
protocol that I was hoping to get out of this evaluation.

I mean, so a protocol gets a P+ score for some requirement. Does that
mean that it had nothing, and everything had to be added? Or does it
mean that it fell just a smidge short and only a little has to be added?

I for one would like to know the difference between these two cases. But
the difference is lost with the scoring system we are using.

Information of this same kind if further being lost, I believe, by
allowing protocols a recieve a T score for a security requirement by
saying, "TLS or IPsec provide the capability, so just run the protocol
over them."

Like all the protocols being extensible, all the protocols can run over
TLS or IPsec. But what "native" security capabilities do they have?

TLS or IPsec is fine if I need all the security capabilities they offer
and I'm willing to pay the price to run them. But what if I don't need
all the capabilities they offer or I'm reluctant to pay the price to run
them? I'd sure like to know what security I get from a protocol natively,
with it running over TLS or IPsec.

Equivalently, I'd like to know how much a protocol needs to be extended
to get natively a given level of security, just as I'd like to now how
much a protocol has to be extended to meet other MIDCOM requifrements.

And maybe isn't a protocol that has a particular security capability as a
native capability better in some sense than a protocol that has to rely
on TLS or IPsec for that capability? I think I'd like to know that.

Maybe, at least in this case, that information is contained in the text
accompanying the score. But I'd like to see it reflected in the protocols
score also.

So what to do.

Well, what I'd idewally like to see is a protocol evaluation that is
scored with this scoring system:
T - the requirement is fully satisfied by the protocol as is and
    without the use of TLS or IPsec
P - the requirement is partially met by the protocol as is, without the
    use of TLS or IPsec
F - the requirement is not met by the protocol as is without the use of
    TLS or IPsec

I don't think we should change what we're doing now, or hold it up to add
this to it. But, if other folk also think this would be valuable, maybe we
should add this later.

Comments welcome, expected, and appreciated.

Jim

_______________________________________________
midcom mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/midcom

---

From: Melinda Shore <[email-masked]>
Date: Mon, 05 Aug 2002 09:46:54 -0400

> score for any other requirement.

Just because a protocol is "extensible" doesn't mean that
it's arbitrarily extensible or that an extension in a
particular direction isn't awkward or unidiomatic.  

On the scoring system: what we're trying to do is get a
rough, general sense of how closely a particular protocol
matches the requirements and framework we've established.
This process is necessarily not particularly nuanced, and I
don't think that's a bad thing, either.  After all, one of
the first things we're going to have to do is NOT to choose
the closest fit, but rather to eliminate the weakest
candidates.

Melinda