Subject: Re: [conex] draft-ietf-conex-concepts-uses: New Intro text

From: Bob Briscoe <[email-masked]>
Date: Mon, 04 Jul 2011 16:02:27 +0100

Georgios,

Yes, it's true that domains wouldn't act on unverifiable information.

ConEx is designed to solve that problem; to provide downstream 
congestion measurements at domain boundaries that are verifiable and 
that can be used in bulk without per-flow processing or state.

To achieve that, unfortunately we still need the ConEx audit 
function, which no-one has yet been able to do without per-flow 
state. However, very importantly:
- per-flow ConEx audit is only necessary at the last egress edge of 
the internetwork. Every flow does not need to be checked at borders.
- ConEx audit does not aim to control the rate behaviour of flows, it 
is solely a per-flow information check. This means no-one needs to 
define how individual flows should behave, which is an important achievement.

ConEx audit checks on a per-flow basis that signalling of expected 
congestion is no less than actual congestion. This ensures the 
integrity of the ConEx information so that it can be used for traffic 
management in bulk, e.g at domain boundaries or at the boundary with 
the consumer at the sender end.

(Of course, a network operator could still choose to do per-flow 
traffic management. However if their goal is to control traffic cost 
or to control one consumer's impact on others it is sufficient for a 
network operator to use ConEx information to manage traffic behaviour 
in bulk, not per flow.)

Bob

At 07:39 01/07/2011, Georgios Karagiannis wrote:
>Hi Bob, Hi Michael,
>
>After discussing this issue with Dimitri (also a co-author of RFC 6077) I
>understood that the problem related to the need of network flow state
>in multi-domain scenarios, during congestion control, comes from the
>"non-cooperative" behavior and untrustable nature of the information:
>whether congestion control information exchanges are "implicit" or
>"explicit" and "rate-based" vs "indication/marks" it doesn't
>actually matter: enforcement at the level were the information is
>referring to/referrent (can be per flow or per aggregate or other) will
>be performed at domain boundaries. Meaning that domain edges do never
>trigger any action from "external" information that is not verifiable
>(BGP is the only exception but we would speak at the routing level here).
>
>What RFC 6077 actually mentions wrt to scalability is that if signaling
>is performed on a per flow state basis it will in addition impact
>scaling of domain edges.
>
>Best regards,
>Georgios
>
>
>On 6/26/2011, "SCHARF, Michael" <[email-masked]>
>wrote:
>
> >Georgios, Bob,
> >
> >I've not followed this discussion. But as another author of RFC 6077 I
> >want to back Bob's explanation concerning the context. Those sentences
> >indeed discuss open issues in case that the network performs flow-level
> >congestion control (XCP, RCP, etc.).
> >
> >Michael
> >
> >
> >> -----Original Message-----
> >> From: [email-masked] [mailto:[email-masked]]
> >> On Behalf Of Bob Briscoe
> >> Sent: Sunday, June 26, 2011 11:50 AM
> >> To: Georgios Karagiannis
> >> Cc: [email-masked]
> >> Subject: Re: [conex] draft-ietf-conex-concepts-uses: New Intro text
> >>
> >> Georgios,
> >>
> >> The context of those sentences in RFC6077 (which incidentally
> >> I wrote), was to prove that *if* the network wants to do
> >> congestion control, claims that it can do this without flow
> >> state (e.g. with
> >> XCP/RCP) overlook the need for inter-domain policing, which
> >> will require flow state.
> >>
> >> The statement about flow state wasn't intended to be taken
> >> out of context as a general statement that identifying flows
> >> is always unnecessary. It's only in the context where the
> >> network is already trying to control congestion at the flow level.
> >>
> >> [Anyway, in general (but not in this case) just because a
> >> previous IRTF document (RFC6077) says something is
> >> impossible, doesn't mean a later IETF doc cannot standardise
> >> something that makes it possible.]
> >>
> >>
> >> Bob
> >>
> >> At 23:55 25/06/2011, Georgios Karagiannis wrote:
> >> >Hi Bob,
> >> >
> >> >Sorry for the delay on sending comments!
> >> >
> >> >I have a comment regarding the following paragraph:
> >> >
> >> > >    With ConEx there is no need for the network provider
> >> to identify
> >> > >    specific applications or behaviours at the flow level,
> >> because the
> >> > >    relevant bulk congestion information is revealed at the network
> >> > >    layer.  Also, because ConEx information is added
> >> explicitly at the IP
> >> > >    layer, it is visible to provider and consumer alike.  Therefore
> >> > >    traffic contracts or acceptable use policies can be
> >> based on a metric
> >> > >    that is transparent to both parties and is sufficient to manage
> >> > >    traffic without extra non-transparent wriggle-room and caveats.
> >> >
> >> >In the paragrah above it is mentioned that the network provider to
> >> >identify specific applications or behaviours at the flow
> >> level, because
> >> >the relevant bulk congestion information is revealed at the network
> >> >layer.
> >> >
> >> >However, according to RFC 6077, in multi-domain scenarios,
> >> due to the
> >> >non-cooperative nature of multi-domain  environments, it
> >> seems unlikely
> >> >that network flow state could be  avoided (up to a certain extend)
> >> >given the network's per-packet flow  rate instructions that
> >> would need
> >> >to be compared against variations  in the actual flow rate, which is
> >> >inherently not a per-packet metric.
> >> >
> >> >For RFC 6077, see:
> >> >http://www.rfc-editor.org/rfc/rfc6077.txt
> >> >
> >> >Is it possible to explain in the draft how the congestion control
> >> >related issues associated with multi-domain scenarios
> >> discussed in RFC
> >> >6077 are solved/worked out?
> >> >
> >> >Best regards,
> >> >Georgios
> >> >
> >> >
> >> >
> >> >
> >> >On 6/14/2011, "Bob Briscoe" <[email-masked]> wrote:
> >> >
> >> > >ConEx folks,
> >> > >
> >> > >After the last IETF in Prague, we agreed to work on a revised
> >> > >draft-ietf-conex-concepts-uses. We're planning on using a
> >> lot of the
> >> > >text as is, but we suggested that the Introduction needed
> >> a complete
> >> > >re-write. Below is our joint proposal.
> >> > >
> >> > >Please bash.
> >> > >
> >> > >Rationale for this Introduction:
> >> > >===============================
> >> > >* The main change is a shift to the main target use-case
> >> in the body
> >> > >of the document: traffic management, leaving defining
> >> congestion etc
> >> > >to the definitions and concepts section (section 2).
> >> > >* Introduce enough of the solution to allow the reader to grasp
> >> > the main logic.
> >> > >* Introduce one primary use and mention there are others - to keep
> >> > >focused but hint at breadth.
> >> > >* Start to weave in some benefits bullet points
> >> (transparency, neutrality).
> >> > >* Hint at controversies like the over-provisioning issue
> >> in the first
> >> > >para, but leave addressing this to later, when we can go into the
> >> > >subject properly.
> >> > >
> >> > >
> >> > >
> >> >
> >> >/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
> >> > >\
> >> > >1.  Introduction
> >> > >
> >> > >    The power of Internet technology comes from multiplexing shared
> >> > >    capacity with packets rather than circuits.  Network operators
> >> > >    usually provide sufficient capacity, but whenever too
> >> much packet
> >> > >    load meets too little shared capacity, congestion results.
> >> > >    Congestion appears as either increased delay, missing
> >> packets or
> >> > >    packets explicitly marked with ECN [RFC3168].  The
> >> classic Internet
> >> > >    architecture is arranged so that receivers detect such signs of
> >> > >    congestion and feed them back end-to-end.  Then, ideally, most
> >> > >    senders reduce their rate in response.
> >> > >
> >> > >    The purpose of this document is to motivate a new
> >> congestion exposure
> >> > >    (ConEx) field at the IP layer.  The idea is to add a
> >> third pass to
> >> > >    the feedback loop so that the sender can relay
> >> congestion information
> >> > >    back into the internetwork layer, exposing the level
> >> of congestion
> >> > >    the sender expects packets to experience along their whole path
> >> > >    (Figure 1).  Then certain network nodes can use this
> >> new information
> >> > >    at the IP layer, for example as input to traffic management.
> >> > >
> >> > >    ,---------.
> >>    ,---------.
> >> > >    |Transport|
> >>    |Transport|
> >> > >    | Sender  |
> >>    |Receiver |
> >> > >    |
> >> |<==Feedback=Path==============================<|         |
> >> > >    |     ,---|<--Transport Layer returned Congestion
> >> Signal-<|<--.     |
> >> > >    |     |   |
> >>    |   |     |
> >> > >    |     |   |
> >>    |   |     |
> >> > >    |     |   |             ,-----------.
> >>    |   |     |
> >> > >    |     |   |             |(Congested)|
> >>    |   |     |
> >> > >    |     |   |>=Data=Path=>|  Network
> >> |>=====Data=Path=====>|   |     |
> >> > >    |     |   |             |  Device
> >> |>-Congestion-Signal->|---'     |
> >> > >    |     |   |             `-----------'
> >>    |         |
> >> > >    |     `-->|>---------(new) IP layer ConEx
> >> Signal--------->|         |
> >> > >    |         |        (Carried in Data Packet Headers)
> >>    |         |
> >> > >    `---------'
> >>    `---------'
> >> > >
> >> > >    Figure 1: Where the ConEx Protocol Fits in the Internet
> >> > > Architecture
> >> > >
> >> > >    This document serves as the entry point to the set of ConEx
> >> > >    documentation, giving the 'why' rather than the 'how'.
> >>  A companion
> >> > >    document outlines the ConEx protocol mechanism
> >> [ConEx-Abstract-Mech].
> >> > >
> >> > >    The main aim of ConEx is to support evolution of
> >> alternatives to the
> >> > >    proliferation of traffic management solutions that are over-
> >> > >    constraining, non-transparent and often not
> >> application-neutral.
> >> > >    Traffic management ought to be able to leave end
> >> systems free to
> >> > >    choose how to behave without undue interference, while
> >> simultaneously
> >> > >    satisfying the main concern of network operators; to
> >> control traffic
> >> > >    from some users that excessively limits the freedoms of others.
> >> > >
> >> > >    Freedoms only actually collide when shared capacity becomes
> >> > >    congested--when a link is full so that a greater share
> >> for one user
> >> > >    would necessarily leave less for someone else.  Current traffic
> >> > >    management solutions typically limit volume or
> >> bit-rate in order to
> >> > >    reduce the likelihood of congestion--limiting one
> >> thing in the hope
> >> > >    it might limit another.  However, there is no real
> >> need to count
> >> > >    volume that is sent when there is no congestion, and
> >> there is no real
> >> > >    need to limit bit-rate when there is no congestion.
> >> > >
> >> > >    ConEx markings on packets add the missing information network
> >> > >    operators need to get a handle on congestion itself.
> >> Then they can
> >> > >    directly limit and control traffic based on how much
> >> it contributes
> >> > >    to congestion and leave traffic unconstrained if it
> >> doesn't cause
> >> > >    congestion.  The idea is for the operator to be able
> >> to count and
> >> > >    control the bulk volume or bit-rate of packets marked
> >> for congestion
> >> > >    and disregard packets not contributing to congestion.
> >> > >
> >> > >    With ConEx there is no need for the network provider
> >> to identify
> >> > >    specific applications or behaviours at the flow level,
> >> because the
> >> > >    relevant bulk congestion information is revealed at the network
> >> > >    layer.  Also, because ConEx information is added
> >> explicitly at the IP
> >> > >    layer, it is visible to provider and consumer alike.  Therefore
> >> > >    traffic contracts or acceptable use policies can be
> >> based on a metric
> >> > >    that is transparent to both parties and is sufficient to manage
> >> > >    traffic without extra non-transparent wriggle-room and caveats.
> >> > >
> >> > >    In summary, ConEx is designed to make it simple to do traffic
> >> > >    management that is transparent, neutral and not
> >> over-constrained.
> >> > >    Although there is no implication that network
> >> operators ought to
> >> > >    provide such an unconstrained, transparent, neutral service, it
> >> > >    certainly should not be impossible and ideally it should be the
> >> > >    simplest service to provide.
> >> > >
> >> > >    The IP layer is intended to engender new applications
> >> and behaviours
> >> > >    and to work over all existing and new lower layer technologies.
> >> > >    ConEx is a generative technology in this vein, with a range of
> >> > >    potential uses.  This document focuses initially on traffic
> >> > >    management before widening out to introduce some
> >> additional important
> >> > >    uses for ConEx as well as brifly mentioning a few others.
> >> >
> >> >/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
> >> > >\
> >> > >
> >> > >
> >> > >Bob Briscoe & Rich Woundy
> >> > >
> >> > >
> >> > >
> >> > >_______________________________________________
> >> > >conex mailing list
> >> > >[email-masked]
> >> > >https://www.ietf.org/mailman/listinfo/conex
> >>
> >> ________________________________________________________________
> >> Bob Briscoe,                                BT Innovate & Design
> >>
> >> _______________________________________________
> >> conex mailing list
> >> [email-masked]
> >> https://www.ietf.org/mailman/listinfo/conex
> >>

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Mon, 04 Jul 2011 18:45:05 +0000

Hi Bob,

Please see in line!

On 7/4/2011, "Bob Briscoe" <[email-masked]> wrote:

>state. 

Georgios: Is it not necessay to use a policer?

> However, very importantly:

Georgios: Okay, but I assume that whenever the congestion rate needs to
be measured, then a per-flow state is needed. Please check the following
draft that somehow tries to describe this issue:
http://www.ietf.org/id/draft-karagiannis-conex-throughput-exposure-01.txt

>define how individual flows should behave, which is an important achievement.

Georgios: okay, but I assume that when a policer is used, then the
control of the rate behaviour will be affected!

>network operator to use ConEx information to manage traffic behaviour

Georgios: Do you mean that Conex will not require the use of a per-flow
policer?

Best regards,
Georgios

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 05 Jul 2011 10:11:33 +0100

Georgios,

[Mike, I've also directed this response to you - please see "{Note 
1}" inline that answers a question I recall you asked months ago. I 
don't think I ever answered it - probably got distracted by something.]

>Georgios: Is it not necessay to use a policer?

Three points:
a) I assume you are using the terms 'policer' and 'audit function' as 
in conex-abstract-mech?
b) It isn't necessary to use a policer between networks. Indeed, I 
would not recommend a policer between two networks that both use 
ConEx info to protect their respective networks.{Note 1} A traffic 
contract with penalties for sending too much congestion will be sufficient.
c) A policer does not need to use flow-state. It can act on the bulk 
of traffic passing from the consumer to their ingress network.

>be measured, then a per-flow state is needed.

Not at all. Just like bit-rate, congestion rate is a metric that can 
measure any granularity: flows, aggregates or all the traffic on a 
link. ConEx markings are designed to correctly characterise the 
amount of traffic causing congestion at any granularity - without 
having to measure at finer granularities and add them up.

For instance, a "congestion token bucket policer" measures the 
congestion rate and it can be applied at the granularity of a link 
without looking at flows. Just like a normal token bucket can be 
applied without looking at flows. Whenever a ConEx marked packet 
arrives, tokens are removed from this bucket, without caring about flows.

>Please check the following

Ah! Your draft *assumes* that the congestion rate is being measured 
per flow, then essentially it says it would be complex to measure the 
congestion rate per flow.

I think you've seriously misunderstood. Although ConEx *could* be 
used to make sure individual flows conform to the TCP behaviour, that 
is neither necessary nor the goal. In fact it is a non-goal.{Note 2}

The argument is that it would actually be very wrong to try to get 
flows to conform to the TCP behaviour. Embedding such policers in the 
network would prevent future evolution of better congestion controls.

I will add this to the list of misconceptions in conex-uses.

>control of the rate behaviour will be affected!

Of course. But if it is a bulk policer, it only limits the bulk rate 
- it imposes the same level of discard for all flows.

If the audit function detects non-compliance, it will drop packets 
from the non-complying flow(s), which of course reduces their rate. 
But this is to make the outgoing congestion *information* compliant, 
not because it has a particular view of how each flow should make 
their rate respond to congestion. This complies with the requirement 
in conex-abstract-mech:

"Transport Oblivious:
Audit MUST NOT be designed around one particular rate response, such 
as any particular TCP congestion control algorithm or one particular 
resource sharing regime such as TCP-friendliness 
<http://www.jungle.bt.co.uk/people/rbriscoe/ext/projects/conex/draft-ietf-conex-abstract-mech-01.html#RFC3448>[RFC3448]. 
An important goal is to give ingress networks the freedom to 
unilaterally allow different rate responses to congestion and 
different resource sharing regimes 
<http://www.jungle.bt.co.uk/people/rbriscoe/ext/projects/conex/draft-ietf-conex-abstract-mech-01.html#Evol_cc>[Evol_cc], 
without having to coordinate with downstream networks; "

[BTW, when I talk about the audit function, you keep switching to the 
policing function. Are you sure you are using the terms as defined in 
conex-abstract-mech?]

>Georgios: Do you mean that Conex will not require the use of a per-flow

Very definitely yes. That is the main goal of the whole exercise. If 
you missed this point, I can understand why you wrote your draft. 
This is why I have been arguing that conex-concepts-uses needs to say 
what we are NOT trying to do.

Footnotes
_________
{Note 1}: If only the downstream network uses ConEx, then it may not 
be able to get the upstream network to agree to a traffic contract 
that includes ConEx information. But it would still want to deploy a 
ConEx policer at the border.

This policer could choose not to use flow-state. However, then one 
upstream sender could cause the policer to hurt traffic from other 
upstream senders. Two possibilities:
i) The downstream network might say to the upstream network "Tough, 
if you want to protect your users from DoS attackers on your network, 
police them using ConEx info."
ii) However, more likely, the downstream network will want a 
congestion policer that does some sampling to detect ConEx flows that 
are causing excessive congestion, and it will separate them out to be 
the ones that discards are focused on whenever the whole bulk of 
traffic is out of contract.

If a DoS attacker was spoofing different source addresses, this 
policer could do full per-flow processing (like DoS protection does 
today). Then it would be easy to discard any packets that say they 
are ConEx enabled but belong to a flow that has not started with 
sufficient positive ConEx markings (green or black). For instance 
ConEx-enabled SYNs without a 'green' marking could automatically be discarded.

But if I was the downstream network, I would charge the upstream 
network for sorting out their DoS problems for them (that would pay 
for the equipment I would buy to do all this flow-processing). If 
they don't want to pay, they always have option (i) available to them.

Option (i) is the goal, because once the upstream network deploys a 
ConEx policer at the ingress from the sender's network, it doesn't 
need to do flow processing. If any DoS sources within the sender's 
network affect other traffic from that same sender, the problem is 
confined to the sending network hurting itself.

{Note 2}: The original SIGCOMM paper on re-feedback may have caused 
your confusion. Because the prevailing religion of most SIGCOMM 
reviewers at the time assumed the goal was to police individual TCP 
behaviour, to get the paper accepted we had to show that re-feedback 
could do this. But then we showed you didn't need to in a following 
section (a later paper "Policing Freedom..." focused on bulk policing).

If the source of confusion was one of the current ConEx w-g 
documents, not this SIGCOMM paper, please say. We will need to fix that.

Cheers

Bob

---

From: John Leslie <[email-masked]>
Date: Tue, 5 Jul 2011 11:48:58 -0400

Bob has done a rather good introduction to policing here. I can
only add the view of an operator: that I would leave all policing off
by default.

   Turning it on would, in all likelihood, be at the request of a
_customer_ who was experiencing what amounts to a mini-DoS -- where
the link he's willing to pay for is getting overloaded. (Alas, in
the US it's far cheaper to do policing than upgrade a link.)

   I might also turn it on for what amounts to "transit" -- where
traffic coming from a peer exceeds my (non-contractual) allowance
for _volume_ of congestion-expected, and I want to ensure that I
don't exceed my (contractual) allowance going out to a different
peer. Generally, though, it would be easier to police the outgoing
traffic to peers where I had a contractual allowance.

   Only in the case where I'm doing it for a customer would I care
to try to match policing to individual flows. (And only on the
outgoing link to a customer would I consider that practical -- but
even if it were practical I wouldn't want to be bothered.)

   I believe this issue deserves to be in a separate draft. It's
inherently confusing, and IMHO it's far better for an introductory
RFC to mention it _only_ by reference.

Bob Briscoe <[email-masked]> wrote:
>> Is it not necessay to use a policer?
> 
> Three points:
> 

   This gives the sending network an incentive to do policing, which
hopefully wouldn't need to drop/disincent anything most of the time.

>> Okay, but I assume that whenever the congestion rate needs to be
>> measured, then a per-flow state is needed.

   As an operator, I see no reason to measure "congestion rate" at
all. It does nothing to help me. Congestion-Expected _volume_ is the
metric I'm interested in.

>> Please check the following draft that somehow tries to describe this
>> issue:

   (It might be helpful to list such "non-goals" in an introductory
RFC...)

> I will add this to the list of misconceptions in conex-uses.

   I continue to recommend against any such list in Concepts-Uses.
Listing "non-goals" is far better. "Misconceptions" tends to pick a
fight with anyone who believes the conception.

>> okay, but I assume that when a policer is used, then the control of
>> the rate behaviour will be affected!
> 

   Bulk Policer is the easiest to implement, and it _is_ sufficient
for many needs.

> their rate respond to congestion. This complies with the requirement 

   Possible confusion: to me "audit function" implies _only_ measuring.
Dropping packets belongs to some other function.

> <http://www.jungle.bt.co.uk/people/rbriscoe/ext/projects/conex/draft-ietf-conex-abstract-mech-01.html...

   I'd really prefer not to get bogged down in minutiae of Abstract-Mech.

> policing function. Are you sure you are using the terms as defined in 

   In fairness, Bob, there _isn't_ a definition of "audit function" in
Abstract-Mech. And there are enough occurances of "sanction" in Section
4.3 to make it _very_ easy for Georgios to believe "audit funciton"
includes policing.

   But, as I said, I'd really prefer not to get bogged down in minutiae
of Abstract-Mech. (And we _definitely_ shouldn't be trying to solve
ambiguity of Abstract-Mech in Concepts-Uses!)

> ...

   "might want", please.

>     congestion policer that does some sampling to detect ConEx flows
>     that are causing excessive congestion, and it will separate them
>     out to be the ones that discards are focused on whenever the
>     whole bulk of traffic is out of contract.

   Per-flow state at a major peering point would have to be limited
to a "reasonable" number of flows...

> ConEx-enabled SYNs without a 'green' marking could automatically be 

   (Technical issue here, Bob: I'll let you work it out...)

> they don't want to pay, they always have option (i) available to them.

   Agreed.

> confined to the sending network hurting itself.

   Agreed (though in DDos, an attacker might find enough botted
computers to cause trouble without any of them exceeding their allowance.)

> documents, not this SIGCOMM paper, please say. We will need to fix

   I do recommend some attention to Section 4.3 of Abstract-Mech, Bob.

--
John Leslie <[email-masked]>

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 05 Jul 2011 21:34:48 +0100

John,

>RFC to mention it _only_ by reference.

Agree.

We (rightly) have nothing specific on policing in conex-concepts-uses.
We have some overview in conex-abstract-mech.

A good scholarly paper on congestion-policing from someone would be 
useful to refer to. Especially if they were willing to donate the 
text to the IETF, for the w-g to knock into an RFC. I'm working on 
such a thing, but only in my spare time (huh!).

>metric I'm interested in.

I glossed over that point. I imagine a token-bucket as the mechanism, 
or preferably a leaky-token-bucket. The contracted rate of the bucket 
can be thought of as limiting congestion-volume over a duration, so 
in that sense I would agree with John that the instantaneous 
congestion-rate at any one time would not concern an operator.

However, an operator would want to limit the size of bursts of 
congestion-rate. That's where the leaky part of the 
leaky-token-bucket would come in handy. So in that sense (and only 
that), an operator would be interested in congestion-rate.

>fight with anyone who believes the conception.

OK, I see your point, but it's made without knowing the proposed text 
- I'll start a new thread next email.

>    Bulk Policer is the easiest to implement, and it _is_ sufficient

Indeed. Once someone is out of order, it doesn't so much matter if 
you punish some of their flows more than they deserve and others 
less. Their remedy is to get back in contract, then it won't matter.

>Dropping packets belongs to some other function.

True. I should not have said "it will drop packets".

>ambiguity of Abstract-Mech in Concepts-Uses!)

Indeed.

I don't think Georgios is particularly focusing on concepts-uses. He 
seems to be trying to understand ConEx from reading lots of docs 
(including non-w-g stuff I think).

Nonetheless, under Requirements, abstract-mech does say:
"The auditing mechanism must have a capability for providing 
sufficient disincentives against misreported congestion, such as by 
throttling traffic that reports less congestion than it is actually 
experiencing. "

> > ConEx policer at the border.

Yup

>to a "reasonable" number of flows...

Yup. Hence sampling.

>computers to cause trouble without any of them exceeding their allowance.)

All ConEx can claim to do is raise the bar - by n times to be 
precise, where n = 1/p* and p* is the typical average congestion 
level the operator designs for over the peak period (without DDoS). 
So if p* = 0.1%, then n = 1000. That means ConEx would require 
botnets to conscript 1000x more bots for the same effect.

n would be less if the policers were at borders remote from the bots, 
sized for many users behind whole networks. That's harder to work out 
(meaning I haven't tried).

>    I do recommend some attention to Section 4.3 of Abstract-Mech, Bob.

Specifically?

I do notice that it doesn't actually say what Audit is. It goes 
straight into the middle of a conversation about implementation.

We (matt & I) have an -02 nearly ready. But I'm focusing on 
concepts-uses at the mo.

Bob

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Wed, 06 Jul 2011 05:18:21 +0000

Hi Bob,

Thank you very much for the detailed explanation.

I will try to explain my thoughts on why I think that a Conex congestion
policer needs per flow state.

The main solution proposed in [draft-ietf-conex-abstract-mech-01] to
audit Conex Signals against actual losses (i.e., the measured congestion
rate) an auditor could use one of the following tecniques "The auditor
could monitor TCP flows or aggregates of flows, only holding state on a
flow if it first sends a Credit or a Re-Echo-Loss marking. The auditor
could detect retransmissions by monitoring sequence numbers.  It would
assure that (volume of retransmitted data) <= (volume of data marked
Re-Echo-Loss).  Traffic would only be auditable in this way if it
conformed to the standard TCP protocol and the IP payload was not
encrypted (e.g. with IPsec). ", from
[draft-ietf-conex-abstract-mech-01].

  In other words, an audit device needs to keep per flow (individual or
aggregated) state after it receives a credit or Re-echo-Loss marking in
order to measure the per flow congestion rate. In addition to this  the
audit device must be able to read the TCP header in order to observe the
TCP header numbers. This is complex to be implemented and it is not
always possible, e.g., when the IP payload is encrypted (e.g. with
IPsec). In this way, the audit device is also able to take into
consideration also the number of ECN CE marked packets that were dropped
by nodes located upstream.

 Now regarding the Conex congestion policer,
[draft-ietf-conex-abstract-mech-01] specifies: "A congestion policer
can be implemented in a very similar way to a bit-rate policer, but its
effect can be focused solely on traffic causing congestion downstream,
which ConEx signals make visible. Without ConEx signals, the only way to
mitigate congestion is to blindly limit traffic bit-rate, on the
assumption that high bit-rate is more likely to cause congestion.
A congestion policer monitors all ConEx traffic entering a network, or
some identifiable subset.  Using ConEx signals, it measures the amount
of congestion that this traffic is contributing to somewhere downstream.
 If this exceeds a policy-configured 'congestion-bit-rate' the
congestion policer will limit all the monitored ConEx traffic.", from
from [draft-ietf-conex-abstract-mech-01].

 My concern is related to the fact that the conex congestion plolicer
will use ConEx signals, as a prediction of the amount of congestion that
this traffic is contributing to somewhere downstream, and then police
traffic. Note that policing traffic might severely affect the rate
behaviour of traffic.

What I am actually saying is that in the downstream path from a sender
towards the Conex congestion policer data packets will be dropped and
lost. These lost packets contribute to the amount of congestion that
this traffic is contributing to somewhere downstream, but they are not
represented by the received Conex Signals. Thus the Conex Policer will
police traffic based on a somehow inaccurate prediction of the amount of
congestion that this traffic is contributing to somewhere downstream.
In
http://www.ietf.org/id/draft-karagiannis-conex-throughput-exposure-01.txt
it is explained that this inaccuracy could be solved by using the same
method as the one applied in audit devices, i.e., using per flow state
and measuring the rate of lost packets associated with a certain flow.
In this way the Conex congestion policer can use the information provided
by the Conex Signals and the rate of lost packets to accurately predict
the amount of congestion that this traffic is contributing to somewhere
downstream

 So the open issues associated with these concerns are:

 What is the accuracy of the Conex Signals on representing the amount of
congestion that traffic is contributing to somewhere downstream?

 How the location of the Congestion Policer and Audit devices influence
the accuracy of the Conex Signals?

 It will be great if these open issues could be clarified in both drafts
(use cases and [draft-ietf-conex-abstract-mech-01])

Best regards,
Georgios

On 7/5/2011, "Bob Briscoe" <[email-masked]> wrote:

>Bob Briscoe,                                BT Innovate & Design )

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 06 Jul 2011 10:07:56 +0100

Georgios,

[I've changed the subject line to be more relevant.] Inline...

>by nodes located upstream.

A/ Flow state
_____________
Any audit device needs to keep per flow state. I've said that all 
along at the IETF (since the very first presentation of re-ECN at a 
BoF in Nov 2005).

But, be careful not to just use buzz-word evaluation:

There are three potential problems with flow-state in the network:
a. Scalability and cost of required memory
b. Vulnerability to memory exhaustion attacks
c. Vulnerability to source spoofing
d. Violation of the shared-fate design principle

We have designed two audit functions (and implemented one) that hold 
flow state but avoid problems 'b', 'c' and 'd' by design. They still 
have problem 'a' though.

B/ TCP seq nos
______________
You are confusing what audit *needs* with what one type of audit 
*uses*. Let me explain:

Abstract-mech describes three possible ways to implement audit.
In draft-01 they are in S.2 under the para starting
         "It is important to note that the auditing requirement..."
at the first bullet and the second two sub-bullets. So let's call them:
#1 (ECN-based), [has been implemented]
#2a (loss-based TCP-only) [not implemented yet]
#2b (loss-based, single-bottleneck, transport oblivious) [not implemented yet]

Design #1 & #2b prove that:
* Audit does not NEED to read TCP sequence numbers.
Design #2a USES TCP sequence numbers.
It has disadvantages mentioned in abstract-mech (mainly it's specific 
to TCP and the source can make it ineffective by using IPsec). But 
its advantage is that it can be placed near the ingress, so it could 
be useful for initial deployment. But if it started to be the target 
of attacks, it has vulnerabilities. Then audit would have to be done 
at the egress using #2b or #1.

C/ More on flow state
_____________________
All the designs so far have not been able to avoid using flow-state. 
Therefore we can tentatively say:
a: Audit NEEDS to read either the flow 5-tuple or SPI (if IPsec).

We can't be sure though. Just because all current designs need a flow 
ID, does not say that a design without flow ID can never be invented.

Nonethelss, we have avoided the three other problems with flow-state:

b: By introducing credit packets into the ConEx protocol, the audit 
device can protect itself against memory exhaustion attacks

c: Audit does not NEED the source address or port to be true (ie 
routable). It just uses it as a uniqueness label. Audit designs #1 & 
#2b are invulnerable to source spoofing, even different spoofs on each packet.

d: Audit designs #1 & #2b use soft-state in the ConEx markings of 
packets. So if a flow moves to a different audit device on a 
different path, the state automatically gets recreated on the second 
audit device. The same happens if one audit device fails (e.g. power 
off) and another takes over.

>congestion that this traffic is contributing to somewhere downstream.

Despite your confusing use of the word downstream, I think I know 
what you mean. You mean that the policer only handles congestion 
downstream of itself, not upstream of itself?

That is a deliberate design choice.

The insight that originally started the whole re-feedback effort and 
also the ConEx charter, is that a policer at the ingress to network A 
doesn't care about congestion in the network(s) upstream of it. 
Network A thinks "That is your network; that is your problem, not my problem."

Nonetheless, if Network A wants to care about your home network, the 
ConEx *protocol* allows different *policer* design choices.
- The protocol is for standardisation.
- The policer design is an *example*.

The ConEx protocol gives the policer information about congestion 
both upstream and downstream of itself. So the policer COULD police 
congestion in your home, on your behalf. That would just be a 
different policer design.

But you would need to justify why you want your network provider to 
decide what you do in your own home. If you like that idea, well...

>the amount of congestion that this traffic is contributing to somewhere

BTW, when you are talking about a policer, it is confusing to say 
'downstream' when you mean downstream of somewhere else (ie 
downstream of the sender). It would be clearer to say 'whole path' in 
that case.

>congestion that traffic is contributing to somewhere downstream?

I suspect you mean "somewhere downstream of the sender (not the policer)"

If you mean that, you just measure the ConEx markings. They reflect 
losses along the whole path.
So it's accurate. No problem. No issue.

To measure congestion downstream of itself, the policer subtracts ECN 
markings (if available) from ConEx markings.
That's accurate too. No problem. No issue.

If there's no ECN in the home network, the policer can only measure 
whole path congestion (downstream of the sender if you really prefer 
this confusing terminology). That's what you seem to want. But it's 
not what the w-g is designing for.
So it's accurate for your requirement. But for everyone else's 
requirement, the policer accounts for losses in the home network, 
which we don't want it to.

But the inaccuracy in our case is in the safe direction.
* If the home does not deploy ECN, the network polices it more than 
it needs to.
* If the home wants the network to police it less, it should deploy ECN.

>(use cases and [draft-ietf-conex-abstract-mech-01])

I believe I have shown these aren't open issues. They come from you 
wanting to do something different from the charter.

I'm going to stop doing tutorial emails now. I have work to do on drafts.

Bob

> ><http://www.jungle.bt.co.uk/people/rbriscoe/ext/projects/conex/draf 
> t-ietf-conex-abstract-mech-01.html#RFC3448>[RFC3448].
> ><http://www.jungle.bt.co.uk/people/rbriscoe/ext/projects/conex/draf 
> t-ietf-conex-abstract-mech-01.html#Evol_cc>[Evol_cc],

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Wed, 6 Jul 2011 13:43:41 +0200

Hi Bob

Thank again for the explanations! 

In order to minimize the discussion points I will answer only to some
remarks addressed by you!

> Despite your confusing use of the word downstream, I think I know what you
mean. 

> You mean that the policer only handles congestion downstream of itself,
not upstream of itself?

Georgios: No!

According to the text in the [draft-ietf-conex-abstract-mech-01] :

"A congestion policer monitors all ConEx traffic entering a network, or
some identifiable subset.  Using ConEx signals, it measures the amount
of congestion that this traffic is contributing to somewhere downstream.
 If this exceeds a policy-configured 'congestion-bit-rate' the
congestion policer will limit all the monitored ConEx traffic.",

I can derive the following:

The Conex congestion Policer uses the Conex exposure signals to predict the
amount of congestion. 

Based on this, the amount of traffic that contributes to this congestion can
be policed/limited by the Conex congestion policer.

My point is that this prediction could not be 100% accurate. This is because
in the communication paths 

from different sources to the Conex congestion policer, packets (that carry
and do not carry Conex markings) can be dropped.

Here we can distinguish some options:

(Note: Similar to, RFC3168, I am  assuming that some routers may drop
ECN-Capable packets

while other routers set the CE codepoint, for equivalent levels of
congestion.  Similarly, a router might drop a non-ECN-Capable packet

but set the CE codepoint in an ECN-Capable packet, for equivalent  levels of
congestion.)

o) if packets that do NOT carry Conex Exposure signals are dropped (say
dropped_packets_A) AND packets that carry CONEX exposure signals are NOT
dropped, then the policer will limit/police more traffic than it should.
This is because the policer will not be able to know that the amount
dropped_packets_A  is already dropped. So the policer will drop erroneously
an additional amount of traffic equal to dropped_packets_A .

o) if packets that do NOT carry Conex Exposure signals are dropped (say
dropped_packets_A) AND packets that carry CONEX exposure signals are also
dropped, then the policer will limit/police an amount of traffic that might
be different than the targeted amount of traffic that needed to be limited.

o) if packets that do NOT carry Conex Exposure signals are NOT dropped (say
dropped_packets_A) AND packets that carry CONEX exposure signals are
dropped, then the policer will limit/police an amount of traffic that might
be different than the targeted amount of traffic that needed to be limited.

>If you mean that, you just measure the ConEx markings. They reflect losses
along the whole path.
> So it's accurate. No problem. No issue.

Georgios: Yes I mean somewhere downstream of the sender. I think however,
that there is an accuracy problem with the ability of the CONEX exposure
markings to accurately represent the congestion rate on the path from the
sender to receiver, see my explanation about the prediction given above.

This inaccuracy could be acceptable, I do not know. 

In order to find this out some investigation is needed. 

This could be clarified  in both drafts
(use cases and [draft-ietf-conex-abstract-mech-01])

Best regards,

Georgios