Subject: [conex] I-D Action: draft-ietf-conex-concepts-uses-02.txt

From: [email-masked]
Date: Mon, 11 Jul 2011 17:07:01 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Congestion Exposure Working Group of the IETF.

	Title           : ConEx Concepts and Use Cases
	Author(s)       : Bob Briscoe
                          Richard Woundy
	Filename        : draft-ietf-conex-concepts-uses-02.txt
	Pages           : 25
	Date            : 2011-07-11

   This document provides the entry point to the set of documentation on
   a new Congestion Exposure (ConEx) protocol.  It motivates a new ConEx
   field at the IP layer, focusing on the &#39;why&#39; rather than the &#39;how&#39;.
   In the absence of such a protocol, traffic is subjected to numerous
   traffic management checks and limits as it crosses the Internet.  The
   purpose of this protocol is to expose congestion to network
   operators, so that they have no need to intervene at all when there
   is no congestion, and so they have exactly the right information when
   there is congestion.  Then it will at least be possible for traffic
   management to be application-neutral, openly transparent and free of
   unnecessary restraints.  Although traffic management is the focus of
   this document, it also briefly introduces a number of other important
   potential uses for ConEx, demonstrating its role as a generative
   technology and justifying its placement in the IP layer.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-conex-concepts-uses-02.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-conex-concepts-uses-02.txt

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 12 Jul 2011 12:24:20 +0200 (CEST)

On Mon, 11 Jul 2011, [email-masked] wrote:

Hi, I have read this draft. Some comments.

"Users are increasingly seeing congestion at peak times"

This is not my reality. I would like to see this statement backed up by 
data. In my world congestion has decreased over time, not increased.

Also, I don't agree with the whole philosphy and world-view of the text.
Talking about "who causes congestion" just doesn't compute in my world, 
traffic is caused by two parties who want to communicate. Is the 
congestion caused by the web server sending the data packet or the user 
requesting the data packet from the web server? The text seems to want to 
blame congestion on someone, but I can't make out who of the two entities 
I just mentioned is to blame. Is it both?

I also still don't understand congestion rate:

"Conceptually, the instantaneous bit-rate of the traffic multiplied by the 
instantaneous congestion it is experiencing.

What is the multiplier here, is it dependent on queue depth (ie increased 
delay), or is it a 0 or 1 depending on if it's being sent immediately or 
if there was a packet in the output buffer of the router? I don't 
understand the number that is created from the concept of "instananeous 
congestion" and how it's created. This also means I don't understand all 
the other congestion figures either. In my world either a link is 
congested or it isn't (well, a link is little congested a low queue depth 
and low drop rates, and more if either the drop rate or queue depth 
increase), and I don't understand how each flow can contribute more or 
less to this congestion, and how this can be determined. I think this is 
also the reason I don't understand the "ingress policer" and how it's 
supposed to work without understanding flows.

In 5.1, there is talk about blocking traffic. What does this mean? Stop 
the flow completely by means of 100% packet loss?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 12 Jul 2011 13:07:40 +0100

Mike,

This seems to be a mix of comments across two revisions.
Are you sure all your comments apply to the -02 revision?

We removed all the stuff in -01 about increasing congestion being a 
problem (as I said, congestion isn't a problem, it's a signal, and I 
agree, I would be very surprised if congestion is increasing in 
general). I removed the sentence in 5.1 about blocking too.
I've searched -02 and I cannot find the phrases you're complaining 
about (so I'm pretty sure I didn't miss any occurrences).

However, you clearly *have* read the -02 revision if you are 
concerned about the whole conceptual philosophy about blame, which 
was just added in -02.

See inline, for responses on the blame stuff...

At 11:24 12/07/2011, Mikael Abrahamsson wrote:
>This is not my reality. I would like to see this statement backed up 
>by data. In my world congestion has decreased over time, not increased.

No longer in -02 (or if it is, tell us where!)

>Talking about "who causes congestion" just doesn't compute in my 
>world, traffic is caused by two parties who want to communicate. Is 
>the congestion caused by the web server sending the data packet or 
>the user requesting the data packet from the web server? The text 
>seems to want to blame congestion on someone, but I can't make out 
>who of the two entities I just mentioned is to blame. Is it both?

ConEx adds the ability to hold the sender (or sending network) 
responsible for contributing to congestion. That doesn't imply that 
holding the receiver responsible is wrong. Just insufficient. ConEx 
isn't saying you must hold the sender responsible, only that it must 
be possible to be able to, particularly because the sender is the one 
who is more fundamentally responsible for sending (see below). We 
already have the standards in place to expose congestion if we want 
to hold the receiver responsible (ie. ECN).

At the IP layer, the sender is to blame for sending traffic. At the 
transport layer, the receiver is to blame for asking the sender to 
send. But in a packet network we get into all sorts of problems by 
conflating these two layers together and saying receivers cause 
packets to be sent to them. Ultimately:
- senders can choose not to send content to whoever asks for it,
- senders can choose not to send packets,
- senders certainly can (and certainly do) choose how fast to send 
packets when they receive congestion feedback.

THe draft does have text saying that either end can be held 
responsible (e.g. 5.1 discussed egress policing as well as ingress). 
But you are right that this needs explaining earlier, probably in the 
concepts section where blame is first discussed (or whatever word we 
end up using for it).

>"Conceptually, the instantaneous bit-rate of the traffic multiplied 
>by the instantaneous congestion it is experiencing.
>
>What is the multiplier here, is it dependent on queue depth (ie 
>increased delay), or is it a 0 or 1 depending on if it's being sent 
>immediately or if there was a packet in the output buffer of the 
>router? I don't understand the number that is created from the 
>concept of "instananeous congestion" and how it's created. This also 
>means I don't understand all the other congestion figures either. In 
>my world either a link is congested or it isn't (well, a link is 
>little congested a low queue depth and low drop rates, and more if 
>either the drop rate or queue depth increase), and I don't 
>understand how each flow can contribute more or less to this 
>congestion, and how this can be determined. I think this is also the 
>reason I don't understand the "ingress policer" and how it's 

You're right that these definitions aren't helpful if you take the 
word "instantaneous" in its strict sense. You're right that 
instantaneous congestion for a particular packet is either 0 or 1. 
And similarly, the instantaneous bit-rate of one packet isn't a 
useful thing. Strictly the definition needs to say "the product of 
congestion and bit-rate over a short period that tends to instantaneous".

The congestion-rate is just the time-varying rate at which bits are 
lost (either from a flow, a whole link, or whatever granularity you 
choose to measure it at). We just didn't want to call it the 
loss-rate for two reasons:
- loss-rate is already sloppily used to mean the loss probability (in %).
- congestion-rate generalises to loss and ECN.

If you are sending at constant 10Mb/s through a queue of constant 
loss probability 0.1%, you will lose packets at 10kb/s, which is the 
congestion-rate of that flow. But the definition must allow for 
non-constant bit-rate and non-constant congestion. Because it can be 
very INcorrect to say that an average bit-rate of 10Mb/s through 
average congestion of 0.1% will lead to 10kb/s congestion-rate. A 
LEDBAT flow might achieve a much lower congestion-rate by sending 
rate peaks in congestion valleys and rate valleys in congestion peaks.

>In 5.1, there is talk about blocking traffic. What does this mean? 
>Stop the flow completely by means of 100% packet loss?

No longer in -02 (or if it is, tell us where!)

Cheers

Bob

>_______________________________________________
>conex mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/conex

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 13 Jul 2011 07:54:59 +0200 (CEST)

> Are you sure all your comments apply to the -02 revision?

I opened a lot of documents and tabs yesterday to read up on all published 
documents in CONEX, I probably indeed have messed up and had multiple 
revisions.

So, now re-reading 
<http://www.ietf.org/id/draft-ietf-conex-concepts-uses-02.txt> which I 
hope is correct.

I'll take your reply in consideration and start anew on my comments:

So, congestion volume description starts to make sense, for me as a router 
guy this now is "any packet that couldn't be sent immediately (or within a 
very low buffer threshold value) or which was over the rate a policer was 
configured to. This, however, is NOT ECN marking , so basically to 
interoperate with non-ECN traffic, there needs to be a new flag (or one 
needs to start marking/dropping (if it's ECN enabled packet or not) very 
early in the queue depth). Just to make an example, I would like to have 
end systems consider a link congested as soon as a packet was buffered 
more than 1ms, that doesn't mean I want to start dropping non-ECN packet 
at that level, so what do we do? With the current scheme, if I have a WRED 
profile that starts marking/dropping at 30ms queue depth, I'd consider 
this quite heavily congested before the current scheme would do so. So how 
should operators configure their AQM to work best in a conex world? I also 
imagine it should be different depending on link speed and amount of users 
sharing that link (ie a 100GE core link should have different AQM from a 
10 meg user access line). I believe recommendations in this area are 
needed.

In section 4 this is discussed, and for LEDBAT to work, there needs to be 
some buffering before TCP starts to consider the link congested, but 
LEDBAT would notice it before. How is this signalled, is LEDBAT just 
simply more sensitive to delay variations and back off its rate a lot more 
aggressively?

Coming back to "instantaneous", I'd still like this defined a bit more, I 
guess we're talking second scale resolution here, or perhaps minutes?

Apart from that I think I read -01 revision, because most of the things I 
responded to earlier didn't exist in -02.

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 14 Jul 2011 10:56:03 +0100

Mikael,

At 06:54 13/07/2011, Mikael Abrahamsson wrote:
>I opened a lot of documents and tabs yesterday to read up on all 
>published documents in CONEX, I probably indeed have messed up and 
>had multiple revisions.
>
>So, now re-reading 
><http://www.ietf.org/id/draft-ietf-conex-concepts-uses-02.txt> which 
>I hope is correct.
>
>So, congestion volume description starts to make sense, for me as a 
>router guy this now is "any packet that couldn't be sent immediately 
>(or within a very low buffer threshold value) or which was over the 
>rate a policer was configured to. This, however, is NOT ECN marking ,

Why have you suddenly decided this has to be anything over a v low 
threshold and therefore doesn't fit with ECN?

>  so basically to interoperate with non-ECN traffic, there needs to 
> be a new flag (or one needs to start marking/dropping (if it's ECN 
> enabled packet or not) very early in the queue depth). Just to make 
> an example, I would like to have end systems consider a link 
> congested as soon as a packet was buffered more than 1ms, that 
> doesn't mean I want to start dropping non-ECN packet at that level, 
> so what do we do?

There's no need to change anything at all in queues to do ConEx. I'm 
not sure where all this about changing AQM has suddenly come from?

>With the current scheme, if I have a WRED profile that starts 
>marking/dropping at 30ms queue depth, I'd consider this quite 
>heavily congested before the current scheme would do so. So how 
>should operators configure their AQM to work best in a conex world?

The dependency is the other way round. ConEx just reflects whatever 
congestion the AQM signals. If the AQM is good without ConEx, it's 
good with ConEx. If an operator wants shallow queues in their 
buffers, they should do that, irrespective of ConEx (or not ConEx).

>I also imagine it should be different depending on link speed and 
>amount of users sharing that link (ie a 100GE core link should have 
>different AQM from a 10 meg user access line). I believe 
>recommendations in this area are needed.

I'll start another thread, as these are interesting areas for 
discussion, but they're off the ConEx charter. Might not get to it 
today tho (another deadline this pm).

>In section 4 this is discussed, and for LEDBAT to work, there needs 
>to be some buffering before TCP starts to consider the link 
>congested, but LEDBAT would notice it before. How is this signalled, 
>is LEDBAT just simply more sensitive to delay variations and back 
>off its rate a lot more aggressively?

Yup.

>Coming back to "instantaneous", I'd still like this defined a bit 
>more, I guess we're talking second scale resolution here, or perhaps minutes?

Congestion-rate is only mentioned because it can be used as a 
concept, e.g. as the drain-rate of a congestion-policer. Here a token 
bucket would 'measure' the congestion-rate in the sense that it would 
drain at that rate. But a congestion policer never reports the 
congestion-rate to any management system (e.g. as a sequence of 
varying numbers). Similarly, bit-rate is used as a concept all over 
networking. However, usually bit-rate is used within mechanisms, not 
reported as a number in its own right.

So the important part of the definition is how it is measured. The 
conceptually part is optional - it was an attempt to explain it 
mathematically.

It really does mean instantaneouly in the sense of a short period 
tending to zero. Definitely not minutes. Certainly per-packet 
congestion rate really is either 1 or 0. Then over a duration you see 
a varying probability of getting marks/drops, which determines the 
congestion-volume over that duration. If you take congestion-volume 
over a very short period and divide it my the duration of the period, 
that's the (nearly) instantaneous congestion-rate.

Instantaneous bit-rate has similar properties: strictly the bit-rate 
is either line-rate when transmitting a packet or 0 when not. But 
over a short period, you can measure a bit-rate, and people happily 
talk about the instantaneous bit-rate, without worrying that it is 
strictly either line-rate or zero.

>Apart from that I think I read -01 revision, because most of the 
>things I responded to earlier didn't exist in -02.

Understood. -02 is significantly different in sections 1 & 2 particularly.

Cheers

Bob

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 14 Jul 2011 12:13:55 +0200 (CEST)

> threshold and therefore doesn't fit with ECN?

I believe the following to be true:

Congestion is a fact when there is any buffering or drops.
Drop and ECN marking should be equivalent.
I don't want to drop packets for non-ECN traffic at very low queue depth.
I want to give ECN and non-ECN traffic equal treatment when it comes to 
dropping (which in ECN is setting the "would have dropped bit").

So in short terms, I want to signal congestion before I drop non-ECN 
packets with WRED.

> There's no need to change anything at all in queues to do ConEx. I'm not 
> sure where all this about changing AQM has suddenly come from?

I feel there is a need to signal congestion at low queue depths, but not 
to drop the traffic.

> tending to zero. Definitely not minutes. Certainly per-packet congestion 
> rate really is either 1 or 0. Then over a duration you see a varying 
> probability of getting marks/drops, which determines the 
> congestion-volume over that duration. If you take congestion-volume over 
> a very short period and divide it my the duration of the period, that's 
> the (nearly) instantaneous congestion-rate.

Oki, for me congestion is basically that there are more than a few packets 
in the queue a significant amount (let's say 50% to give an example) of a 
second.

In an ideal world, I would like to set a "congestion bit" as soon as the 
buffer depth reaches more than 1ms worth of packets, but I do not want to 
drop non-ECN traffic at this depth.

But if we now drop my definition of congestion and go back to what I 
interpret your position to be, then for example, if I have WRED configured 
to start dropping packets (or set ECN drop bit) at 30ms queue depth on my 
core links, then that by definition would be congestion signalling. Is the 
+30ms delay also a congestion signal, and how do the end systems know that 
this +30ms buffer delay is true? Does it look at the packet delay variance 
and discern that buffering is likely to be in effect? Does LEDBAT back off 
when there is PDV? Considering wifi or half duplex links usually induce 
PDV, I guess this isn't easy either?

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 14 Jul 2011 15:55:02 +0100

Hi Mikael,

it surely would have been nice, if there would have been mechanisms to
signal better-than-binary congestion levels. There are numerous papers
describing what benefits a more granular signaling of congestion (ie.
no/light/medium/heavy congestion) could have for transports. However,
all these protocols would require all the routers and transport
protocols to be changed - thus it won't happen in the internet any time
soon...

Conex has to do with what is available as congestion signals today (ie.
ECN and/or loss; PDV sensitive transports like LEDBAT, Chirp, Compound,
(Vegas) could also use that as input signals into Conex though).

In order to incentivize ECN over regular TCP, you need to start marking
instead of dropping *at the same levels*. Otherwise, the status quo (no
ECN deployment in the core routers) will persist indefinitely. 

Anyway this discussion appears to be orthogonal to what Conex is all
about.

Basically, however a transport sender obtained the congestion signal (be
it loss, marks or increasing one-way-delay), Conex is providing this
information to the upstream (from the bottleneck point) part of the
network with that information.

One discussion not entirely off charter may be to provide means in
Conex, to signal different levels of congestion though, at a finer
granularity than possible as currently envisioned (all bits in a
conex-marked packet are accounted 100% as congestion-volume; if a small
packet carried the ECN mark (loss), but a large packet carries the conex
mark, you are introducting small errors in the congestion volume.
However, I believe these are 2nd order problems...

Richard Scheffenegger

> -----Original Message-----
> From: Mikael Abrahamsson [mailto:[email-masked]]
> Sent: Donnerstag, 14. Juli 2011 12:14
> To: Bob Briscoe
> Cc: [email-masked]
> Subject: Re: [conex] I-D Action: draft-ietf-conex-concepts-uses-02.txt
> 
> not
> congestion
> over
> in the queue a significant amount (let's say 50% to give an example)
of
> a

---

From: John Leslie <[email-masked]>
Date: Thu, 14 Jul 2011 21:31:43 -0400

> Congestion is a fact when there is any buffering or drops.

   I agree that a definition of congestion which takes note of buffering
could be useful; I don't believe it's within our charter to pursue that
at the present (though it could posssibly be future work).

> Drop and ECN marking should be equivalent.

   That is the definition we must work under, though in principle it
would be nice to have _some_ kind of early-warning signal which comes
before any packets may be dropped.

> dropping (which in ECN is setting the "would have dropped bit").

   I'm not clear how you think you can satisfy both of these...

> So in short terms, I want to signal congestion before I drop non-ECN 

   A commendable desire! But I think such work is out-of-charter.

> ... for me congestion is basically that there are more than a few packets 

   I'm sympathetic to that definition.

> drop non-ECN traffic at this depth.

   That sounds like ICCRG work.

> interpret your position to be, then for example, if I have WRED
> configured to start dropping packets (or set ECN drop bit) at 30ms
> queue depth on my core links, then that by definition would be
> congestion signalling.

   Yes. ConEx cannot enforce any particular AQM rules.

> Is the +30ms delay also a congestion signal,

   Not to ConEx -- though receivers are free to deduce delay (or even
use some delay-based signal which may be standardized in the future)
and pass that information back to the sender, we do not expect the
sender to inject that as a ConEx Congestion-Expected signal.

   Nor, IMHO, would such a signal by the sender be useful for our
purposes. To us, anything like LEDBAT that backs off due to delay-based
signals isn't contributing to "congestion" in the ConEx sense.

> and how do the end systems know that this +30ms buffer delay is true?

   That's totally out-of-scope for ConEx.

--
John Leslie <[email-masked]>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 15 Jul 2011 09:00:59 +0200 (CEST)

>   That's totally out-of-scope for ConEx.

Wouldn't it be good if ConEx could give guidance to protocol designers how 
the network behaves and what constitutes a congestion signal in networks? 
Also to give BCP guidance to network operators and vendors how they should 
configure the network and how it should behave to work the best way. Just 
see the bufferbloat discussions which I thought was no news at all, but 
seems to have been news to some people. There is definitely need of 
guidance how AQM should be done.

Or is this purely ICCRG?