Subject: [conex] New draft(s) on TCP modifications for ConEx

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 6 Jul 2011 10:22:35 +0200

Hello,

we submitted two new drafts regarding the needed TCP modifications for ConEx. 
As ConEx relies (partially) on ECN, a more accurate ECN feedback (more than 
max. one signal per RTT) is needed. The same information would be valuebale 
for other currently proposed TCP mechanisms e.g. DCTCP. This is why, we 
decided to split this modification into a separate and tried to specify this 
independent of ConEx. Thus we have two drafts now:

1.  draft-kuehlewind-conex-accurate-ecn
This draft currently proposes three different coding scheme to realize a more 
accurate ECN feedback. All approaches use the available 'classic' ECN bit 
space as only one mechanism (classic ECN or the new more accurate scheme) is 
needed at a time. The goal is to chose one of the coding options and have the 
other option and all discussions removed at later version of this draft.

2. draft-kuehlewind-conex-tcp-modifications
This draft describes all other modifications/recommendations to use ConEx with 
TCP. This is the use of ECN and SACK and recommendations on the credit bit. 
This document contains several points which need further discussion e.g. the 
handling and validity of credits. 

Please have a look at the drafts; Feedback is very welcome!

Mirja and Richard

Subject: New Version Notification for draft-kuehlewind-conex-accurate-ecn-00.txt
From: [email-masked]
To: [email-masked]
Cc: [email-masked], [email-masked]
Date: Mon, 04 Jul 2011 13:13:53 -0700

A new version of I-D, draft-kuehlewind-conex-accurate-ecn-00.txt has been successfully submitted by Mirja Kuehlewind and posted to the IETF repository.

Filename:	 draft-kuehlewind-conex-accurate-ecn
Revision:	 00
Title:		 Accurate ECN Feedback in TCP
Creation date:	 2011-07-04
WG ID:		 Individual Submission
Number of pages: 23

Abstract:
   Explicit Congestion Notification (ECN) is an IP/TCP mechanism where
   network nodes can mark IP packets instead of dropping them to
   indicate congestion to the end-points.  An ECN-capable receiver will
   feedback this information to the sender.  ECN is specified for TCP in
   such a way that only one feedback signal can be transmitted per
   Round-Trip Time (RTT).  Recently new TCP mechanisms like ConEx or
   DCTCP need more accurate feedback information in the case where more
   than one marking is received in one RTT.

The IETF Secretariat

Subject: New Version Notification for draft-kuehlewind-conex-tcp-modifications-00.txt
From: [email-masked]
To: [email-masked]
Cc: [email-masked], [email-masked]
Date: Mon, 04 Jul 2011 13:24:25 -0700

A new version of I-D, draft-kuehlewind-conex-tcp-modifications-00.txt has been successfully submitted by Mirja Kuehlewind and posted to the IETF repository.

Filename:	 draft-kuehlewind-conex-tcp-modifications
Revision:	 00
Title:		 TCP modifications for Congestion Exposure
Creation date:	 2011-07-04
WG ID:		 Individual Submission
Number of pages: 13

Abstract:
   Congestion Exposure (ConEx) is a mechanism by which senders inform
   the network about the congestion encountered by previous packets on
   the same flow.  This document describes the necessary modifications
   to use ConEx with the Transmission Control Protocol (TCP).

The IETF Secretariat

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Wed, 6 Jul 2011 14:26:34 +0200

Hi Mirja

I have (quickly)  read the document draft-kuehlewind-conex-accurate-ecn-00. 
So sorry if I have not understood the concept well!

The described solution in this draft is based on [draft-briscoe-
tsvwg-re-ecn-tcp-09] and is actually used to 
carry the required feedback from the receiver to the sender such that the
sender could calculate 
the experienced congestion rate (from sender to receiver).

In my opinion this solution being similar to the one proposed in
[draft-briscoe- tsvwg-re-ecn-tcp-09]
seems to be quite complex to be deployed, since protocol changes need to be
accomplished in order to modify the 
 semantics of the TCP header, i.e., the semantics of the flags: NS, CWR and
ECE.

  In
http://www.ietf.org/id/draft-karagiannis-conex-congestion-calculation-00.txt
another solution on 
  calculating the congestion rate at the sender is described where these
changes in the semantics of
  TCP are not required.

 This document provides a solution to this problem as follows, see 
 Figure 2. in draft-karagiannis-conex-congestion-calculation-00.txt

 When the sender needs to reduce its sending rate, then the sender can 
 calculate the exposed CONGESTION RATE by subtracting the TCP 
 throughput calculated during a Round Trip Time (RTT), i.e., (RTTi) 
 from the TCP throughput calculated at the same sender side during the 
 previous RTT, i.e., (RTTi-1), where i is an integer equal or higher 
 than 1.

The TCP throughput at the sender side can be calculated using, e.g., 
the following equation specified in [RFC5348]:

   "The throughput equation for X_Bps, TCP's average sending rate in
   bytes per second, is:

                                s
   X_Bps = ----------------------------------------------------------
           R*sqrt(2*b*p/3) + (t_RTO * (3*sqrt(3*b*p/8)*p*(1+32*p^2)))

   Where:

      X_Bps is TCP's average transmit rate in bytes per second.  (X_Bps
      is the same as X_calc in RFC 3448.)

      s is the segment size in bytes (excluding IP and transport
      protocol headers).

      R is the round-trip time in seconds.

      p is the loss event rate, between 0 and 1.0, of the number of loss
      events as a fraction of the number of packets transmitted.

      t_RTO is the TCP retransmission timeout value in seconds.

      b is the maximum number of packets acknowledged by a single TCP
      acknowledgement.", copied from [RFC5368]. 

Note that the transport path throughput calculated at the sender is defined
as: The per flow transport sending rate as a function of the congestion
rate, round-trip time, and segment size. The transport path throughput
calculated at the sender using the TCP throughput equation specified in
[RFC5348]. Note that in [RFC5348] the term congestion rate is denoted as
loss event rate. According to [RFC5348] a loss event is defined as one or
more lost or marked packets from a window of data, where a marked packet
refers to a congestion indication (CE) from Explicit Congestion Notification
(ECN) [RFC3168]; 

Best regards,
Georgios

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Mirja Kühlewind
> Sent: woensdag 6 juli 2011 10:23
> To: [email-masked]
> Subject: [conex] New draft(s) on TCP modifications for ConEx
> 
> Hello,
> 
> max. one signal per RTT) is needed. The same information would be
> valuebale for other currently proposed TCP mechanisms e.g. DCTCP. This is
> why, we decided to split this modification into a separate and tried to
specify
> this independent of ConEx. Thus we have two drafts now:
> 
> space as only one mechanism (classic ECN or the new more accurate scheme)
> is needed at a time. The goal is to chose one of the coding options and
have
> the other option and all discussions removed at later version of this
draft.
> This draft describes all other modifications/recommendations to use ConEx
> with TCP. This is the use of ECN and SACK and recommendations on the
> credit bit.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 6 Jul 2011 15:49:04 +0100

Hi Georgios,

Thanks for the quick reply.

I'm replying as the co-author of this draft.

Your comment is only about one (of the three proposed) mechanisms to signal more accurate ECN feedback. As you have noticed, we introduce three different signaling encodings and semantics, each with increasing feature set (and accuracy) over the previous, but also with increasing complexity at implementation.

Also note, that draft-kuehlewind-conex-accurate-ecn-00, even while submitted in the ConEx WG, really has a broader scope, any deals with the TCP layer alone. Conex is but one possible framework, where this more accurate ECN feedback will be required. Submitting this draft to TCPM was also shortly discussed internally, but ConEx really has a current demand. Nevertheless, the broader scope (and different constraints) of TCP should not get lost when discussing this draft.

Reading your draft, I fail to understand how this may work. Using standard RFC3168 feedback, only a single (!) CE per RTT can be signaled, and then TCP will always react with a single CC response per RTT... regardless of how many CE marks the receiver actually received during the RTT following the initial CE.

Calculating the sending rate at the sender doesn't seem to accomplish what you aim at (or the ingenious idea is missing in that draft). With the widespread adoption of TCP SACK, and alternate congestion controller responses than NewReno (e.g. CUBIC, Compound, ...), the closed formula you quote is only a crude guess at the actual sending rate. These newer TCP congestion control reactions will conform to that formula only under certain (limited) circumstances, at best. (The formula gives a worst case estimate, but the typical sending rate will be higher). Furthermore, the goal of ECN is, to avoid loss (denoted "p" in the formula) due to congestion as much as possible. Because of the limited signal bandwidth (one bit per RTT) of RFC3168, compared to congestion loss (one bit per segment sent - conveyed more or less accurately by ACKs/SACKs), putting standard ECN feedback into "p" will not work. 

Furthermore, the congestion volume, if I followed ConEx correctly, is the volume of bytes (measured as IP-layer bytes, NOT IP payload, or TCP payload bytes) which were received with CE set. With only a single bit of feedback per RTT, the congestion volume therefore has a lower bound of PMTU bytes per RTT (actually, the minimum packet size possible, when 1-byte TCP payload segments are sent for whatever reason), and an upper bound of floor(FlightSize/MSS)+1*(IP_Header+TCP_Header[TCP Options])+FlightSize.

Your formula would, as it seems to me, always estimate the congestion volume to be close to cwnd/2 (excluding the IP Header, TCP Headers, and also excluding the actual number of congestions during that RTT). This may be a (very) conservative estimate during low congestion periods, but a very aggressive (=much too small) estimate during heavy congestion periods (e.g. 1 RTT, where each segment is CE marked).

Again, the goal for Accurate ECN feedback is to allow an exact (under common circumstances) feedback of the CE marks as seen by a TCP receiver, to be sent back to the sender. As TCP can not rely on external means to enforce honesty (Conex would be one such external means), ECN Nonce support would be beneficial.

One more word w.r.t. complexity: I believe that the gap between network performance and CPU performance will not shrink in the future. All developments over the last decade points to an ever widening gap. Therefore, schemes which couldn't be hoped to have any chance to be implemented, because of their complexity, such as Conex, are today running at wirespeed in 10G environments. The times, where one major design goal of a protocol was to use as few as possible machine instructions to encode/decode and interpret, are long past.

I have seen early SACK and TS work was discussed in terms of number of machine instructions added to the TCP codepath (which is one piece of the puzzle to understand why timestamp semantics are defined the way they are). 

(Also, calculating the integer square root is more involved than doing some bit-banging ;)

Richard Scheffenegger

> From: Georgios Karagiannis [mailto:[email-masked]]
> Sent: Mittwoch, 06. Juli 2011 14:27
> To: 'Mirja Kühlewind'; [email-masked]
> Subject: Re: [conex] New draft(s) on TCP modifications for ConEx
> 
> Hi Mirja
> 
> > we submitted two new drafts regarding the needed TCP modifications
> for
> > ConEx.
> > As ConEx relies (partially) on ECN, a more accurate ECN feedback
> (more
> than
> > This draft currently proposes three different coding scheme to
> realize a
> more
> > This document contains several points which need further discussion
> e.g.
> the
> _______________________________________________
> conex mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/conex

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Wed, 06 Jul 2011 19:17:25 +0000

Hi Richard,

Please see in line!

>Your comment is only about one (of the three proposed) mechanisms to signal more accurate ECN feedback. As you have noticed, we introduce three different signaling encodings and semantics, each with…

Georgios: Yes, sorry for not mentioning them, I was referring to the last
two solutions.

>Also note, that draft-kuehlewind-conex-accurate-ecn-00, even while submitted in the ConEx WG, really has a broader scope, any deals with the TCP layer alone. Conex is but one possible framework, whe…

Georgios: Okay, but my point regarding the fact that the the two latter
proposed solutins biy your draft are complex to be deployed, is still
valid, since protocol changes need to be accomplished in order to modify
the semantics of the TCP header, i.e., the semantics of the flags: NS,
CWR and ECE.

>Reading your draft, I fail to understand how this may work. Using standard RFC3168 feedback, only a single (!) CE per RTT can be signaled, and then TCP will always react with a single CC response pe…

Georgios: Note that the solution uses the implementation specified in
RFC5348. According to this RFC5348,

"The receiver periodically sends feedback messages to the sender.
   Feedback packets SHOULD normally be sent at least once per RTT,
   unless the sender is sending at a rate of less than one packet per
   RTT, in which case a feedback packet SHOULD be sent for every data
   packet received.  A feedback packet SHOULD also be sent whenever a
   new loss event is detected without waiting for the end of an RTT, and
   whenever an out-of-order data packet is received that removes a loss
   event from the history.

   If the sender is transmitting at a high rate (many packets per RTT),
   there may be some advantages to sending periodic feedback messages
   more than once per RTT as this allows faster response to changing RTT
   measurements and more resilience to feedback packet loss."

>Calculating the sending rate at the sender doesn't seem to accomplish what you aim at (or the ingenious idea is missing in that draft). With the widespread adoption of TCP SACK, and alternate conges…

Georgios: As I mentioned the RFC5348 (TCP Friendly Rate Control (TFRC))
is used by this solution. So the accuracy of the congestion rate
calculation will depend on the accuracy of the TFRC. Moreover, according
to RFC5348:
 "The throughput equation currently REQUIRED for TFRC is a slightly
   simplified version of the throughput equation for Reno TCP from
   [PFTK98].  Ideally, we would prefer a throughput equation based on
   selective acknowledgment (SACK) TCP, but no one has yet derived the
   throughput equation for SACK TCP, and simulations and experiments
   suggest that the differences between the two equations would be
   relatively minor [FF99] (Appendix B).", from [RFC5348]

Furthermore, note that p is the rate of loss events and is depending on
the loss rate. Loss rate measurement is performed at the receiver, based
on the detection of lost or marked packets from the sequence numbers of
arriving packets.

p can be send from receiver towards the sender in feedback packets.
In TFRC several feedback packets can be sent towards the sender within
one RTT, depending on the sender data rate.

>Your formula would, as it seems to me, always estimate the congestion volume to be close to cwnd/2 (excluding the IP Header, TCP Headers, and also excluding the actual number of congestions during t…

Georgios: As I mentioned the RFC5348 (TCP Friendly Rate Control (TFRC))
is used by this solution. So the accuracy of the congestion rate
calculation will depend on the accuracy of the TFRC.

>Again, the goal for Accurate ECN feedback is to allow an exact (under common circumstances) feedback of the CE marks as seen by a TCP receiver, to be sent back to the sender. As TCP can not rely on…

Georgios: Yes I agree it might be beneficial.

>(Also, calculating the integer square root is more involved than doing some bit-banging ;)

Georgios: I think that you refer to my statement regarding implementation
complexity of the Re-ECN solution. The term that I have used is not the
correct one. I should have used deployment complexity instead of
implmentation complexity.

Best regards,
Georgios

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Wed, 6 Jul 2011 20:55:33 +0100

Hi Georgios,

So, you are proposing something orthogonal to TCP encoding (DCCP transports using CCID3/4)? Now this was the part I have been missing.

Regarding deployment complexity of accurate-ecn: 

We envision, that only one signaling/semantic will ultimately be chosen by the IETF. For that single method, a backwards-compatible handshake is described (section 2); 

If accurate ecn semantics are not negotiated between end hosts, a compatibility mode is also sketched (section 3.4). This approach allows to extract more than one bit per RTT as ECN feedback from regular RFC3168 tcp receivers. While in that state, the sender itself would obviously adhere to normal RFC3168 semantics. However, a sender may chose not to implement the advanced compatibility mode - but such a TCP sender could underestimate congestion volume (especially during heavy congestion episodes) and then be throttled.

Thus, there is a roadmap for incremental deployment of the accurate ecn feedback (for TCP).

Best regards,

Richard Scheffenegger

> Sent: Mittwoch, 06. Juli 2011 21:17
> To: Scheffenegger, Richard; Mirja Kühlewind; [email-masked]
> Subject: RE: [conex] New draft(s) on TCP modifications for ConEx
> 
> Hi Richard,
> 
> Please see in line!
> 
> 
> >Your comment is only about one (of the three proposed) mechanisms to
> signal more accurate ECN feedback. As you have noticed, we introduce
> three different signaling encodings and semantics, each with increasing
> feature set (and accuracy) over the previous, but also with increasing
> complexity at implementation.
> 
> >Also note, that draft-kuehlewind-conex-accurate-ecn-00, even while
> submitted in the ConEx WG, really has a broader scope, any deals with
> the TCP layer alone. Conex is but one possible framework, where this
> more accurate ECN feedback will be required. Submitting this draft to
> TCPM was also shortly discussed internally, but ConEx really has a
> current demand. Nevertheless, the broader scope (and different
> constraints) of TCP should not get lost when discussing this draft.
> 
> >Reading your draft, I fail to understand how this may work. Using
> standard RFC3168 feedback, only a single (!) CE per RTT can be
> signaled, and then TCP will always react with a single CC response per
> RTT... regardless of how many CE marks the receiver actually received
> during the RTT following the initial CE.
> 
> >Calculating the sending rate at the sender doesn't seem to accomplish
> what you aim at (or the ingenious idea is missing in that draft). With
> the widespread adoption of TCP SACK, and alternate congestion
> controller responses than NewReno (e.g. CUBIC, Compound, ...), the
> closed formula you quote is only a crude guess at the actual sending
> rate. These newer TCP congestion control reactions will conform to that
> formula only under certain (limited) circumstances, at best. (The
> formula gives a worst case estimate, but the typical sending rate will
> be higher). Furthermore, the goal of ECN is, to avoid loss (denoted "p"
> in the formula) due to congestion as much as possible. Because of the
> limited signal bandwidth (one bit per RTT) of RFC3168, compared to
> congestion loss (one bit per segment sent - conveyed more or less
> accurately by ACKs/SACKs), putting standard ECN feedback into "p" will
> not work.
> 
> >Furthermore, the congestion volume, if I followed ConEx correctly, is
> the volume of bytes (measured as IP-layer bytes, NOT IP payload, or TCP
> payload bytes) which were received with CE set. With only a single bit
> of feedback per RTT, the congestion volume therefore has a lower bound
> of PMTU bytes per RTT (actually, the minimum packet size possible, when
> 1-byte TCP payload segments are sent for whatever reason), and an upper
> bound of floor(FlightSize/MSS)+1*(IP_Header+TCP_Header[TCP
> Options])+FlightSize.
> >
> >Your formula would, as it seems to me, always estimate the congestion
> volume to be close to cwnd/2 (excluding the IP Header, TCP Headers, and
> also excluding the actual number of congestions during that RTT). This
> may be a (very) conservative estimate during low congestion periods,
> but a very aggressive (=much too small) estimate during heavy
> congestion periods (e.g. 1 RTT, where each segment is CE marked).
> 
> >Again, the goal for Accurate ECN feedback is to allow an exact (under
> common circumstances) feedback of the CE marks as seen by a TCP
> receiver, to be sent back to the sender. As TCP can not rely on
> external means to enforce honesty (Conex would be one such external
> means), ECN Nonce support would be beneficial.
> 
> >One more word w.r.t. complexity: I believe that the gap between
> network performance and CPU performance will not shrink in the future.
> All developments over the last decade points to an ever widening gap.
> Therefore, schemes which couldn't be hoped to have any chance to be
> implemented, because of their complexity, such as Conex, are today
> running at wirespeed in 10G environments. The times, where one major
> design goal of a protocol was to use as few as possible machine
> instructions to encode/decode and interpret, are long past.
> >
> >I have seen early SACK and TS work was discussed in terms of number of
> machine instructions added to the TCP codepath (which is one piece of
> the puzzle to understand why timestamp semantics are defined the way
> they are).
> >
> >> carry the required feedback from the receiver to the sender such
> that
> >> the
> >> sender could calculate
> >> seems to be quite complex to be deployed, since protocol changes
> need
> >> to be
> >>  semantics of the TCP header, i.e., the semantics of the flags: NS,
> CWR
> >> and
> >> ECE.
> >>
> >>   In
> >> http://www.ietf.org/id/draft-karagiannis-conex-congestion-
> calculation-
> >> 00.txt
> >> another solution on
> >> [RFC5348]. Note that in [RFC5348] the term congestion rate is
> denoted
> >> as
> >> loss event rate. According to [RFC5348] a loss event is defined as
> one
> >> or
> >> > why, we decided to split this modification into a separate and
> tried
> >> to
> >> specify
> >> > accurate ECN feedback. All approaches use the available 'classic'
> ECN
> >> bit
> >> > is needed at a time. The goal is to chose one of the coding
> options
> >> and
> >> have
> >> > This draft describes all other modifications/recommendations to
> use
> >> ConEx

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 06 Jul 2011 21:25:46 +0100

Georgios,

The dependency goes like this:

1. Actual-Congestion-rate
2.  -> Receiver's-Measured-Congestion-rate
3.    -> Feedback-to-sender
4.      -> Sender's-Measured-Congestion-rate
5.        -> TCP-algo
6.          -> TCP-bit-rate

Information is lost between stages 2-4 because of 
the way TCP feedback works in the case of ECN. So 
(1) = (2) <= (3) <= (4), where <= means less than or equal to.

The TCP equation is derived by setting the 
Sender's-Measured-Congestion-rate (stage 4) to p, 
running p through a mathematical model of the 
TCP-algo (stage 5) to derive the TCP-bit-rate (stage 6).

If the sender derives p from its TCP-bit-rate, by 
reversing the theoretical equation, it will get 
metric (4). Unsurprisingly, it will get the 
Congestion-rate that it has been responding to with the TCP algo.

It won't get back any further to metrics 1-3. So 
the approach in your I-D achieves precisely nothing.

Another point: You're confusing complexity with difficulty.

It might be difficult to change the semantics of 
fields in the TCP header (because we have to 
persuade all sorts of strange people at the IETF 
;) But the algorithm to run is not complex - 
receiver adds to a counter, sends the counter in 
a field, sender reads counter, does a subtract 
and shift or two and we're done. There are two 
implementations, and there are a lot more complicated things in TCP than this.

The really difficult bit of standardisation is 
largely done: we are chartered to produce an 
experimental standard. Importantly, we have one 
good way to negotiate that the new semantics are 
being used, which is essential for backwards 
compatibility. I think Richard & Mirja are right 
to suggest alternatives for the last stage - 
actually doing the feedback. Because reliability 
is quite tough for this - it's OK most of the 
time, but we don't know how prevalent the corner-cases are.

Finally, if you want something complex (and 
difficult), try implementing an algorithm to 
derive p from X in the rate equation in your I-D.

I think you're I-D just shows you've allowed 
yourself to get in a muddle over what depends on what.

Bob

At 20:17 06/07/2011, Georgios Karagiannis wrote:
> >Your comment is only about one (of the three 
> proposed) mechanisms to signal more accurate 
> ECN feedback. As you have noticed, we introduce 
> three different signaling encodings and 
> semantics, each with increasing feature set 
> (and accuracy) over the previous, but also with 
> increasing complexity at implementation.
>
> >Also note, that 
> draft-kuehlewind-conex-accurate-ecn-00, even 
> while submitted in the ConEx WG, really has a 
> broader scope, any deals with the TCP layer 
> alone. Conex is but one possible framework, 
> where this more accurate ECN feedback will be 
> required. Submitting this draft to TCPM was 
> also shortly discussed internally, but ConEx 
> really has a current demand. Nevertheless, the 
> broader scope (and different constraints) of 
> TCP should not get lost when discussing this draft.
>
> >Reading your draft, I fail to understand how 
> this may work. Using standard RFC3168 feedback, 
> only a single (!) CE per RTT can be signaled, 
> and then TCP will always react with a single CC 
> response per RTT... regardless of how many CE 
> marks the receiver actually received during the RTT following the initial CE.
>
> >Calculating the sending rate at the sender 
> doesn't seem to accomplish what you aim at (or 
> the ingenious idea is missing in that draft). 
> With the widespread adoption of TCP SACK, and 
> alternate congestion controller responses than 
> NewReno (e.g. CUBIC, Compound, ...), the closed 
> formula you quote is only a crude guess at the 
> actual sending rate. These newer TCP congestion 
> control reactions will conform to that formula 
> only under certain (limited) circumstances, at 
> best. (The formula gives a worst case estimate, 
> but the typical sending rate will be higher). 
> Furthermore, the goal of ECN is, to avoid loss 
> (denoted "p" in the formula) due to congestion 
> as much as possible. Because of the limited 
> signal bandwidth (one bit per RTT) of RFC3168, 
> compared to congestion loss (one bit per 
> segment sent - conveyed more or less accurately 
> by ACKs/SACKs), putting standard ECN feedback into "p" will not work.
>
> >Furthermore, the congestion volume, if I 
> followed ConEx correctly, is the volume of 
> bytes (measured as IP-layer bytes, NOT IP 
> payload, or TCP payload bytes) which were 
> received with CE set. With only a single bit of 
> feedback per RTT, the congestion volume 
> therefore has a lower bound of PMTU bytes per 
> RTT (actually, the minimum packet size 
> possible, when 1-byte TCP payload segments are 
> sent for whatever reason), and an upper bound 
> of floor(FlightSize/MSS)+1*(IP_Header+TCP_Header[TCP Options])+FlightSize.
> >
> >Your formula would, as it seems to me, always 
> estimate the congestion volume to be close to 
> cwnd/2 (excluding the IP Header, TCP Headers, 
> and also excluding the actual number of 
> congestions during that RTT). This may be a 
> (very) conservative estimate during low 
> congestion periods, but a very aggressive 
> (=much too small) estimate during heavy 
> >Again, the goal for Accurate ECN feedback is 
> to allow an exact (under common circumstances) 
> feedback of the CE marks as seen by a TCP 
> receiver, to be sent back to the sender. As TCP 
> can not rely on external means to enforce 
> honesty (Conex would be one such external 
> >One more word w.r.t. complexity: I believe 
> that the gap between network performance and 
> CPU performance will not shrink in the future. 
> All developments over the last decade points to 
> an ever widening gap. Therefore, schemes which 
> couldn't be hoped to have any chance to be 
> implemented, because of their complexity, such 
> as Conex, are today running at wirespeed in 10G 
> environments. The times, where one major design 
> goal of a protocol was to use as few as 
> possible machine instructions to encode/decode and interpret, are long past.
> >
> >I have seen early SACK and TS work was 
> discussed in terms of number of machine 
> instructions added to the TCP codepath (which 
> is one piece of the puzzle to understand why 
> timestamp semantics are defined the way they are).
> >
> >(Also, calculating the integer square root is 
> more involved than doing some bit-banging ;)
>

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Wed, 06 Jul 2011 21:06:36 +0000

Hi Bob,

The I-D uses the RFC5348 (TCP Friendly Rate Control (TFRC)). So the
accuracy of the congestion rate calculation will depend on the accuracy
of TFRC, see also my previous email.

Best regards,
Georgios

---

From: Michael Welzl <[email-masked]>
Date: Wed, 6 Jul 2011 23:11:02 +0200

On Jul 6, 2011, at 10:25 PM, Bob Briscoe wrote:

<snip>

> Finally, if you want something complex (and difficult), try  
> implementing an algorithm to derive p from X in the rate equation in  
> your I-D.

I second that. Stein and I spent countless hours trying to produce an  
efficient fixed point version of our seemingly straightforward  
extension of that equation, in MulTFRC.
(but soon getting close to the finish line, I think / hope)
It was quite surprising for us to see how hard that can be.

Cheers,
Michael

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 06 Jul 2011 23:51:55 +0100

Georgios,

I think it is best to stop this thread.

It is off-charter. And beyond all reasoning.

Bob

At 22:06 06/07/2011, Georgios Karagiannis wrote:
> >> possible machine instructions to 
> encode/decode and interpret, are long past.
> >> >

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 07 Jul 2011 05:20:13 +0000

Hi Bob,

Do you mean that is off-charter because is using for feedback TFRC
instead of TCP?

Regarding the beyond reasoning statement, it is better to not discuss it
via email, but face to face. Will you be in Quebec during the next IETF
meeting?

Best regards,
Georgios

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 07 Jul 2011 05:24:14 +0000

Hi Michael,

Thanks for the information!

Best regards,
Georgios

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 07 Jul 2011 06:08:26 +0000

Hi Bob,

This was too early for me!

In the below email I meant:

Do you mean that is off-charter because is using for feedback DCCP & TFRC
(RFC5348, RFC5622, RFC4342) instead of TCP?

Best regards,
Georgios

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 07 Jul 2011 10:28:09 +0100

Georgios,

> >via email, but face to face. Will you be in Quebec during the next IETF

I meant that your response was not engaging in the reasoning put to you.

You can go back and respond to the reasoning 
already given. But I don't need to say any more.

My main point was that the whole idea of your 
draft goes round in a loop that achieves nothing. 
You avoided this critical point and jumped to a 
detail (a detail within that loop that already achieves nothing).

Now you are saying you meant something very 
different to what you had written (you now say 
the draft was about DCCP, but the title of your 
draft says it is about TCP and DCCP isn't 
mentioned). Anyway, DCCP takes you off-charter instead (see below).

>(RFC5348, RFC5622, RFC4342) instead of TCP?

Yes.

ConEx is primarily at the IP layer, but some 
transports need optional changes to their 
feedback. The charter says we will focus on TCP first.

You are criticising a draft written to satisfy 
this charter objective. You are saying it is 
difficult to change the semantics of TCP header 
fields (which we are already chartered to do). So 
you propose everyone should have to use DCCP in 
order to use ConEx. That would require every 
application in the world that uses TCP to be 
changed to call DCCP instead, in order to use 
ConEx. And most middleboxes (NATs, firewalls) would have to be changed too.

When you're in a hole, stop digging.

Bob

> >>> >> of 
> floor(FlightSize/MSS)+1*(IP_Header+TCP_Header[TCP Options])+FlightSize.
> >>> >> >
> >>> >> >> I have (quickly)  read the document 
> draft-kuehlewind-conex-accurate-
> >>> >> >> ecn-00.
> >>> >> >> carry the required feedback from the 
> receiver to the sender such that
> >>> >> >> the
> >>> >> >> sender could calculate
> >>> >> >> seems to be quite complex to be 
> deployed, since protocol changes need
> >>> >> >> to be
> >>> >> >>  semantics of the TCP header, i.e., 
> the semantics of the flags: NS, CWR
> >>> >> >> and
> >>> >> >> ECE.
> >>> >> >>
> >>> >> >>   In
> >>> >> >> 
> >>> >> >>    X_Bps = 
> ----------------------------------------------------------
> >>> >> >> [RFC5348]. Note that in [RFC5348] the 
> term congestion rate is denoted
> >>> >> >> as
> >>> >> >> loss event rate. According to 
> [RFC5348] a loss event is defined as one
> >>> >> >> or
> >>> >> >> > why, we decided to split this 
> modification into a separate and tried
> >>> >> >> to
> >>> >> >> specify
> >>> >> >> > accurate ECN feedback. All 
> approaches use the available 'classic' ECN
> >>> >> >> bit
> >>> >> >> > is needed at a time. The goal is to 
> chose one of the coding options
> >>> >> >> and
> >>> >> >> have
> >>> >> >> > the other option and all 
> discussions removed at later version of this
> >>> >> >> draft.
> >>> >> >> >
> >>> >> >> > This draft describes all other 
> modifications/recommendations to use
> >>> >> >> ConEx

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 7 Jul 2011 12:50:31 +0200

Hi Bob,

Please see in line!

> From: Bob Briscoe [mailto:[email-masked]]
> Sent: donderdag 7 juli 2011 11:28
> To: Georgios Karagiannis
> Cc: [email-masked]
> > >Regarding the beyond reasoning statement, it is better to not discuss
> > >it via email, but face to face. Will you be in Quebec during the next
> > >IETF meeting?
> 
> You can go back and respond to the reasoning already given. But I don't
need
> to say any more.
> 
> My main point was that the whole idea of your draft goes round in a loop
> that achieves nothing.
> You avoided this critical point and jumped to a detail (a detail within
that loop
> that already achieves nothing).

Georgios: I am not sure if this loop  applies to what is written in my
draft.

> 
> Now you are saying you meant something very different to what you had
> written (you now say the draft was about DCCP, but the title of your draft
> says it is about TCP and DCCP isn't mentioned). Anyway, DCCP takes you
off-
> charter instead  (see below).

In the drafts I had not yet specified which transport protocol should be
used, what I have specified was that the congestion exposure 
can be calculated using the TCP throughput formula used in TFRC (RFC5348).
And according to [RFC5348] "TFRC is a congestion control mechanism designed
for unicast flows operating in
   an Internet environment and competing with TCP traffic [FHPW00].
   Instead of specifying a complete protocol, this document simply
   specifies a congestion control mechanism that could be used in a
   transport protocol such as DCCP (Datagram Congestion Control
   Protocol) [RFC4340], in an application incorporating end-to-end
   congestion control at the application level, or in the context of
   endpoint congestion management [BRS99].", from [RFC5348].

So I do not think that I was saying something in the previous email that is
different than what is stated in my draft.

> >Do you mean that is off-charter because is using for feedback DCCP &
> >TFRC (RFC5348, RFC5622, RFC4342) instead of TCP?
> 
> Yes.
> 
> ConEx is primarily at the IP layer, but some transports need optional
changes
> to their feedback. The charter says we will focus on TCP first.

Georgios: In the charter the following is mentioned:
"Today, the network may signal
congestion by ECN markings or by dropping packets, and the receiver
passes this information back to the sender in transport-layer
acknowledgements. The mechanism to be developed by the CONEX WG
will enable the sender to also relay the congestion information
back into the network in-band at the IP layer, such that the total
level of congestion is visible to all IP devices along the path,
from where it could, for example, be provided as input to traffic
management." , from [Conex charter]

Further down the charter mentions:
 
"Primary work items are:

* An Informational document containing an abstract description of
the congestion exposure mechanism that is independent of specific
transport protocols and congestion information encoding techniques
needed for different IP protocol versions.

* An Experimental specification of an IPv6 packet structure that
encapsulates CONEX information, defining a packet format and an
interpretation.

* An Experimental specification of a modification to TCP, for the
timely transport of congestion information from the destination to
the sender.", from [Conex charter]

By reading the first paragraph from above, I can  understand that Conex WG
will use 
transport-layer acknowledgement to pass the congestion information
(signalled by 
ECN markings or by dropping packets to the receiver) from the receiver to
the sender. 
Thus any types of transport protocols can be used for this purpose (e.g.,
TCP, DCCP).

By reading the third bullet from above,  I can understand that the CONEX WG
will primarily focus on TCP.

However, to my understanding this does not imply that other types of
transport protocols, e.g. .,  DCCP,
are off-charter.  This is in my opinion not explicitly  mentioned in the
charter.

Could the WG chairs comment on this issues, see below:

Conex needs transport protocols to carry congestion information from a
receiver to a sender. 
Are transport protocols such as DCCP out of the Conex charter scope?

> 
> You are criticising a draft written to satisfy this charter objective. You
are
> saying it is difficult to change the semantics of TCP header fields (which
we
> are already chartered to do). So you propose everyone should have to use
> DCCP in order to use ConEx. That would require every application in the
> world that uses TCP to be changed to call DCCP instead, in order to use

Georgios: Sorry Bob, but this is your interpretation. 

Another interpretation could be:
It is good to provide the means such that Conex is used in combination with
applications that are using not only TCP as a transport protocol, 
but also the ones that are using DCCP as transport protocol. 
Furthermore, start with a solution to calculate congestion,  in this case
the TFRC [RF5348], that is simple and does not imply many standardization 
changes on the transport protocols using this solution. 
Moreover, note that I am in favour that a TCP based transport solution
should be specified in parallel.

Best regards,
Georgios

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 7 Jul 2011 12:56:15 +0100

Hi Georgios,

If you choose to use formulas to estimate the transport rate, that formula obviously has to apply to the transport protocol used. TFRC is applicable to DCCP CCID3/4, and perhaps to legacy TCP reno/newreno. However, all bets are off for *ANY* other transport (CBR VoIP over UDP / RTP).

Furthermore, each of these transports has to have some form of feedback mechanism from the receiver to the sender, so that the sender can actually react to congestion. Thus, your scheme is implicitly completely reliant on whatever feedback is available from the transport.

RTP, for example, will have very extensive feedback (much more extensive than what TCP, SCTP, DCCP can do) - see  http://tools.ietf.org/id/draft-ietf-avtcore-ecn-for-rtp-00.txt

What Bob tried to point out is exactly that - your method applies a specific, post-hoc method. This method implicitly relies on transport feedback already, but then simply appears to stuff all transports into a TCP-friendly corset, which may not applicable at all.

The goal of our drafts (accurate-ecn and conex-tcp-mods) is to address the critical part, where the transport receiver gives the *appropriate* (not some estimated) congestion information back to the transport sender. With TCP, we are unfortunately very constrained due to the very widespread use (and also because TCP processing is poured into hardware these days), which severly limits what can be done in a backwards-compatible way.

Best regards,

Richard Scheffenegger

NetApp
[email-masked]
+43 1 3676811 3146 Office (2143 3146 - internal)
+43 676 654 3146 Mobile
www.netapp.com  

EURO PLAZA
Gebäude G, Stiege 7, 3.OG
Am Euro Platz 2
A-1120 Wien

> Sent: Donnerstag, 07. Juli 2011 12:51
> To: 'Bob Briscoe'
> Cc: [email-masked]
> > ConEx. And most middleboxes (NATs, firewalls) would have to be
> changed
> > too.
>

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 7 Jul 2011 14:09:59 +0200

Hi Richard,

You are right! The solution described in my draft only applies to transport
protocols that are using TFRC .

So my proposal is:
Is it possible to combine our solutions, such that TCP based applications,
but also DCCP based applications can use Conex?

Best regards,
Georgios

> From: Scheffenegger, Richard [mailto:[email-masked]]
> Sent: donderdag 7 juli 2011 13:56
> To: Georgios Karagiannis; Bob Briscoe
> Cc: [email-masked]
> If you choose to use formulas to estimate the transport rate, that formula
> obviously has to apply to the transport protocol used. TFRC is applicable
to
> DCCP CCID3/4, and perhaps to legacy TCP reno/newreno. However, all bets
> are off for *ANY* other transport (CBR VoIP over UDP / RTP).
> 
> Furthermore, each of these transports has to have some form of feedback
> mechanism from the receiver to the sender, so that the sender can actually
> react to congestion. Thus, your scheme is implicitly completely reliant on
> whatever feedback is available from the transport.
> 
> RTP, for example, will have very extensive feedback (much more extensive
> than what TCP, SCTP, DCCP can do) - see
http://tools.ietf.org/id/draft-ietf-
> avtcore-ecn-for-rtp-00.txt
> 
> 
> What Bob tried to point out is exactly that - your method applies a
specific,
> post-hoc method. This method implicitly relies on transport feedback
> already, but then simply appears to stuff all transports into a
TCP-friendly
> corset, which may not applicable at all.
> 
> 
> The goal of our drafts (accurate-ecn and conex-tcp-mods) is to address the
> critical part, where the transport receiver gives the *appropriate* (not
some
> estimated) congestion information back to the transport sender. With TCP,
> we are unfortunately very constrained due to the very widespread use (and
> also because TCP processing is poured into hardware these days), which
> severly limits what can be done in a backwards-compatible way.
> 
> Best regards,
> 
> Richard Scheffenegger
> 
> NetApp
> [email-masked]
> > > Now you are saying you meant something very different to what you
> > > had written (you now say the draft was about DCCP, but the title of
> > > your
> > draft
> > In the drafts I had not yet specified which transport protocol should
> > be used, what I have specified was that the congestion exposure can be
> > calculated using the TCP throughput formula used in TFRC (RFC5348).
> > And according to [RFC5348] "TFRC is a congestion control mechanism
> > designed for unicast flows operating in
> > So I do not think that I was saying something in the previous email
> > that is different than what is stated in my draft.
> >
> >
> > >
> > > >Do you mean that is off-charter because is using for feedback DCCP
> > > >& TFRC (RFC5348, RFC5622, RFC4342) instead of TCP?
> > >
> > > Yes.
> > >
> will
> > enable the sender to also relay the congestion information back into
> > the network in-band at the IP layer, such that the total level of
> > congestion is visible to all IP devices along the path, from where it
> > could, for example, be provided as input to traffic management." ,
> > from [Conex charter]
> >
> >
> > * An Informational document containing an abstract description of the
> > congestion exposure mechanism that is independent of specific
> > timely transport of congestion information from the destination to the
> > sender.", from [Conex charter]
> >
> >
> > By reading the first paragraph from above, I can  understand that
> > Conex WG will use transport-layer acknowledgement to pass the
> > congestion information (signalled by ECN markings or by dropping
> > packets to the receiver) from the receiver to the sender.
> > Thus any types of transport protocols can be used for this purpose
> > (e.g., TCP, DCCP).
> >
> >
> > By reading the third bullet from above,  I can understand that the
> > CONEX WG will primarily focus on TCP.
> >
> > transport protocols, e.g. .,  DCCP, are off-charter.  This is in my
> > opinion not explicitly  mentioned in the charter.
> >
> > It is good to provide the means such that Conex is used in combination
> > with applications that are using not only TCP as a transport protocol,
> > Furthermore, start with a solution to calculate congestion,  in this
> > case the TFRC [RF5348], that is simple and does not imply many
> > standardization changes on the transport protocols using this
> > solution.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 7 Jul 2011 13:36:27 +0100

Hi Georgios,

DCCP already features better-than-TCP ECN feedback (see section 11.4 of RFC4340).

Each Ack Vector option clearly indicated the exact stream of CE-marks in the received packets. (And two distinct ACK vector options provide the ECN Nonce feedback).

Using that information, you can figure out the *exact* congestion volume of a DCCP stream, without reverting to any post-hoc heuristic.

But I believe Bob has already pointed out, that TCP is the only transport which is in scope with conex WG?

Richard Scheffenegger

> Sent: Donnerstag, 07. Juli 2011 14:10
> To: Scheffenegger, Richard; 'Bob Briscoe'
> Cc: [email-masked]
> > > > > >it via email, but face to face. Will you be in Quebec during
> the
> > > next
> > > > > >IETF meeting?
> > > >
> > > > I meant that your response was not engaging in the reasoning put
> to
> > > you.
> > > >
> > > > My main point was that the whole idea of your draft goes round in
> a
> > > loop
> > > > that achieves nothing.
> > > > says it is about TCP and DCCP isn't mentioned). Anyway, DCCP
> takes
> > > you
> > > off-
> > > > saying it is difficult to change the semantics of TCP header
> fields
> > > (which
> > > we
> > > > are already chartered to do). So you propose everyone should have
> to
> > > use
> > > > DCCP in order to use ConEx. That would require every application
> in
> > > the
> > > > world that uses TCP to be changed to call DCCP instead, in order
> to
> > > use

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 7 Jul 2011 14:49:26 +0200

Hi Richard,

Best regards,
Georgios

> Sent: donderdag 7 juli 2011 14:36
> Each Ack Vector option clearly indicated the exact stream of CE-marks in
the
> received packets. (And two distinct ACK vector options provide the ECN
> Nonce feedback).
> 
> Using that information, you can figure out the *exact* congestion volume
of
> a DCCP stream, without reverting to any post-hoc heuristic.
> 
> 
> > You are right! The solution described in my draft only applies to
> > transport protocols that are using TFRC .
> >
> > So my proposal is:
> > Is it possible to combine our solutions, such that TCP based
> > applications, but also DCCP based applications can use Conex?
> >
> > Best regards,
> > Georgios
> >
> >
> >
> > > > > You can go back and respond to the reasoning already given. But
> > > > > I
> > > > don't
> > > > need
> > > > > to say any more.
> > > > >
> > > > Georgios: I am not sure if this loop  applies to what is written
> > > > in
> > my
> > > > draft.
> > > >
> > > > >
> > > > > Now you are saying you meant something very different to what
> > > > > you had written (you now say the draft was about DCCP, but the
> > > > > title
> > of
> > > > > your
> > > > draft
> > > > be used, what I have specified was that the congestion exposure
> > > > can
> > be
> > > > So I do not think that I was saying something in the previous
> > > > email that is different than what is stated in my draft.
> > > >
> > > >
> > > > >
> > > > congestion by ECN markings or by dropping packets, and the
> > > > receiver passes this information back to the sender in
> > > > transport-layer acknowledgements. The mechanism to be developed
> by
> > > > the CONEX WG
> > > will
> > > > transport protocols, e.g. .,  DCCP, are off-charter.  This is in
> > > > my opinion not explicitly  mentioned in the charter.
> > > >

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 07 Jul 2011 17:32:50 +0100

Georgios,

I don't think it's fair to describe your draft as a solution.
Whatever the protocol. Whatever the congestion 
control algorithm. It achieves precisely nothing, except warming the processor.

Your draft merely suggests a sender can calculate information it already has...

Existing situation for any transport sender:
- Congestion feedback information comes in to the 
sender at rate pX (the loss event rate).
- The sender uses this to drive the algorithm 
that determines how it changes its rate X

Your proposal
- The sender measures the change in the rate X 
every RTT and from this works out what p would 
have caused this, using the equation for its own algorithm in reverse.
- from p it can work out pX, the loss event rate

Results:
- the sender has worked out pX, which it already knew accurately at the start.
- the CPU has got warmer.

Therefore, as I said, this achieves precisely 
nothing except increasing the amount of entropy in the Universe.

Bob

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Thu, 07 Jul 2011 19:56:09 +0000

Hi Bob,

The goal of the solution provided in my I-D is that TFRC is used to
calculate the congestion exposure rate in a much simpler manner than the
one proposed by Re-ECN. The calculated congestion rate will be then
exposed into the IP network in a similar way as described in the
"conex-abstract-meth".

Richard already mentioned that by using TFRC the congestion exposure rate
can be calculated much easier than I thought. Thus instead of
calculating the congestion exposure rate from the TCP throughput, one
can just calculate that from the received pX directly.

So my congestion exposure solution is much much simpler than I thought.
Thus Richard thanks for this hint!

These changes will be included in the next version of the draft.

Best regards,
Georgios

On 7/7/2011, "Bob Briscoe" <[email-masked]> wrote:

>Your draft merely suggests a sender can calculate information it already has..
>
>