Subject: [conex] DDoS and ConEx-awareness in the forwarding plane

From: Bob Briscoe <[email-masked]>
Date: Thu, 14 Apr 2011 17:22:30 +0100

Suresh,

Something is bugging me about using a destination option as the ConEx 
v6 encoding. I'm worried about how many cycles it would take for 
general forwarding machinery (if suitably modified) to find the ConEx 
destination option.

Let me explain why I'm asking... Recently, I have emphasised (for 
Mike Abrahamsson) that every queue is NOT intended to police traffic 
using ConEx info. However,... at the risk of confusing matters, one 
day ConEx info could OPTIONALLY be used at any queue - at least at 
any L3 queue. Not for policing, but for preferential discard. This 
should quite strongly insulate responsive traffic from unresponsive 
flooding traffic (see background below).

I don't imagine ConEx awareness would be deployed in forwarding 
hardware until some time after ConEx had been deployed for more 
general traffic management purposes at the 'edge' of a network. But 
we ought not to forget the possibility of this when choosing the encoding.

Bob

Background
----------
Within a Diffserv class, discard order would be:
1. discard Non-ConEx traffic first*
2. then discard ConEx Unmarked
3. and lastly discard Re-Echo and Credit marked ConEx traffic.

(* = leaving a min rate to prevent starvation of genuine non-ConEx sources)

Reasoning: if DDoS was causing heavy congestion there would be lots 
of loss (or ECN marking). So well-behaved responsive sources would 
slow down and send most packets with a ConEx Re-Echo marking.

An attack source by definition would not reduce its rate in response 
to the congestion. For ConEx marking, it could adopt three strategies 
(or a hybrid):
a) not use ConEx (then the queue drops it by rule 1)
b) use ConEx, but not Re-Echo congestion (then the queue drops it by rule 2)
c) use ConEx properly Re-Echoing congestion (then the congestion 
policer quickly exhausts its allowance and the attack source gets 
strongly throttled).

This Preferential Drop behaviour was documented in the re-ECN spec, 
although we never implemented a queue to evaluate it:
<http://tools.ietf.org/html/draft-briscoe-tsvwg-re-ecn-tcp-09#section-5.3>

Bob

[PS. For the avoidance of doubt, this email is not about whether we 
should write up a DDoS use-case. It is about whether our choice of 
encoding would prevent us being able to use CoNEx against DDoS in future.]

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Suresh Krishnan <[email-masked]>
Date: Fri, 15 Apr 2011 18:59:50 -0400

Hi Bob,

> general forwarding machinery (if suitably modified) to find the ConEx 

Given the current number of destination options, it would not take a 
node too many extra cycles to find a destination option. What would 
worry me more is how this destination option search can be used against 
the *conex node* itself by sending it a datagram stuffed to the gills 
with destination options. In this regard, I share your concern. As I 
said before, destination options are not the best alternative. They are 
simply the best alternative with any chance of adoption.

I have a possible mitigation in mind, but I need to verify if I am way 
out of line. What I was thinking about was to restrict the conex-aware 
senders to always insert the conex destination option as the first 
destination option if it is present at all. I think this would take care 
of all my DDoS concerns and your processing load concerns. What do you 
think.

Thanks
Suresh

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 16 Apr 2011 05:50:58 +0200 (CEST)

On Thu, 14 Apr 2011, Bob Briscoe wrote:

> This should quite strongly insulate responsive traffic from unresponsive 

How would the node know that the traffic is responsive without keeping 
flow state?

> 3. and lastly discard Re-Echo and Credit marked ConEx traffic.

This makes sure all DDoS traffic will be marked as such. The people making 
these tools are not stupid (well, some are, but there are very clever 
people as well).

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 16 Apr 2011 12:11:02 +0100

Suresh,

At 23:59 15/04/2011, Suresh Krishnan wrote:
>>Something is bugging me about using a destination option as the 
>>ConEx v6 encoding. I'm worried about how many cycles it would take 
>>for general forwarding machinery (if suitably modified) to find the 
>>ConEx destination option.
>
>worry me more is how this destination option search can be used 
>against the *conex node* itself by sending it a datagram stuffed to 
>the gills with destination options. In this regard, I share your 
>concern. As I said before, destination options are not the best 
>alternative. They are simply the best alternative with any chance of adoption.
>
>I have a possible mitigation in mind, but I need to verify if I am 
>way out of line. What I was thinking about was to restrict the 
>conex-aware senders to always insert the conex destination option as 
>the first destination option if it is present at all. I think this 
>would take care of all my DDoS concerns and your processing load 
>concerns. What do you think.

This would solve our problem, but potentially create a problem for 
someone else in the future who also needs to be first.

However, as we're the first one to want to solve the problem of 
needing non-critical options that are readable at L3, I guess we have 
no other option (sorry for the pun).

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 16 Apr 2011 12:37:16 +0100

Mikael,

At 04:50 16/04/2011, Mikael Abrahamsson wrote:
>>This should quite strongly insulate responsive traffic from 
>>unresponsive flooding traffic (see background below).
>
>How would the node know that the traffic is responsive without 
>keeping flow state?

I explained how it works in the "(see background below)" part of the 
original posting that you have snipped.

Nothing there needs flow state. Which part are you implying needs flow state?

(note a robust audit function does require flow state, but the DDoS 
mitigation aspect of the protocol works without that).

>This makes sure all DDoS traffic will be marked as such. The people 
>making these tools are not stupid (well, some are, but there are 
>very clever people as well).

Please re-read what I carefully wrote lower down the original posting.

It caters for the cases where DDoS traffic is and is not marked 
correctly (cases b & c). In one case it gets caught at a bulk 
congestion policer. In the other case it gets caught at the bulk 
preferential drop in the queue.

I don't mind valid criticism if you've read the material.

(BTW, this went through a year of review by DDoS researchers before I 
brought it to the IETF.)

Bob

---

From: "Toby Moncaster" <[email-masked]>
Date: Sat, 16 Apr 2011 14:13:52 +0100

Trying to recap:

preferential drop gives priority to marked traffic. Also, ISPs are expected
to be policing/rate limiting/whatever marked traffic at the access.

This means any DDoS host can't send unmarked traffic (because it will get
preferentially dropped as soon as a node starts to congest) and if it sends
marked traffic it gets limited by the policing function at the access. Hence
the impact of the DDoS traffic is significantly limited at the first
congested node along the path...

This requires no auditing, indeed the preferential drop actually reduces the
need for audit since it provides an incentive to mark traffic during
congestion...

Is that a fair summary?

Toby

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Bob Briscoe
> Sent: 16 April 2011 12:37
> To: Mikael Abrahamsson
> Cc: ConEx IETF list
> Subject: Re: [conex] DDoS and ConEx-awareness in the forwarding plane
> 
> Mikael,
> 
> _______________________________________________
> conex mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/conex

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 16 Apr 2011 15:33:19 +0200 (CEST)

> Nothing there needs flow state. Which part are you implying needs flow 

The part where the source isn't on a conex aware network and thus there is 
no ingress policer, but the node still uses Conex markings in the traffic 
it generates.

> (BTW, this went through a year of review by DDoS researchers before I 

Well, again, I might miss the point totally. I still don't understand how 
this is supposed to work in the real world. For me it's still a lot of 
hand-waving and high-level.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 16 Apr 2011 15:36:02 +0200 (CEST)

On Sat, 16 Apr 2011, Toby Moncaster wrote:

> preferentially dropped as soon as a node starts to congest)

RFC2827 is now over 10 years old, I would imagine the original document is 
a lot older.

Still a lot of networks do not do basic ingress filtering towards their 
customers, enabling spoofed packets to get through.

I'm sceptical that they would implement a conex policer, at least not all 
of them.

---

From: "Toby Moncaster" <[email-masked]>
Date: Sat, 16 Apr 2011 14:46:52 +0100

> From: Mikael Abrahamsson [mailto:[email-masked]]
> Sent: 16 April 2011 14:36
> To: Toby Moncaster
> Cc: 'Bob Briscoe'; [email-masked]

No, not all of them will implement it, but the any that do so will see the
benefit -- remember that ConEx is destined for experimental track in the
first instance, precisely because some people are sceptical about its
effectiveness.

Incidentally, I take it you aren't proposing that the IETF only produce work
that is guaranteed to be adopted by all operators? If so we may as well pack
up and go home because there are always operators that will try and limp
along with older equipment!

Incidentally, I know I keep referring people to it, but do read "Why the
Internet only just works" by Mark Handley
<http://www.cs.ucl.ac.uk/staff/m.handley/papers/only-just-works.pdf>. It
gives a very good explanation of how and why the Internet has tended to
stagnate in recent years.

Toby

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 16 Apr 2011 16:59:24 +0200 (CEST)

On Sat, 16 Apr 2011, Toby Moncaster wrote:

> No, not all of them will implement it, but the any that do so will see 
> the benefit -- remember that ConEx is destined for experimental track in 
> the first instance, precisely because some people are sceptical about 
> its effectiveness.

Ok, let's back up a bit.

I'm saying that any DDoS:er with any kind of skill will set the correct 
connex options to get the highest impact of his DDoS. I'm also saying DDoS 
Zombies will exist on all networks, both CONEX enabled and non-CONEX 
enabled. Last I heard the proposal was also to use destination options to 
carry the conex information?

So a non-conex source network with a host sending conex marked traffic 
will not have well-behaving flows, because the flows are completely under 
the sender machine control, not any conex mechanism in that network.

Since destination options are used, I have a hard time imagining that the 
border router between the conex network and the non-conex network would 
change the conex destination options on the packets?

> along with older equipment!

No, but I'm proposing that the fact that 100% penetration by conex is 
never going to happen is taken into account when writing text.

> stagnate in recent years.

There is nothing in there that was any news to me. What's your point?

---

From: John Leslie <[email-masked]>
Date: Sat, 16 Apr 2011 18:15:18 -0400

Mikael Abrahamsson <[email-masked]> wrote:
> conex options to get the highest impact of his DDoS. I'm also saying DDoS 

   Yes.

> the sender machine control, not any conex mechanism in that network.

   Yes.

> change the conex destination options on the packets?

   Nonetheless, it could...

   Actually, I'd like to digress on that: I envision an experimental phase
where few if any backbone routers are at all ConEx-aware. Instead, I
expect separate boxes for auditing / policing functions. It's entirely
practical to build such boxes rather cheaply at gigabit-ethernet speeds,
to evaluate any number of IPv6 extension headers.

   These could run at wire-speed, changing anything within the ConEx
destination options header that may need changing, with the only practical
issue being jitter in the time before a decision to drop can be made.

   As a practical matter, 1500-byte packets at gig-E speed can't cause
more than 12 microseconds of jitter -- this isn't likely to be a problem
during the experimental period (though I could imagine a future where it
might become a problem).

>... I'm proposing that the fact that 100% penetration by conex is 

   To be fair, we don't expect this DDoS use-case to be in our initial
document at all.

   Nonetheless, I'd be delighted to discuss a DDoS use case on-list.
Fundamentally, at any peering point there could be a default policy on
accepting ConEx marking (in the absence of actual contractual policy)
which limits (by any of several algorithms) the number of congestion-
expected marks you accept to just a bit more than the number you send.

   In essence, you would apply the same allowance-based rule you would
apply to a direct customer (in the absence of a contractual agreement
to pay for excess marks).

   This means that the congestion-expected marks from DDoS-bots would
disappear -- and peers sending well-behaved traffic would get preferred
forwarding.

   (Granted, well-behaved customers of peers that also send bad traffic
_wouldn't_ get preferred forwarding; but they're not _your_ customers,
so you have no obligation to give them preferred forwarding: their
beef is with their upstream.)

   (Also, if you wish to give _your_ customer the benefit of preferred
forwarding toward them from certain sources, you could work out some
stateful deep-packet-inspection scheme to do so -- but that's NOT a
ConEx problem.)

--
John Leslie <[email-masked]>