Subject: Re: [lisp] Fwd: Anybody can participate in the IETF (Was: Why is IPv6 broken?)

From: [email-masked] (Noel Chiappa)
Date: Mon, 18 Jul 2011 20:47:52 -0400 (EDT)

<Turning to the DoS issue...>

    > From: Jeff Wheeler <[email-masked]>

    > If you don't want to ignore DoS, then you should concern yourself more
    > with ensuring that it is practical for an xTR to carry all of the
    > possible mappings and not rely on a cache at all. That is what is
    > needed for a targeted xTR to be "DoS proof" to the same degree that
    > common BGP routers are today

You've identified a problem (DoS), but why do you insist that the only
solution is "for an xTR to carry all of the possible mappings"? That's kind of
like saying that gasoline is going to become scaree, therefore all cars should
get 2,000 MPG - ignoring alternative solutions like other fuels, etc, etc. As
long as the solution _works_, what do you care what the mechanism is?

(Incidentally, that approach to handling mappings was considered extensively
several years ago - see NERD - and it has problems of its own that led to it
being dropped.)

    > the same problem exists for ETRs if they are going to do any kind of
    > source address validation.

So I can understand this fully, why would they be wanting to do this?

    > I think the efforts of LISP developers should be focused on this issue
    > ... If this problem can be overcome, LISP has a great potential.

Perhaps you'd be interested in working on this issue, then (especially in
light of your first comment immediately below)? I take it from the fact that
you didn't comment on most of the earlier material in that message - e.g. my
three-part threat area characterization (cache churn, control traffic,
missing-mapping user data packets) - that the stuff up until this point was
OK? Perhaps that could all be used as the basis for an I-D on the subject?

    > I like the concept of LISP more than is apparent from my postings --
    > but I also live in the practical world where I see 10Mpps DoS attacks
    > every day.

Understood.

    > it is really the only huge, show-stopping issue I see that makes it
    > impractical for Internet-scale use.

Actually, there is at least one more I can think of... :-)

    > imagine that a malicious program is installed on one subscriber's PC
    > behind an ITR that serves ten thousand subscribers (nominal.) You don't
    > want service for all those subscribers to be disrupted because of one
    > infected machine.

I got an off-line response to an earlier message which suggested that one
solution to the 'bounce' attack would be a per-source classifier for the
mapping lookup rate-limiter; that way, the source causing all the new cache
entries would be slowed down, and nobody else would be affected.

I'm not sure how useful that would be for 'bounce' attacks (which could be
directed at all of the servers at a site), but it would certainly work well
against this particular one (and ones that look like it, like the mixed
data-center one you mentioned next).

	Noel

---

From: Jeff Wheeler <[email-masked]>
Date: Mon, 18 Jul 2011 21:36:27 -0400

> solution is "for an xTR to carry all of the possible mappings"? That's kind of

That is the only solution which would have parity with the
DoS-tolerance of today's "BGP routers."  If you can invent a better
means of handling this problem, that will be good news for everyone.
But so far, LISP is simply relying on the promise that the cache
management mechanisms will be good enough.

>    > the same problem exists for ETRs if they are going to do any kind of
>    > source address validation.
>

Perhaps I am confused about this, but my understanding is that the
LISP community is quite interested in the ability to prevent source
address spoofing and does want the ETR to somehow have this
capability.  It has been suggested, in the past few days in fact, that
it would be a tool for preventing the reflection-based ITR DoS I have
supposed.  However, that is not possible if ETRs aren't able to do
this.

It is fairly common for carrier networks on today's Internet to run
loose uRPF on their IP backbones.  It stops them from having to carry
useless traffic for which there can be no legitimate response, while
still allowing them to bill for that traffic at their customer ingress
points.  An ETR also can't do this if it is not able to know at least
all positive mappings.  For it to know all positive mappings, it
either must not rely on the cache, or be subject to the negative
response attack and resulting churn problems I have outlined.

> OK? Perhaps that could all be used as the basis for an I-D on the subject?

I have barely scratched the surface in the area of control traffic /
MS infrastructure, etc.  I do not see value in even examining these
things in detail until there is serious effort to make progress on the
problem of DoS-induced cache churn, and a broader understanding of how
a dramatic increase in the number of multi-homed SOHO networks will
affect the number of cache entries that must be supported by ITRs.

If the walls of a building were about to collapse and no one knew yet
how to repair them, I would not worry too much about a leaky roof!

>    > it is really the only huge, show-stopping issue I see that makes it
>    > impractical for Internet-scale use.
>

If you are imagining an excess of signalling traffic within the MS
infrastructure (and my understanding in this area is very weak) I
believe that is a solvable problem.  If you have some other problem in
mind, I would be interested in reading about it.

> entries would be slowed down, and nobody else would be affected.

This is practical to implement in some network designs, and is hard in
others.  It is a good idea but it doesn't solve the problem that the
targeted host can still be disabled with a very small amount of
malicious traffic.  As I keep mentioning, modern routers can only do
around 20k FIB updates per second.  So with your
per-L2ADJ-MS-punt-policer concept, any host can still be disabled with
a DoS attack of < 20k PPS.

> directed at all of the servers at a site)

Indeed.  :-/

-- 
Jeff S Wheeler <[email-masked]>
Sr Network Operator  /  Innovative Network Concepts

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 19 Jul 2011 09:15:24 +0200

Hi Noel,

> entries would be slowed down, and nobody else would be affected.

As already mentioned on this thread I presented the case that such 
massive bouncing behaviour would not be due to attack, but by legitimate 
use of the server with some attractive content.

I would question if the answer should be in slowing down the data.

I would rather see also as already mentioned to fall-back/punt to some 
more powerful proxies rather then slowing down.

And btw I think NERD's idea was not pursued due to issues found. I think 
ALT was chosen just to get something rolling. IMHO we should really 
experiment with number of mapping planes not just one. FWIW I like 
NERD's approach .. well if not to all sites .. maybe to only those which 
may ever be subject to such data patterns like discussed above.

Best regards,
R.

---

From: Patrick Frejborg <[email-masked]>
Date: Tue, 19 Jul 2011 10:21:14 +0300

> supposed.  However, that is not possible if ETRs aren't able to do

I believe this scenario is a real threat for cache churn, i.e. if the
ETR is going to prevent source address spoofing. The hacking culture
has recently moved into a new era, see
http://www.time.com/time/business/article/0,8599,2079423,00.html

The communities do have really large botnets, with millions of
"clients". Depending on how the clients are located - worst case is
that each client is located at a specific DFZ prefix (which is not
likely) - a botnet might perhaps create a cache churn and there is no
way the ETR can separate fake traffic from real traffic.

The reflective attack at an ITR can be prevented by a rate limiter (as
Noel pointed out), by identifying the botnet client by the source
address. But what if the botnet client changes randomly source address
within the subnet for each request??

Patrick

---

From: Jeff Wheeler <[email-masked]>
Date: Tue, 19 Jul 2011 14:22:43 -0400

> within the subnet for each request??

Noel's suggestion is not to rate-limit based on Internet source addresses,
or any layer-3 addresses in fact; but based on the layer-2 adjacency from
which packets requiring a punt and MS resolve arrive at the ITR.  So if you
had a network like (please excuse my use of gmail Rich Text):

       Internet
          |
         ITR
          |
        switch
        /    \
       /      \
    host1    host2

Packets may arrive at the ITR from host1 at a huge rate and cause cache
churn (DoS) but the ITR may have two policers, one for punts from host1
(based on L2 address) and one for punts from host2.  The excessive punts
from host1 may be marked-down to a lower priority so, when the limit of the
punts the ITR can process is reached, host2 will still be able to establish
communication to new clients, while host1 would be affected by the DoS
directed against it.

I have had a bit more time to think about this idea, and cache eviction is
also a problem here.  The traffic from host1 could still cause enough churn
in the ITR that all mappings might be evicted from the FIB, which would then
cause all hosts to appear to be churning even though they are sending only
routine traffic.  One might say, don't evict a "busy" cache entry to install
a mapping for only one packet if the cache is currently full; but it is not
safe to assume that an attack would not simply send several packets (in
fact, TCP stack on host1 would do this normally, in the case of a reflection
attack where it is sending SYN|ACK several times.)

So even if punts are policed per L2ADJ, cache eviction is very much
non-trivial in this situation.  Would you decide to limit the amount of
mappings that should be installed as a result of traffic from a single L2ADJ
also?  Refuse to evict entries on behalf of the L2ADJ that is hitting its
punt policer during that time?  In this case, a somewhat slower attack that
does not reach that high water mark might still effectively DoS the ITR.

---

From: Patrick Frejborg <[email-masked]>
Date: Wed, 20 Jul 2011 09:59:37 +0300

> way the ETR can separate fake traffic from real traffic.
I had some more thoughts about the ETR scenario...
If there is an ETR then there is also an EID at the site, thus there
is somewhere most likely an ITR related to that EID, right?
So when the botnet clients are sending their SYN traffic through the
ETR, the server replies with an ACK and then the ACK will hit the ITR.
If the botnet clients are located in the EID name space the ITR will
create needed cache entries - today this is not an issue because the
testbed is quite small (compared to Internet), but if LISP deployment
is successful the probability that botnet clients resides at prefix
dispersed EIDs increases. And the more prefix dispersed botnet the
community owns the more cache entries it can create at a selected ITR.

I'm not an expert in LISP, thus do I miss something here?

Patrick