Subject: RE: draft-ietf-multi6-multihoming-requirements-06.txt

From: Randy Bush <[email-masked]>
Date: Thu, 12 Jun 2003 04:53:05 +0900

An Ops-Dir reviewer had the appended comments.  excuse the droll
phrasing.  any chance we could address them?

randy

---

Let's see: load balancing is not a realistic requirement.  For the
architecture to be automatic, we have to be dynamic and we've
proven that we don't know how to do that yet.  Even to give people
some tools seems to lead to abuse.  Who controls the balancing? 
The source?  The destination?  Don't say 'both'.  No packet can
serve two masters.

Performance.  Exact same arguments since the only way to affect
performance is to load balance.

Policy: so ill stated as to be meaningless.  If this is a call
for policy routing, that's fine, but it has nothing to do with
multihoming.

Simplicity: motherhood, apple pie and chevrolet.

Transport survivability: Well, ok, but this is just a refinement
of the redundancy requirement.  How about we remove the redundant
redundancy
requirement?

Compatible with DNS: meaningless

Packet filtering: meaningless

Scalability: see simplicity

routers: create a new architecture, but don't change the routers, change
the hosts

hosts: create a new architecture, but don't change the hosts, change the
routers

Interaction between hosts & routers: tighter coupling between subsystems
has never been an architectural good idea.  So let's do that and change
both.

O&M: see pie, apple.

multiple solutions: please don't give us one size fits all, that's
not confusing enough

security: don't break security

Net:
The real requirements are:

	- Provide multihoming
	- Provide scalable routing
	- Transport survivability

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 12 Jun 2003 00:55:55 +0200

> The source?  The destination?  Don't say 'both'.  No packet can

No load balancing whatsoever means having a primary connection and a 
backup. For most multihomers paying for bandwidth you only get to use 
less than 1% of the time is a non-starter.

> performance is to load balance.

Having a fast link, adding a slow one and having performance limited by 
the slow one is again unaccaptable.

And it's not like providing some basic load balancing is a huge deal. A 
simple preference value for each of multiple AAAA records or some such 
would go a long way.

> 	- Transport survivability

The real real requirement is: it must be good enough that people will 
want to trade in their IPv4 provider independent address space for 
this. If it isn't, either they'll stay in IPv4 or they'll take the IPv4 
multihoming mess with them during the transition to IPv6.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 12 Jun 2003 09:10:58 +0300 (EEST)

> less than 1% of the time is a non-starter.

The real question was about *controlled, fine-grained* load balancing.  
That's entirely different than e.g. advertising identical prefix to two
ISP's in an identical fashion.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 12 Jun 2003 09:22:43 +0200

On donderdag, jun 12, 2003, at 08:10 Europe/Amsterdam, Pekka Savola 

> ISP's in an identical fashion.

Our anonymous friend seems opposed to any kind of load balancing: "Even 
to give
people some tools seems to lead to abuse."

The "abuse" we see today is mostly due to people wanting to prevent 
certain traffic from taking a certain route as long as there is full 
connectivity rather than trying to balance traffic over two or more 
connections. For the former you quickly arrive at announcing more 
specifics, for the latter this is seldom necessary.

And also: "Who controls the balancing? The source?  The destination?  
Don't say 'both'." This shows the author brings some assumptions to the 
table that don't hold up for load balancing in multihomed networks. 
Sure, if A and B are directly connected over two circuits and A prefers 
to use 1 while B wants to use 2, you have a problem. But with 
multihoming A's choice of the outgoing circuit doesn't automatically 
dictate B's incoming circuit. And even if it did, shifting the traffic 
where the other end doesn't care should be sufficient in most cases.

---

From: Joe Abley <[email-masked]>
Date: Thu, 12 Jun 2003 12:58:49 +0300

On Wednesday, Jun 11, 2003, at 22:53 Africa/Kampala, Randy Bush quoted:

> proven that we don't know how to do that yet.

For load balancing to be possible, the architecture does not need to be 
fully automatic. Load balancing is perfectly possible today using 
automatic mechanisms which are tuned manually.

> Even to give people
> some tools seems to lead to abuse.

Some clarification on this sentence would be useful; I am struggling to 
understand it.

>   Who controls the balancing?

If I am multi-homed, then I control the relative egress loading on my 
various transit circuits by managing my exit selection policies to 
choose exits which tend to balance traffic in a way that makes me happy.

I control the relative ingress loading by supplying clues to my transit 
providers and thereby attempting to influence the exit selection 
policies of those transit providers, and so on for all inbound traffic 
through the chain of ASes which leads to its source.

Both ingress and egress loading policies are necessarily malleable, 
since policy in external ASes and traffic demands vary and circuit 
sizes often do not.

This is not a deterministic, calculable process, but it works 
sufficiently today that people do it. Removing the possiblity of load 
balancing means that N-way multi-homed ASes will suffer an increase in 
transit fees which approaches a factor of N, and this is not a 
reasonable compromise to expect people to make in the interests of 
architectural simplicity. Most of the world does not enjoy cheap 
external connectivity.

> performance is to load balance.

In the context of current practices, the performance requirement is met 
by the ability of an AS to control its own exit selection policy.

> for policy routing, that's fine, but it has nothing to do with

It is common practice in many ISPs to multi-home between a cheap, 
poorly-performing transit service and a more expensive, 
better-performing transit service (a common example is expensive 
terrestrial fibre vs. satellite).

By arranging for batch, non-interactive traffic (mail, usenet, HTTP 
from pre-fetching web caches, FTP mirror updates, etc) to use the 
satellite service, there is more of the expensive-but-better capacity 
left for interactive services, and the quality of the user's experience 
is increased. In other ISPs, customers are able to buy multiple grades 
of service, corresponding to the quality of the transit used for their 
traffic (for some convenient definition of quality).

Policy routing is one way to do this, but it assumes that the owner of 
the policy has administrative control of the router the other side of 
the satellite link (from the perspective of bandwidth-starved ISPs, 
problem traffic is inbound, not outbound). This is frequently not the 
case.

Another approach is to bundle endpoints for batch traffic into 
particular netblocks which can be advertised in such a way that their 
inbound traffic comes exclusively over the satellite. In the v4 world 
this involves multihoming policy.

In general, this requirement does not arrive unless the site is 
multihomed (since with only one transit path, there is no choice about 
shifting traffic according to protocol or other policy). The comment 
that this has nothing to do with multihoming is hence confusing.

> Simplicity: motherhood, apple pie and chevrolet.

Perhaps there is some cultural reference I am missing, since this means 
nothing to me.

> of the redundancy requirement.  How about we remove the redundant

They are not requirements, they're goals. It is reasonable to assume 
that there will be multihoming architectures which do not meet all 
these goals (that, after all, is why we were unable to shift a draft 
without removing the "requirements" word).

It is possible to gain protection for connectionless traffic in the 
case of single transit failure without providing protection for 
connection-oriented traffic.

> Compatible with DNS: meaningless

I will interpret "meaningless" as "I do not understand this, and 
request clarification".

A multihoming solution which results in increased load on DNS servers 
by a factor of 10,000,000, or requires the behaviour of DNS servers to 
be modified (e.g. by requiring ordering of RRs in a reply to be 
modified according to source address for all multi-homed sites) is 
going to impose operational issues on the Internet. It does not seem an 
unreasonable goal to avoid such issues.

> Packet filtering: meaningless

Unicast RPF checks are commonly applied to external interfaces on ISP 
routers, and these filters demonstrably drop a lot of unwanted traffic. 
A multihoming architecture which prevented unwanted traffic from being 
dropped would result increased junk traffic being carried around the 
Internet. Similarly, to my comment above, it does not seem unreasonable 
to want to retain the capability to drop junk traffic at the ingress 
point of your network.

> routers: create a new architecture, but don't change the routers, 

The text says, in fact "the solutions may require changes to IPv6 
router implementations".

> hosts: create a new architecture, but don't change the hosts, change 

The text says, in fact, "the solution should not destroy IPv6 
connectivity for a legacy host implementing [list of RFCs]". It says 
nothing about not changing hosts.

> has never been an architectural good idea.  So let's do that and change

I see no text which advocates tighter coupling between hosts and 
routers; I see text which does not require the coupling between hosts 
and routers to become looser.

> multiple solutions: please don't give us one size fits all, that's

In fact the text says "multiple solutions will incur a greater 
management overhead, however, and the adopted solutions should attempt 
to cover as many multihoming scenarios and goals as possible".

> security: don't break security

If the anonymous observer thinks that the converse is more desirable, 
perhaps he or she could explain why.

> 	- Transport survivability

Summary: the goals that the anonymous critic finds important are 
covered by the draft. Other goals that the anonymous critic hasn't 
thought of, or doesn't find important, are also covered.

Joe

---

From: Vijay Gill <[email-masked]>
Date: Thu, 12 Jun 2003 08:21:08 -0400

> The source?  The destination?  Don't say 'both'.  No packet can

The load balancing does not have to be automagically done. For example,
with the current mechanisms available to us in v4 + BGP, a large promising
local dialup provider used to make extensive use of selective
announcements, local preference, selective prepends etc to ensure that the
traffic to its various transit providers was evenly balanced since most of
the billing was done on the higher on in+out and so effectively there was
the opportunity to optimize cost by moving someone's higher outbound to
another provider who had a high inbound.

> performance is to load balance.

I think this has been addressed in the satellite link for big bandwidth
latency insensitive transactions (ftp, nntp) but terrestial links for
interactive traffic example.

> for policy routing, that's fine, but it has nothing to do with

see example above of cost optimizing using mechanisms available in the
underlying protocols to move traffic around according to the policy
"minimize cost"

I think abley has covered the rest.

/vijay

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 12 Jun 2003 18:05:29 +0200

Well, I'm not sure how to treat these comments as constructive.
We know that there are conflicting goals here; that's exactly
why the draft was downgraded from being a requirements document.

I think the review is written from the viewpoint of a solution
or class of solutions. The draft is written more from a utopic
viewpoint. So I don't think most of the comments can actually
be transformed into text changes. 

The best fix I can suggest is to take this sentence from the
Abstract:

> This document outlines a set of goals for proposed new IPv6
> site-multihoming architectures.

and add to it something like
 It is recognized that this set of goals is ambitious and that
 some goals may conflict with others, and that the solution or
 solutions adopted will be able to satisfy only some of them.

> hosts: create a new architecture, but don't change the hosts, change the

That isn't what the text says. It is much more carefully written. It says
make modular, backwards-compatible changes to both hosts and routers.
People already shipping product tend to regard that as fundamental.

   Brian

---

From: Randy Bush <[email-masked]>
Date: Fri, 13 Jun 2003 06:24:13 +0900

> viewpoint. So I don't think most of the comments can actually be
> transformed into text changes.

i think the fear is that solutions workers will hit "it does not
meet requirement 14.3"

>  solutions adopted will be able to satisfy only some of them.

works for me

randy

---

From: Joe Abley <[email-masked]>
Date: Fri, 13 Jun 2003 01:39:08 +0300

>>  solutions adopted will be able to satisfy only some of them.

I have added that sentence to the document's abstract. If now is the 
time to cut a -07, someone should let me know (otherwise I'll sit and 
wait).

Joe

---

From: Randy Bush <[email-masked]>
Date: Fri, 13 Jun 2003 08:13:21 +0900

> time to cut a -07, someone should let me know (otherwise I'll sit and 

please cut it and i will put on iesg agenda

randy

---

From: Joe Abley <[email-masked]>
Date: Fri, 13 Jun 2003 11:54:56 +0300

> please cut it and i will put on iesg agenda

Done, sent to internet-drafts (authors, chairs, randy copied). Also 
attached below.

Joe

---

From: S Woodside <[email-masked]>
Date: Sun, 15 Jun 2003 13:29:41 -0400

Preferably there would be a way to automatically load balance though. 
:-)  (or at least it would be easy to set up even for a naïve user.)

simon

--
www.simonwoodside.com -- 99% Devil, 1% Angel

---

From: Vijay Gill <[email-masked]>
Date: Sun, 15 Jun 2003 13:33:33 -0400

> :-)  (or at least it would be easy to set up even for a naïve user.)

I'd say that we do not have the control theory-fu necessary to make
automagic loadbalancing work yet, so I'd leave that work for people
smarter than I.

/vijay