Subject: RE: Fwd: Minutes / Notes

From: "Tony Li" <[email-masked]>
Date: Thu, 17 Jul 2003 10:37:52 -0700

Noel,

|    This is only important if you want TCP connections to be 
|    able to survive
|    having an incoming link fail (i.e. the address on the 
|    local end becomes
|    unreachable to the rest of the network). This may not be 
|    an important goal
|    (e.g. the typical web site wouldn't care).

I believe that the WG has come to rough consensus that this is,
in fact, an important goal for us to solve.  There are 
numerous practical applications that drive this.  More generally,
we (IETF, vendors) are being asked to make the Internet safe
for "mission critical" applications and having broken TCP
connections is simply unacceptable.  Many applications today
are being outsourced: backups, storage, business applications,
interactions within an 'extra-net', etc.

Tony

---

From: "Grovesteen, Harold" <[email-masked]>
Date: Fri, 18 Jul 2003 11:03:23 -0500

Please understand that there is a middle ground between EDI financial based transactions and checking out a recreational web site.  Enterprises which depend upon user's browsing to generate revenue fit in this middle ground.  It may be casual browsing on the part of the customer, but there is nothing casual about the enterprise's approach to reliability to maximize that revenue stream.  When millions of dollars or yen or pounds are at risk, even short disruptions and any broken connections with those casual users represent lost revenue.

I had wanted to stay out of this discussion, but I had not anticipated the results of my simple affirmation that TCP connection survival is a good thing and I representing an Enterprise am glad this is still a "should" and "on the table."  I do not even have any issue that some solutions provide this and others do not.  I can select a solution that does and support my organization as needed--provided it is not incompatible with the other solutions.

Harold Grovesteen

-----Original Message-----
From: Brian E Carpenter [mailto:[email-masked]]
Sent: Friday, July 18, 2003 3:25 AM
To: Tony Li
Cc: [email-masked]
Subject: Re: Minutes / Notes

Tony,

Yes, but you can't use the same argument for casual web browsing.

BTW, this is why commercial message queueing systems exist; they
can bridge reliable transactions over significant disconnects.

   Brian

Tony Li wrote:
> 
> Brian,
> 
> If we do not fix this, then we can just close up shop and go home,
> because the business world is NOT going to accept a solution that
> doesn't fulfill this.  They would rather use IPv4 and PI addresses.
> 
> Tony
> 
> |    -----Original Message-----
> |    From: Brian E Carpenter [mailto:[email-masked]]
> |    Sent: Friday, July 18, 2003 12:58 AM
> |    To: [email-masked]
> |    Subject: Re: Minutes / Notes
> |
> |
> |    Well, this is one of the many "shoulds" in our agreed list
> |    of goals. Whether we can achieve this "should" simultaneously
> |    with enough of the others is very much an open question
> |    in my mind, and it's one of the reasons why we may end up
> |    with more than one solution. In some scenarios, this may
> |    be a dominant goal; in other scenarios, it may be unimportant.
> |
> |      Brian
> |
> |    "Grovesteen, Harold" wrote:
> |    >
> |    > YES!
> |    >
> |    > -----Original Message-----
> |    > From: Tony Li [mailto:[email-masked]]
> |    > Sent: Thursday, July 17, 2003 12:38 PM
> |    > To: J. Noel Chiappa; [email-masked]
> |    > Subject: RE: Fwd: Minutes / Notes
> |    >
> |    > Noel,
> |    >
> |    > |    This is only important if you want TCP connections to be
> |    > |    able to survive
> |    > |    having an incoming link fail (i.e. the address on the
> |    > |    local end becomes
> |    > |    unreachable to the rest of the network). This may not be
> |    > |    an important goal
> |    > |    (e.g. the typical web site wouldn't care).
> |    >
> |    > I believe that the WG has come to rough consensus that this is,
> |    > in fact, an important goal for us to solve.  There are
> |    > numerous practical applications that drive this.  More generally,
> |    > we (IETF, vendors) are being asked to make the Internet safe
> |    > for "mission critical" applications and having broken TCP
> |    > connections is simply unacceptable.  Many applications today
> |    > are being outsourced: backups, storage, business applications,
> |    > interactions within an 'extra-net', etc.
> |    >
> |    > Tony

---

From: "Tony Li" <[email-masked]>
Date: Sat, 19 Jul 2003 14:37:32 -0700

|    Frankly, I am not sure about the "pick one". It supposes 
|    that we know exactly what we want to design, and that we 
|    can commit to one right now. 

That's fine by me.  Let's bring back TUBA then.

;-) ;-) ;-)

Tony

---

From: "J. Noel Chiappa" <[email-masked]>
Date: Sun, 20 Jul 2003 18:50:15 -0400

> From: "Christian Huitema" <[email-masked]>

Christian, you've raised two important points. I'm going to reply to this one
first, separately.

    > there are very different trade-offs. For example, a fundamental design
    > choice) is the layer at which identifiers are maintain. Is it IP,
    > transport or session?
    > ...
    > Another quite important design choice is the scope of the identifier. ..
    > Is it a host? .. Consider clusters on one hand, multi-user systems on
    > the other.
    > ...
    > Yet another design choice is the life-time of the identifier. .. And
    > there are very different privacy and reliability implications between
    > the two variations.
    > ...
    > I trust that much more than a identifier/location split designed by
    > committee.

I think you're confusing two separate but related points.

Your points (expressed above) relate to the semantics and properties of a
potential "identity" name.

However, this is a *separate* set of questions from "do you split location
and identity".

If you do certain functional things, you *are* separating location and
identity, whether you admit it or not. For instance, MIPv6 separates location
and identity. The namespace they use for the "identity" name happens to be the
same namespace as the "location" namespace, but that's an orthagonal issue.

Anytime you separate location and identity, that separation brings with it
certain inevitable architectural and engineering points - and if you don't
believe me, take a look at the length of the MIPv6 spec. You have to secure
the binding, handle broken bindings, etc, etc, etc.

Similarly, iff you i) handle multi-homing with multiple addresses, and ii)
allow any "connection" which identifies the entity at the far end by its
address (such as TCP, as currently defined) to survive that address becoming
non-working - then you are again separating "location" and "identity", with
all the architectural and engineering issues that brings - *even if the
namespace you use for the "identity" name happens to be the same namespace as
the "location" namespace*, as in MIPv6.

It is for that reason that I press people to decide whether or not they really
need to support that capability, of having TCP connections survive the
failure of one address.

Having decided to separate location and identity, then you get to the
question of i) whether you use the same namespace for both, and if not ii)
what the properties of the second namespace are - the questions you raise
above.

But don't mistake these questions for the first question - of whether you do
the separation.

	Noel

---

From: "Christian Huitema" <[email-masked]>
Date: Sun, 20 Jul 2003 22:01:38 -0700

> Now, if you don't want to play, I understand, but doing something
> to fix the current situation is absolutely essential.  We cannot
> have scalable routing with PI addresses.  These are also a tax on
> everyone and a much higher tax where we can ill afford it.

I definitely believe that we need a solution for multi-homed sites. I just want to make sure that whatever tax we create is only paid by those who benefit from multi-homing, i.e. multi-homed hosts and their correspondents. Moreover, I submit that the tax should only be paid py those applications that actually benefit from TCP resiliency, i.e. those whose "mean connection time" is not small compared to the "mean time between rehoming events". Even so, applications that have their own identification and reconnection mechanisms should not have to pay the identifier tax.

-- Christian Huitema

---

From: "Tony Li" <[email-masked]>
Date: Sun, 20 Jul 2003 23:15:04 -0700

I think a tax that is of a bigger concern is that we need to affect most
of the predominant implementations out there.  Once we get past that
hurdle, having a host that knows that it's multi-homed or it's
corresponding
with a multi-homed host is a small additional tax with substantial
benefits.

Tony

|    -----Original Message-----
|    From: Christian Huitema [mailto:[email-masked]] 
|    Sent: Sunday, July 20, 2003 10:02 PM
|    To: Tony Li; J. Noel Chiappa; [email-masked]
|    Cc: [email-masked]
|    Subject: RE: Fwd: Minutes / Notes
|    
|    
|    > Now, if you don't want to play, I understand, but doing something
|    > to fix the current situation is absolutely essential.  We cannot
|    > have scalable routing with PI addresses.  These are also a tax on
|    > everyone and a much higher tax where we can ill afford it.
|    
|    I definitely believe that we need a solution for 
|    multi-homed sites. I just want to make sure that whatever 
|    tax we create is only paid by those who benefit from 
|    multi-homing, i.e. multi-homed hosts and their 
|    correspondents. Moreover, I submit that the tax should 
|    only be paid py those applications that actually benefit 
|    from TCP resiliency, i.e. those whose "mean connection 
|    time" is not small compared to the "mean time between 
|    rehoming events". Even so, applications that have their 
|    own identification and reconnection mechanisms should not 
|    have to pay the identifier tax.
|    
|    -- Christian Huitema
|    
|    
|

---

From: "J. Noel Chiappa" <[email-masked]>
Date: Mon, 21 Jul 2003 12:08:56 -0400

>> If you have two separate multi-homing mechanisms, then the cost, in
    >> system complexity, implementation size, amount of protocol
    >> specification work needed, etc, etc just seems to me out of all
    >> proportion to the benefit. .. Pick one.

    > Frankly, I am not sure about the "pick one". It supposes that we know
    > exactly what we want to design, and that we can commit to one right
    > now.

That's the job of an architect - to see things at a deeper level than just
today's requirements, to see past them to the fundamental basics and create
structures with lasting utility.

    > a fundamental design choice) is the layer at which identifiers are
    > maintain. Is it IP, transport or session? .. It is very unclear to me
    > that IP would in fact be the right layer for the job.

This is an interesting point, because I've often thought that with a blank
slate, perhaps a better allocation of mechanism to layers of functional
abstraction might be possible. E.g. when I think about maintaining a
connection to something like a pool of servers, you might move functions
around to different layers.

However, we have to live with the system we've got, and it still seems to me
that as long as we have an end-end reliable stream level which is fairly
ubiquitous (i.e. TCP), and we have the routing architecture we've got, and we
want the system overall to have certain capabilities (e.g. mobility, switching
to a different multi-homed inbound link), then that layer needs names for the
entities.

    > Another quite important design choice is the scope of the identifier.
    > .. The scope of an identifier system, by definition, has to be wider.
    > Yet, we don't really know how wide. Is it a host? .. Consider clusters
    > on one hand, multi-user systems on the other. Would you have an
    > identifier per node in the cluster, or one per user in a multi-user
    > system? What about process mobility?

One of the fundamental concepts of the end-end reliable stream layer is the
notion of fate-sharing. So I have said, for many years now, that the thing
that needs to be named by an end-end RSL name is a fate-sharing region.

E.g. if you have two machines in a pool, and one is (temporarily) handling a
TCP connection, and the other doesn't have all the latest TCP state
information to take over the connection if the first one fails, then the two
are not a fate-sharing region, and cannot share an end-end RSL name.

    > I heard many say during the multi6 debate that applications cannot
    > possibly handle multiple addresses well. To disprove that, I suggest
    > you take a look at the variations of "swarmcast" implemented by P2P
    > systems .. Such a system would handle multi-addressing beautifully.

By definition, if we do multi-homing with multiple addresses, *something* is
going to have to handle multiple addresses. It may be the application (some
will no doubt care), it may be some other layer.

But you only have a need for a separate name for the end-end entity *if you
wish to provide certain functionalities*, e.g. mobility, or multi-homing, or
anything that (in general) requires the ability to switch to a different
address while the connection to that end-end entity is active. (And don't
forget, a set of addresses is an end-end name, too... viz SCTP.)

    > But my real point when bringing up the P2P example is that innovation
    > happens. .. Office automation systems use transaction processing. The
    > list goes on. What we are observing is a distributed innovation process
    > arbitraged by the market place. I trust that much more than a
    > identifier/location split designed by committee.

Yes, and they all are working within an architectural framework that was
thought out in advance to provide the kind of flexibility needed to do this,
by providing certain key fundamentals, and leaving the rest open.

What we need to do here is identify the key architectural fundamentals, and
provide them.

	Noel

---

From: "Christian Huitema" <[email-masked]>
Date: Mon, 21 Jul 2003 09:55:18 -0700

> with a multi-homed host is a small additional tax with substantial

I don't know about most implementations. We definitely need a solution
that is "upward compatible" with the current implementations, so we
should assume that there will be co-existence for a long time. And I
could definitely see why some implementations would just decide to
forget the benefits of multi-homing, or at least those of TCP
survivability. But I am concerned mostly with the tax on applications
that run on a multi-homed system, yet are perfectly happy dealing
directly with addresses/locators. I would assert that this is the bulk
of current applications.

-- Christian Huitema

---

From: "Tony Li" <[email-masked]>
Date: Mon, 21 Jul 2003 10:40:41 -0700

|    I don't know about most implementations. We definitely 
|    need a solution
|    that is "upward compatible" with the current implementations, so we
|    should assume that there will be co-existence for a long 
|    time. And I
|    could definitely see why some implementations would just decide to
|    forget the benefits of multi-homing, or at least those of TCP
|    survivability. But I am concerned mostly with the tax on 
|    applications
|    that run on a multi-homed system, yet are perfectly happy dealing
|    directly with addresses/locators. I would assert that this 
|    is the bulk
|    of current applications.

Back in Ye Olden CIDR days, we estimated that about 10% of all sites
were multihomed and that the most common ones were large enterprises.
Those same large enterprises typically run a large variety of different
OS's, so to provide this benefit, we clearly need to hit a high 
percentage of the v6 implementations.

The size of the tax is not high.  If we work it so that we exchange
identifiers as a TCP option only on SYNs, then we're talking about
something like 40 bytes of additional overhead.  And this is in
a packet that is normally quite small.

I realize that there are some applications out there that could
be munged to handle addresses and locators.  However, changing them
one at a time is both architecturally ugly and very unlikely to
let us make substantial forward progress.  While this may well be
the bulk of current applications, our job is not to architect for
today's applications.  It's to deal with tomorrow's.

Tony

---

From: "J. Noel Chiappa" <[email-masked]>
Date: Mon, 21 Jul 2003 20:12:38 -0400

> From: Keith Moore <[email-masked]>

    > the question should not be whether TCP connections should be able to
    > survive link failures - because clearly there are some kinds of link
    > failures that cannot possibly be survived.
    > A better question is: How long should an application be able to expect
    > a TCP connection to last without needing to implement its own
    > ack/retransmit/duplicate-suppression logic? My view is that a TCP
    > connection should not become a significant additional source of
    > failure. So it be able to last as long as its endpoints are likely to
    > stay "up"

Aren't these two points in opposition to each other?

You can't expect a TCP connection to survive a link failure - but TCP
connections should stay up as long as the endpoints are up?

Or are you assuming that the time between link failures is much, much higher
than endpoint uptimes? I can tell you from experience (getting to MIT from
here in VA) that that's not true! :-)

	Noel

---

From: "Tony Li" <[email-masked]>
Date: Mon, 21 Jul 2003 17:56:29 -0700

|        > Unfortunately, we've put the IP address in the 
|    pseudo-header checksum,
|        > so when your locator changes, your connection fails. .. 
|        > Creating an identifier space is one way. 
|    Alternately, changing the
|        > pseudo-header would be another, but we discarded 
|    that as being even
|        > more unlikely.
|    
|    Hmmm. Why would adding a new namespace be any easier than 
|    changing the TCP
|    checksum? 

Because the politics are easier.  No, I don't like that, but
it's very clear to me that pushing rocks up steep hills is
harder than pushing rocks downhill.  I'm learning to live
with the political realities that I don't like and still
make forward progress.

|    In either event, if you're multihomed, and you 
|    want to interoperate
|    with hosts elsewhere, they have to be modified before you 
|    can talk to them.

They have to be modified before they can provide connection
survivability.  You still have basic connectivity.

|    Actually, this leads me to think. If interoperation with 
|    unmodified hosts is
|    important, then all this discussion about alternative 
|    namespaces to identify
|    endpoints is moot. At that point, there are precisely two 
|    mechanisms
|    available:
|    
|    i) Use multiple addresses, and if the link whose address 
|    you're using for the
|    connction fails, the connection fails.
|    
|    i) Wse multiple addresses, and if the link whose address 
|    you're using for the
|    connction fails, use MIPv6 mechanisms to divert it to a 
|    backup address.
|    
|    Note that either of these can be used, and you can use 
|    whichever of these
|    fits your needs best. This does tend to fill Brian's 
|    suggestion that we have
|    a range of solutions available...

The design team came up with an alternative that did not 
require a tunnel endpoint buried inside of an ISP network,
used multiple PA addresses, and provided for connection
survivability.  It has the advantage(?) that locators and
identifiers are both syntactically IPv6 addresses so that
unmodified hosts can continue to function without difficulty.

Yes, there are many different approaches to a specific solution,
but the need to create the namespace and manage it is clearly
overhead.  We don't wish to multiply this by having a variety
of implemented solutions.  While that might ease the standards
body politics, that would be effectively undeployable and thereby
useless.

Tony

---

From: "Tony Li" <[email-masked]>
Date: Wed, 23 Jul 2003 18:15:55 -0700

|    It will IMO also grow if multihoming becomes easier ;-) 
|    With wireless 
|    networks it is much easier to multihome at the PHYSICAL 
|    layer than it 
|    was a few years ago.

Amen.  I'm 'accidentally' multi-homed at my house because
my neighbor and I both connected our cable modems to 802.11b
nodes.  Fortunately, we have different cable providers...

Tony