Subject: Re: [pm-dir] Performance directorate review of draft-ietf-alto-deployments

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Thu, 13 Feb 2014 17:41:32 +0000

Hi Vinayak,

Finally, I found some cycles to re-write the section you flagged. I first wanted to mostly remove the content, but since draft-ietf-alto-protocol includes a reference to this content, I tried instead to rewrite the section to illustrate the high-level idea, whithout going into details (which would probably require another draft). The new text states very explicitly that it does not intend to formally specify any measurement method or metric. The new text can be found here:

http://tools.ietf.org/html/draft-ietf-alto-deployments-09#section-3.4

Please let me know if I missed something or if you have suggestions for further improvement.

Thanks

Michael

From: Vinayak Hegde [mailto:[email-masked]]
Sent: Monday, September 23, 2013 11:33 AM
To: Scharf, Michael (Michael)
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: Performance directorate review of draft-ietf-alto-deployments

On Fri, Sep 13, 2013 at 6:50 PM, Scharf, Michael (Michael) <[email-masked]<mailto:[email-masked]>> wrote:
[snipped]
Thanks again!

You are welcome. Comments inline.
________________________________
From: Vinayak Hegde [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Saturday, September 07, 2013 6:23 PM
To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; Scharf, Michael (Michael)
Cc: [email-masked]<mailto:[email-masked]>
Subject: Performance directorate review of draft-ietf-alto-deployments
Hi,
There was a performance review either requested by you or was triggered by Benoit's script. I am reviewing this from a performance metrics point of view.
I looked at the draft (https://datatracker.ietf.org/doc/draft-ietf-alto-deployments/?include_text=1) These comments are only on the Section 6.3 Monitoring ALTO
Comments on Section 6.3
The authors should define some recommended metrics here (possibly IPPM Type-P metrics) if not recommended metric here. This section can be substantially expanded.

[MS] I am not sure if this document should really formally define metrics (the current wording implies that). The more important aspect seems to me the overall effects that an operator may want to look at when deploying ALTO. This significantly differs between different use cases of ALTO. For instance, if ALTO is used for traffic optimization inside one AS, monitoring a metric that describes inter-domain traffic may not be required. Therefore, the relevant metrics will depend on the use case, as also noted in your review. An alternative document structure would have some discussion of monitoring per use case, but this would imply a significant change of the table of content of the document. We currently look into this, but this requires feedback from the ALTO WG, too.
Yeah a couple of use-cases would really help though it is bit more work. It would make the document more complete and usable for implementers.

Comments on section 6.3.1
There are several metrics defined here but they can be more clearly defined by following the formats specified in RFC 6390. Also useful will some use-cases to explain why these metrics make sense and what needs to be used in which scenario.

[MS] Agreed. As I said, one potential solution is to move this discussion to the descriptions of the use cases (Section 4, 5, etc.), maybe with one "monitoring" subsection per use-case.

Yup. Agree. Makes sense.

As regards to the network hop-count to be used indirectly for latency, it might be better to define a different metric for latency as the average network loads and buffers on each device can be radically different. For example a hybrid network that contains  a backbone network, last mile access network and a mobile network (which is not unusual in many ISPs today). The delay for transit (as well as packet loss) internally from one to other might be drastically different.

[MS] I agree that hop-count and latency may not correlate. Also, measuring hop-count can be difficult in some cases. If the Internet traffic is carried over an MPLS/IP aggregation/core network, the number of routers that are traversed by the traffic can be pretty different to the number of IP hops. Reducing the IP hops does not necessarily imply a more efficient routing. My current thinking is that the document should more focus on "traffic locality" than path monitoring metrics.

Agree. Makes sense. Can you reword the mentioned paragraph ?
Section 6.3.3
The example given here can be expanded to give a more clearer picture of which are the active monitoring agents and which are the passive components. The metrics used for monitoring need to standardized. Also you might want to add some recommendation on the data interchange format and method of delivery. This is especially important as the interacting entities (ISP and Application Owner) are present at different layers and have different priorities.This can also be fixed in section 6.3.1 above.

[MS] Yes, the text and Figure 19 describes the scenario at a very high-level and, indeed, it is not clear. Both the question what somebody can measure, and whether this is done actively or passively, depends on the use case and who that entity actually is. But I don't believe that this document can provide comprehensive guidance on data exchange formats and methods of delivery, because inter-organization communication raises a lot of non-technical issues, and even standardization of new protocols might be needed. That seems outside the scope of this document specifically, and ALTO in general. This seems more related to LMAP.

The document need not be comprehensive. That was not the intention of the suggestion. But non-normative language about possible measures / metrics could be useful for guidance. Data interchange formats such as JSON could be used for easy parsing instead of binary serialisation.
-- Vinayak