Subject: Re: [PMOL] What new work?
From: Daryl Malas <[email-masked]>
To: [email-masked]
Date: Wed, 22 Aug 2007 15:54:31 -0600
List-Id: Performance Metrics at Other Layers <pmol.ietf.org>

Based on no responses to the list, I think we may have our answer.

--Daryl

On Tue, 2007-08-07 at 09:01 -0600, Daryl Malas wrote:
> This question has come up a few times.  For the sake of understanding a
> potential charter of the working group, what valid problems exist in the
> industry today that require this potential WG to address?
> 
> Before any solution can be written, a problem must be identified.  If we
> can understand what problems we think exist, perhaps we can argue
> whether we think they are truly problems or not.  In addition, we can
> post the problems to the actual protocol WG groups as well.
> 
> For example...
> 
> (This is not a REAL example, I just made it up...any actual correlation
> to a real problem is coincidental.)
> 
> HTTP Performance Metrics
> 
> The problem is no HTTP metrics exist today, so individuals are faced
> with best effort expectations.  Also, no hosting companies have the
> ability to understand how well their web applications are performing
> based on the number of requests.
> 
> Okay....pretty simple....Draft idea and simple problem statement.
> 
> --Daryl
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Tue, 07 Aug 2007 11:27:36 -0400
To: Daryl Malas <[email-masked]>, [email-masked]
From: Al Morton <[email-masked]>
Subject: Re: [PMOL] What new work?
Cc: 

At 11:01 AM 8/7/2007, Daryl Malas wrote:
>This question has come up a few times.  For the sake of understanding a
>potential charter of the working group, what valid problems exist in the
>industry today that require this potential WG to address?
>
>Before any solution can be written, a problem must be identified.  If we
>can understand what problems we think exist, perhaps we can argue
>whether we think they are truly problems or not.  In addition, we can
>post the problems to the actual protocol WG groups as well.
Right.

Also, to deal with the question on practical applicability,
the description of the problem should specify *who* will use
the performance metrics, and what they will use them for.
The question of "Who is the Audience?" was raised last Summer
for the SIP PM draft, and again at the BOF for PMs in general.

The vetted problem statement, scope, motivation and applicability
might be captured in the early sections of the draft itself.
This is the typical approach in some PM WGs now.

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Tue, 07 Aug 2007 11:09:34 -0400
To: "Romascanu, Dan (Dan)" <[email-masked]>, "Daryl Malas" <[email-masked]>
From: Al Morton <[email-masked]>
Subject: RE: [PMOL] What is a Directorate?
Cc: [email-masked]

At 10:00 AM 8/7/2007, Romascanu, Dan (Dan) wrote:
>You may be correct, especially taking into account your own experience
>with the sip-performance metrics draft. However here is a concern on the
>other side of the argument that was expressed by some of the IAB members
>who participated in the BOF. Would not a permanent Working Group lower
>the threshold of approval of new work and encourage the apparition of a
>number of RFCs without immediate practical applicability?

This is a valid point of concern for any WG, but we should recognize
that the performance community is a tough audience, at least that's
my IETF experience.

>  ...The point is
>that whatever format this work will take we should be careful to
>establish firm criteria to start new work, that include starting new
>work only in coordination or even only at the request of the working
>group that is responsible upon the technology or protocol to be
>measured.

I too was worried that additional formalized performance work
might prompt an onslaught of proposals. So, the WG description in the
BOF proposal anticipated this problem and dealt with it head-on:

"...The WG would partner with a specific protocol development WG whenever
possible, to take advantage of willing experts and ensure that the metrics
developed are both appropriate and relevant. Last, the WG would create a
BCP document that would provide guidance for development of IETF APM
metrics documents, including the requirement to include relevant protocol
experts in the development.  The BCP may also define the process to
adopt new work in the APM WG, where a unique feature could be that proposals
would be vetted in the relevant protocol WG before reaching the APM WG.
The APM WG would avoid overlap with existing SDOs, and seek IESG
approval for each new Metric RFC before the WG spends any resources
on it."

I've since learned that the memo we called a "BCP" may be more
appropriately categorized as a Framework for Performance Metrics
Development. In any case, I think we've anticipated this problem.

Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: [PMOL] What new work?
From: Daryl Malas <[email-masked]>
To: [email-masked]
Date: Tue, 07 Aug 2007 09:01:18 -0600

This question has come up a few times.  For the sake of understanding a
potential charter of the working group, what valid problems exist in the
industry today that require this potential WG to address?

Before any solution can be written, a problem must be identified.  If we
can understand what problems we think exist, perhaps we can argue
whether we think they are truly problems or not.  In addition, we can
post the problems to the actual protocol WG groups as well.

For example...

(This is not a REAL example, I just made it up...any actual correlation
to a real problem is coincidental.)

HTTP Performance Metrics

The problem is no HTTP metrics exist today, so individuals are faced
with best effort expectations.  Also, no hosting companies have the
ability to understand how well their web applications are performing
based on the number of requests.

Okay....pretty simple....Draft idea and simple problem statement.

--Daryl

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: RE: [PMOL] What is a Directorate?
From: Daryl Malas <[email-masked]>
To: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Tue, 07 Aug 2007 08:51:42 -0600
Cc: [email-masked]

I find 100% validity in the concern, and it should be carefully
considered prior to the establishment of any working group...including
this one.

In many ways, I think it is a catch 22.  There are risks and advantages
to both sides.  I think I would then spend time investigating the
potential scope of work, and then validating it's need in the industry.
Perhaps then, we can establish better perspective on what to do;
however, how long will that take?!

Perhaps another option would be to immediately establish a Directorate,
while further investigation occurs to determine if a WG is warranted.

--Daryl

On Tue, 2007-08-07 at 16:00 +0200, Romascanu, Dan (Dan) wrote:
> You may be correct, especially taking into account your own experience
> with the sip-performance metrics draft. However here is a concern on the
> other side of the argument that was expressed by some of the IAB members
> who participated in the BOF. Would not a permanent Working Group lower
> the threshold of approval of new work and encourage the apparition of a
> number of RFCs without immediate practical applicability? The point is
> that whatever format this work will take we should be careful to
> establish firm criteria to start new work, that include starting new
> work only in coordination or even only at the request of the working
> group that is responsible upon the technology or protocol to be
> measured. 
> 
> Dan
> 
> 
>  
> 
> 
> > -----Original Message-----
> > From: Daryl Malas [mailto:[email-masked]] 
> > Sent: Tuesday, August 07, 2007 1:00 AM
> > To: Romascanu, Dan (Dan)
> > Cc: [email-masked]
> > Subject: Re: [PMOL] What is a Directorate?
> > 
> > Dan,
> > 
> > Thanks for this additional perspective.  My concern with the 
> > Directorate is purely the requirement for agenda time to 
> > fully hash out this type of work in other WG's.  Their 
> > agenda's and charter's are already very full in some cases, 
> > so the drafts would not receive the attention or priority to 
> > supersede protocol enhancements or bug fixes.
> > 
> > Correct me if I am wrong, this may only be my perception.
> > 
> > --Daryl
> > 
> 
> 
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] What is a Directorate?
Date: Tue, 7 Aug 2007 16:00:14 +0200
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: "Daryl Malas" <[email-masked]>
Cc: [email-masked]

You may be correct, especially taking into account your own experience
with the sip-performance metrics draft. However here is a concern on the
other side of the argument that was expressed by some of the IAB members
who participated in the BOF. Would not a permanent Working Group lower
the threshold of approval of new work and encourage the apparition of a
number of RFCs without immediate practical applicability? The point is
that whatever format this work will take we should be careful to
establish firm criteria to start new work, that include starting new
work only in coordination or even only at the request of the working
group that is responsible upon the technology or protocol to be
measured.=20

Dan

=20
=20

> -----Original Message-----
> From: Daryl Malas [mailto:[email-masked]]=20
> Sent: Tuesday, August 07, 2007 1:00 AM
> To: Romascanu, Dan (Dan)
> Cc: [email-masked]
> Subject: Re: [PMOL] What is a Directorate?
>=20
> Dan,
>=20
> Thanks for this additional perspective.  My concern with the=20
> Directorate is purely the requirement for agenda time to=20
> fully hash out this type of work in other WG's.  Their=20
> agenda's and charter's are already very full in some cases,=20
> so the drafts would not receive the attention or priority to=20
> supersede protocol enhancements or bug fixes.
>=20
> Correct me if I am wrong, this may only be my perception.
>=20
> --Daryl
>=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Should establish working group...
Date: Tue, 7 Aug 2007 15:47:54 +0200
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: "Daryl Malas" <[email-masked]>
Cc: [email-masked]

Thanks.=20

On a separate note (or not) I also believe that this is useful work and
I plan to push it ahead together with Cullen as AD-sponsored if no
framework is created in the near future.=20

Dan
=20

=20
=20

> -----Original Message-----
> From: Daryl Malas [mailto:[email-masked]]=20
> Sent: Tuesday, August 07, 2007 1:34 AM
> To: Romascanu, Dan (Dan)
> Cc: [email-masked]
> Subject: RE: [PMOL] Should establish working group...
>=20
> Some background and context regarding my draft...
>=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: RE: [PMOL] Should establish working group...
From: Daryl Malas <[email-masked]>
To: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 06 Aug 2007 16:34:05 -0600
Cc: [email-masked]

Some background and context regarding my draft...

I introduced the draft to the SIPPING WG a year ago.  It received good
feedback on the mailing list; however, more feedback through direct
email.  I requested agenda time to review it with the SIPPING WG last
summer.  After I introduced it, the WG chairs took a hum as to the
interest in it.  There was overwhelming interest in the topic, but the
WG chair lamented there was a very long queue of WG items; and, this was
not likely to make the list any time soon.  Also, there were questions
over where the draft should truly live, options included IPPM, BMWG, and
SIPPING.  I narrowed the scope and separated the work from what was
going on in the BMWG group (SIP Benchmarking).  It, then, fell outside
of it's charter and that of IPPM (back to SIPPING).  I continued to make
revisions and presented on those updates at the SIPPING WG.  Over and
over again, there was overwhelming interest and discussions at the
microphone.  In Prague, a couple of people came to the microphone and
expressed concern over when this draft would move forward as a WG item.
A number of people have voiced the desire for this draft to become an
RFC; however, there was concern the SIPPING WG did not have the level of
expertise to review it.  I simply asked the question if the SIPPING WG
wanted me to continue to update the draft.  I was told yes, and also
asked to continue with it by the AD (Area Director).  At this point,
Cullen Jennnings, RAI co-AD, expressed he would conference with WG
chairs from various groups and determine where the draft would
ultimately reside.  It was at this point, the idea of a new WG or
Directorate came to light.  After discussing it, Cullen decided he would
submit the draft as an AD sponsored draft.  I updated the draft with the
latest set of feedback and sent it to him and the SIPPING WG.  The draft
then stalled again, because the idea of this BOF was resurrected or
created.  Still, the draft sits awaiting the outcome of this.  An
additional note, after the BOF several people came up and again voiced
concern of WHEN this draft would move forward.

Hopefully, this provides some additional context.

Thanks...--Daryl

On Thu, 2007-08-02 at 16:31 +0200, Romascanu, Dan (Dan) wrote:
> Thanks Daryl for being the first to trigger the discussions on the new
> list. 
> 
> If possible I would suggest you and other folks who may express support
> for the formation of a working group to also make clear which of the two
> options discussed in the BOF you would support: 
> - permanent WG for definition of Performance Metrics at Other Layers,
> modeled on the IPPM WG 
> - short-lived WG to define a framework ('how to start, write, and review
> PMOL documents in the IETF') document and a couple of test cases
> 
> Also, it would be very useful if you can make a short history of your
> tribulations with draft-malas-performance-metrics, maybe in a separate
> mail. Many participants in the list may not be aware about the sinuous
> and yet uncompleted history of this document which was one of the test
> cases that let to the BOF. 
> 
> Dan
> 
> 
>  
> 
> 
> > -----Original Message-----
> > From: Daryl Malas [mailto:[email-masked]] 
> > Sent: Thursday, August 02, 2007 4:57 PM
> > To: [email-masked]
> > Subject: [PMOL] Should establish working group...
> > 
> > Okay, after taking a break from the BOF and talking with 
> > others about this.  I have become further convinced a working 
> > group should be established to handle this type of work.  If 
> > we only move forward with a Directorate, the work will have a 
> > very high unlikelihood of ever making the appropriate agenda 
> > time in the existing working groups to be fully developed and 
> > prioritized.
> > 
> > In the BOF, it was overwhelmingly decided this type of work 
> > is absolutely valuable to the members of the IETF.  
> > Furthermore, it was overwhelmingly decided many will commit 
> > their time to this work.  Those two decisions indicate this 
> > work will receive the necessary review, input, and expertise 
> > necessary to make it valuable to the members of the IETF and 
> > the industry as a whole.
> > 
> > Summary...establish a working group for APM, PMOL, or 
> > whatever we want to call it.
> > 
> > My two cents...
> > 
> > --Daryl
> > 
> > 
> > _______________________________________________
> > PMOL mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/pmol
> > 

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: Re: [PMOL] What is a Directorate?
From: Daryl Malas <[email-masked]>
To: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Mon, 06 Aug 2007 15:59:49 -0600
Cc: [email-masked]

Dan,

Thanks for this additional perspective.  My concern with the Directorate
is purely the requirement for agenda time to fully hash out this type of
work in other WG's.  Their agenda's and charter's are already very full
in some cases, so the drafts would not receive the attention or priority
to supersede protocol enhancements or bug fixes.

Correct me if I am wrong, this may only be my perception.

--Daryl

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Date: Mon, 6 Aug 2007 10:31:47 +0200
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: <[email-masked]>
Subject: [PMOL] What is a Directorate? 

The discussions in Chicago and on the list show that there is a
consistent sense of the community that the IETF should do something to
provide a framework for performance metrics definition at other layers
than IP. Many participants in the discussion expressed a preference for
a Working Group format.=20

In order to make sure that we have a full picture of all the options at
hand I would suggest that we look also at the cross-area directorate
alternative and understand what this option means and why it was
suggested as a starting point of the discussion.=20

Directorates are defined in RFC 2418:

   In many areas, the Area Directors have formed an advisory group or
   directorate.  These comprise experienced members of the IETF and the
   technical community represented by the area.  The specific name and
   the details of the role for each group differ from area to area, but
   the primary intent is that these groups assist the Area Director(s),
   e.g., with the review of specifications produced in the area.

The Area Directors often employ directorates and review teams for
assistance in tasks such as=20

- Document reviews as a part of their AD or IESG reviews, or to solicit
architectural input when the document is in its early stages.=20
- Discussion forum to talk about topics associated with a particular
technology.=20
- Help in forming an opinion about a rechartering or WG approval
decision.=20
- Tracking discussion and efforts related to a particular topic.=20

These teams can be created at the request of the Area Directors.
Typically both ADs for an area decide to create a team, but sometimes
the teams are more focused on one AD's topics, or even serve multiple
areas. The teams comprise experienced members of the IETF and the
technical community. The ADs are responsible for the selection of the
team members.=20

For efficient management, the teams typically need a chair. This could
be one of the ADs or one of the team members. Experience tells us that
review teams need to have an active person responsible for the
assignment and tracking of review tasks. The nature of the team
determines how large it can be. Most directorates are of size that can
easily meet for dinner at an IETF, while review teams can have a large
pool of reviewers. It is the AD's task to ensure that the membership of
the team is up to date and active. It is typically easy to get members
and have them perform initial work, but sometimes some members drop out
of active work in the directorate, or even the entire directorate grows
old and tired.=20

The current list of directorates and review teams can be found from
http://www.ietf.org/IESG/content/directorates.html. Example directorate
charter and description can be found in
http://www.ietf.org/IESG/content/mobdir.html.=20

When we discussed in the IESG how to support APM (now PMOL) it was the
idea of a cross-area directorate that came up first. The directorate
would be composed of individuals with experience in the field of
performance metrics definition, review and implementation at various
layers, who can volunteer to dedicate a slice of their time for this
activity. The directorate would be chartered for the following tasks:=20

- define a set of criteria and procedures to engage in IETF performance
metrics definition at other layers than IP
- support and advice working groups in the IETF that engage in work on
performance metrics definition - including advice on chartering and
early architectural advice
- coordinate between work items in the IETF and in other SDO in order to
identify and avoid duplication of efforts
- review documents that define performance metrics that have reached
IESG review phases

As you can see the directorate charter is not significantly different
than the possible charter of a PMOL WG. The difference is that a Working
Group would be specifically chartered with the creation of the framework
document for criteria and procedures and would possibly take a number of
performance metrics documents as work items. The directorate option
clearly favors doing all performance metrics definition in the
respective technology working groups and tries to encourage and
accelerate this process which did not work very well lately by providing
a framework and expert advice.=20

Dan

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Date: Fri, 3 Aug 2007 18:45:17 -0700
From: "Loki Jorgenson" <[email-masked]>
To: <[email-masked]>
Subject: [PMOL] Substance of PMOL work

I would suggest that a reasonable rule-of-thumb could be established for
routinely distinguishing between work that belongs primarily in the
proposed PMOL WG and work that arises in others and should remain there.
It seems given that there may always be a need to call on the expertise
of another WG in the process - it seems likely from the discussions to
date that there will be an integral cross-disciplinary nature to this
WG.

For example, following Daryl's comments, if a given draft can be
described by a series of attributes (e.g. SIP, performance metric) and
they can be ordered as to their significance, they might be compared to
the charter of the respective WGs.  In this case, it would seem to me
that the malas draft is primarily about performance metrics and SIP is a
qualifier (rather than the other way around).  The core competency then
should be performance-focused.

> On Thu, 2007-08-03 at 12:52 -0400, Daryl Malas wrote:
> I'm not sure I agree with keeping your drafts in AVT.  I have not read
> your drafts yet, but if they are "performance metrics" based drafts;
> they should be in this new WG.  As with my SIP metrics draft, the
> SIPPING WG has provided a good forum to receive feedback; however, the
> draft is not truly within the charter of the group.  Many drafts that
> exist between other WG's are given an opportunity to be reviewed by
> other WG's in order to make sure the appropriate expertise is able to
> evaluate it.   For example, the ICE draft developed in the MMUSIC WG
has
> been reviewed several times in the SIPPING and/or SIP WG's.  Although
it
> is an MMUSIC WG item, it has required expertise review in these other
> WG's.  I see a similar process occurring with this proposed WG.  The
> metrics drafts will be introduced, developed, and reviewed in this
> draft; however, along the way, they need to be presented in other WG's
> such as AVT and SIPPING in order to gain the additional expertise
> perspective.
>
> > On Thu, 2007-08-02 at 11:26 -0400, Alan Clark wrote:
> > for example it may make sense for the current work
> > on voice/video/audio metrics to stay within AVT due to the domain
knowledge
> > required. =20

-- Loki=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: RE: [PMOL] Should establish working group...
From: Daryl Malas <[email-masked]>
To: Al Morton <[email-masked]>
Date: Fri, 03 Aug 2007 12:56:55 -0600
Cc: [email-masked]

This clarification is helpful.  I am okay with the idea of establishing
a WG with a Framework objective and also identify an existing set of
drafts as determined by the group to move forward on in the initial
charter.  Once that concludes, I think it will become very apparent
whether or not the value of the WG extends forward or ceases to exist at
that point.  Even if the WG concludes, I think a Directorate at that
point should be an option in case future performance type work warrants
it.

--Daryl

On Thu, 2007-08-02 at 11:05 -0400, Al Morton wrote:
> Responding to Dan's question for clarification,
> and Bob's response...
> 
> At 10:44 AM 8/2/2007, Natale, Bob wrote:
> >
> >I favor a "permanent" WG for this work -- i.e., for the lifetime of the
> >milestones (to be) identified in the charter.
> >
> >Cheers,
> >BobN
> 
> I think the main difference between a long-lived and short-lived
> Working Group is the number and breadth of milestones on the charter
> when the WG is created.  Other factors influence the actual lifetime,
> such as the ability to complete the initial set of milestones.
> 
> I like the idea, suggested by Leslie Nagle at the mic, of a
> Working Group that prepares a "How to" Framework and a small
> number of Performance Metrics RFCs (~2). A successful outcome of this
> work would be the proof that this is an area of interest in IETF,
> and the charter could then be revisited/expanded.
> 
> willing to start small,
> Al
> 
> 
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: Re: [PMOL] Should establish working group
From: Daryl Malas <[email-masked]>
To: Alan Clark <[email-masked]>
Date: Fri, 03 Aug 2007 12:52:31 -0600
Cc: [email-masked]

I'm not sure I agree with keeping your drafts in AVT.  I have not read
your drafts yet, but if they are "performance metrics" based drafts;
they should be in this new WG.  As with my SIP metrics draft, the
SIPPING WG has provided a good forum to receive feedback; however, the
draft is not truly within the charter of the group.  Many drafts that
exist between other WG's are given an opportunity to be reviewed by
other WG's in order to make sure the appropriate expertise is able to
evaluate it.   For example, the ICE draft developed in the MMUSIC WG has
been reviewed several times in the SIPPING and/or SIP WG's.  Although it
is an MMUSIC WG item, it has required expertise review in these other
WG's.  I see a similar process occurring with this proposed WG.  The
metrics drafts will be introduced, developed, and reviewed in this
draft; however, along the way, they need to be presented in other WG's
such as AVT and SIPPING in order to gain the additional expertise
perspective.

--Daryl

On Thu, 2007-08-02 at 11:26 -0400, Alan Clark wrote:
>  My opinion is that a new working group should be established.  
> 
> Initial work would be a framework RFC, potentially Daryl Malas' draft and
> other items. Work would only be transferred from other WG's with the
> agreement of both the WG and Area Directors (key criteria would be that the
> required expertise lay within the APM/PMOL WG rather than the WG currently
> progressing the draft); for example it may make sense for the current work
> on voice/video/audio metrics to stay within AVT due to the domain knowledge
> required.  
> 
> The issue of whether this is long-lived vs short-lived would appear to be a
> charter question - as WG's generally expire when they meet their charter
> (?).  If the charter initially had two deliverables and the WG was able to
> review new I-D's and request amendments to the charter to address
> appropriate new work items -  then presumably the WG would either complete
> its work within a short timeframe or the charter would be extended and it
> would continue for a longer period?
> 
> Alan
> 
> 
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 03 Aug 2007 19:43:57 +0200
From: dimitri papadimitriou <[email-masked]>
To: Al Morton <[email-masked]>
Subject: Re: [PMOL] Should establish working group...
Cc: [email-masked]
Reply-To: [email-masked], [email-masked]

al

- i support this idea - this will serve as proof case.

noticing however that the milestones/timing would constraint the initial 
work (indeed, it could some important topics would require a longer 
duration then what could be reasonably considered)

-d.

Al Morton wrote:
> Responding to Dan's question for clarification,
> and Bob's response...
> 
> At 10:44 AM 8/2/2007, Natale, Bob wrote:
>>
>> I favor a "permanent" WG for this work -- i.e., for the lifetime of the
>> milestones (to be) identified in the charter.
>>
>> Cheers,
>> BobN
> 
> I think the main difference between a long-lived and short-lived
> Working Group is the number and breadth of milestones on the charter
> when the WG is created.  Other factors influence the actual lifetime,
> such as the ability to complete the initial set of milestones.
> 
> I like the idea, suggested by Leslie Nagle at the mic, of a
> Working Group that prepares a "How to" Framework and a small
> number of Performance Metrics RFCs (~2). A successful outcome of this
> work would be the proof that this is an area of interest in IETF,
> and the charter could then be revisited/expanded.
> 
> willing to start small,
> Al
> 
> 
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol
> 
> .
> 

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Fri, 03 Aug 2007 07:34:30 +0200
From: Henk Uijterwaal <[email-masked]>
To: Al Morton <[email-masked]>
Subject: Re: [PMOL] Should establish working group...
Cc: [email-masked]

All,

> I like the idea, suggested by Leslie Nagle at the mic, of a
> Working Group that prepares a "How to" Framework and a small
> number of Performance Metrics RFCs (~2). A successful outcome of this
> work would be the proof that this is an area of interest in IETF,
> and the charter could then be revisited/expanded.

I fully agree with this proposal.

Henk

-- 
------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.amsterdamned.org/~henk
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

# Lawyer: "Now sir, I'm sure you are an intelligent and honest man--"
# Witness: "Thank you. If I weren't under oath, I'd return the compliment."

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Thu, 2 Aug 2007 16:33:07 +0000 (GMT)
From: Mehmet Ersue <[email-masked]>
Subject: RE: [PMOL] Should establish working group
To: [email-masked]

--===============1901092701==
Content-Type: multipart/alternative; boundary="0-458994479-1186072387=:36480"

--0-458994479-1186072387=:36480
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

=0AA WG runs as long as it developes WG items, then it gets =0Arechartered =
or conludes. A directorate could be long-living =0Abut I believe a director=
ate wouldn't be very useful today. =0A=0AAs Al says, let us start small and=
 define the framework first =0A(metric types, test cases, BCP, etc.). =0A=
=0AAs somebody stated in the BOF session we need to use also =0Athe work wh=
ich already exists (e.g. RAQMON).=0A=0ARegards, =0AMehmet =0A=0A=0A> -----O=
riginal Message-----=0A> From: ext Alan Clark [mailto:alan.d.clark@telchemy=
.com] =0A> Sent: Thursday, August 02, 2007 5:27 PM=0A> To: [email-masked]=0A=
> Subject: [PMOL] Should establish working group=0A> =0A>  =0A> My opinion =
is that a new working group should be established.  =0A> =0A> Initial work =
would be a framework RFC, potentially Daryl =0A> Malas' draft and=0A> other=
 items. Work would only be transferred from other WG's with the=0A> agreeme=
nt of both the WG and Area Directors (key criteria =0A> would be that the=
=0A> required expertise lay within the APM/PMOL WG rather than the =0A> WG =
currently=0A> progressing the draft); for example it may make sense for the=
 =0A> current work=0A> on voice/video/audio metrics to stay within AVT due =
to the =0A> domain knowledge=0A> required.  =0A> =0A> The issue of whether =
this is long-lived vs short-lived would =0A> appear to be a=0A> charter que=
stion - as WG's generally expire when they meet =0A> their charter=0A> (?).=
  If the charter initially had two deliverables and the =0A> WG was able to=
=0A> review new I-D's and request amendments to the charter to address=0A> =
appropriate new work items -  then presumably the WG would =0A> either comp=
lete=0A> its work within a short timeframe or the charter would be =0A> ext=
ended and it=0A> would continue for a longer period?=0A> =0A> Alan=0A> =0A>=
 =0A> _______________________________________________=0A> PMOL mailing list=
=0A> [email-masked]=0A> https://www1.ietf.org/mailman/listinfo/pmol=0A> =0A=
=0A=0A=0A=0A      __________________________________  Die etwas anderen Inf=
os rund um das Thema Reisen. BE A BETTER WELTENBUMMLER!  www.yahoo.de/cleve=
r
--0-458994479-1186072387=:36480
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

<html><head><style type=3D"text/css"><!-- DIV {margin:0px;} --></style></he=
ad><body><div style=3D"font-family:arial,helvetica,sans-serif;font-size:10p=
t"><div><br>A WG runs as long as it developes WG items, then it gets <br>re=
chartered or conludes. A directorate could be long-living <br>but I believe=
 a directorate wouldn't be very useful today. <br><br>As Al says, let us st=
art small and define the framework first <br>(metric types, test cases, BCP=
, etc.). <br><br>As somebody stated in the BOF session we need to use also =
<br>the work which already exists (e.g. RAQMON).<br><br>Regards, <br>Mehmet=
 <br><br><br>&gt; -----Original Message-----<br>&gt; From: ext Alan Clark [=
mailto:[email-masked]] <br>&gt; Sent: Thursday, August 02, 2007 =
5:27 PM<br>&gt; To: [email-masked]<br>&gt; Subject: [PMOL] Should establish =
working group<br>&gt; <br>&gt;&nbsp; <br>&gt; My opinion is that a new work=
ing group should be established.&nbsp; <br>&gt; <br>&gt; Initial work
 would be a framework RFC, potentially Daryl <br>&gt; Malas' draft and<br>&=
gt; other items. Work would only be transferred from other WG's with the<br=
>&gt; agreement of both the WG and Area Directors (key criteria <br>&gt; wo=
uld be that the<br>&gt; required expertise lay within the APM/PMOL WG rathe=
r than the <br>&gt; WG currently<br>&gt; progressing the draft); for exampl=
e it may make sense for the <br>&gt; current work<br>&gt; on voice/video/au=
dio metrics to stay within AVT due to the <br>&gt; domain knowledge<br>&gt;=
 required.&nbsp; <br>&gt; <br>&gt; The issue of whether this is long-lived =
vs short-lived would <br>&gt; appear to be a<br>&gt; charter question - as =
WG's generally expire when they meet <br>&gt; their charter<br>&gt; (?).&nb=
sp; If the charter initially had two deliverables and the <br>&gt; WG was a=
ble to<br>&gt; review new I-D's and request amendments to the charter to ad=
dress<br>&gt; appropriate new work items -&nbsp; then presumably the
 WG would <br>&gt; either complete<br>&gt; its work within a short timefram=
e or the charter would be <br>&gt; extended and it<br>&gt; would continue f=
or a longer period?<br>&gt; <br>&gt; Alan<br>&gt; <br>&gt; <br>&gt; _______=
________________________________________<br>&gt; PMOL mailing list<br>&gt; =
[email-masked]<br><span>&gt; <a target=3D"_blank" href=3D"https://www1.ietf.=
org/mailman/listinfo/pmol">https://www1.ietf.org/mailman/listinfo/pmol</a><=
/span><br>&gt; <br></div></div><br>=0A      <hr size=3D1> =0AAlles was der =
Gesundheit und Entspannung dient.=0A<a href=3D"http://de.rd.yahoo.com/evt=
=3D48734/*http://de.promotions.yahoo.com/clever/be-a-better/medizinmann.htm=
l" target=3D_new >=0A<b>BE A BETTER MEDIZINMANN!</b></a></body></html>
--0-458994479-1186072387=:36480--

--===============1901092701==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

--===============1901092701==--

Date: Thu, 02 Aug 2007 12:23:38 -0400
From: "Lu, Hui-Lan (Huilan)" <[email-masked]>
Organization: Alcatel Lucent
To: Al Morton <[email-masked]>
Subject: Re: [PMOL] Should establish working group...
Cc: [email-masked]

Al Morton wrote:
> I like the idea, suggested by Leslie Nagle at the mic, of a
> Working Group that prepares a "How to" Framework and a small
> number of Performance Metrics RFCs (~2). A successful outcome of this
> work would be the proof that this is an area of interest in IETF,
> and the charter could then be revisited/expanded.

I like the idea too.

Hui-Lan

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-Class: urn:content-classes:message
Importance: normal
Priority: normal
Date: Thu, 2 Aug 2007 16:50:40 +0100
From: "Ravi Raviraj" <[email-masked]>
To: <[email-masked]>
Subject: [PMOL] What will we be doing?

I do favor a separate WG. However, it would be helpful to understand the
kind of work we will be doing in that group.
I have heard of at least two different versions of it.
Any chance of some sort of early wording for charter?

Ravi.

This email and attachment are confidential. If you have received this =
message in error, please notify the sender immediately and delete the =
message from your computer. Psytechnics Ltd has taken steps to ensure =
that this email is virus free but the recipient should take necessary =
steps to make certain.

Company Name : Psytechnics Ltd
Registered Number : 4110724
Registered Office : Fraser House, 23 Museum St, Ipswich, Suffolk, IP1 =
1HN, UK
Registration : England and Wales

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

From: "Alan Clark" <[email-masked]>
To: <[email-masked]>
Subject: [PMOL] Should establish working group
Date: Thu, 2 Aug 2007 11:26:57 -0400

My opinion is that a new working group should be established.  

Initial work would be a framework RFC, potentially Daryl Malas' draft and
other items. Work would only be transferred from other WG's with the
agreement of both the WG and Area Directors (key criteria would be that the
required expertise lay within the APM/PMOL WG rather than the WG currently
progressing the draft); for example it may make sense for the current work
on voice/video/audio metrics to stay within AVT due to the domain knowledge
required.  

The issue of whether this is long-lived vs short-lived would appear to be a
charter question - as WG's generally expire when they meet their charter
(?).  If the charter initially had two deliverables and the WG was able to
review new I-D's and request amendments to the charter to address
appropriate new work items -  then presumably the WG would either complete
its work within a short timeframe or the charter would be extended and it
would continue for a longer period?

Alan

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Should establish working group...
Date: Thu, 2 Aug 2007 11:16:51 -0400
From: "Natale, Bob" <[email-masked]>
To: <[email-masked]>

=20
Al's approach is fine with me (and I appreciate the clarification of
what differentiates long- and short-lived WGs) -- mileage may vary for
individuals depending upon the target content of the selected "small
number of Performance Metrics RFCs".

Cheers,
BobN
-----Original Message-----
From: Al Morton [mailto:[email-masked]]=20
Sent: Thursday, August 02, 2007 11:06 AM
To: [email-masked]
Subject: RE: [PMOL] Should establish working group...

Responding to Dan's question for clarification,
and Bob's response...

At 10:44 AM 8/2/2007, Natale, Bob wrote:
>
>I favor a "permanent" WG for this work -- i.e., for the lifetime of
the
>milestones (to be) identified in the charter.
>
>Cheers,
>BobN

I think the main difference between a long-lived and short-lived
Working Group is the number and breadth of milestones on the charter
when the WG is created.  Other factors influence the actual lifetime,
such as the ability to complete the initial set of milestones.

I like the idea, suggested by Leslie Nagle at the mic, of a
Working Group that prepares a "How to" Framework and a small
number of Performance Metrics RFCs (~2). A successful outcome of this
work would be the proof that this is an area of interest in IETF,
and the charter could then be revisited/expanded.

willing to start small,
Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Date: Thu, 02 Aug 2007 11:05:43 -0400
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: RE: [PMOL] Should establish working group...

Responding to Dan's question for clarification,
and Bob's response...

At 10:44 AM 8/2/2007, Natale, Bob wrote:
>
>I favor a "permanent" WG for this work -- i.e., for the lifetime of the
>milestones (to be) identified in the charter.
>
>Cheers,
>BobN

I think the main difference between a long-lived and short-lived
Working Group is the number and breadth of milestones on the charter
when the WG is created.  Other factors influence the actual lifetime,
such as the ability to complete the initial set of milestones.

I like the idea, suggested by Leslie Nagle at the mic, of a
Working Group that prepares a "How to" Framework and a small
number of Performance Metrics RFCs (~2). A successful outcome of this
work would be the proof that this is an area of interest in IETF,
and the charter could then be revisited/expanded.

willing to start small,
Al

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Should establish working group...
Date: Thu, 2 Aug 2007 10:44:38 -0400
From: "Natale, Bob" <[email-masked]>
To: <[email-masked]>

=20
I favor a "permanent" WG for this work -- i.e., for the lifetime of the
milestones (to be) identified in the charter.

Cheers,
BobN
-----Original Message-----
From: Romascanu, Dan (Dan) [mailto:[email-masked]]=20
Sent: Thursday, August 02, 2007 10:32 AM
To: Daryl Malas; [email-masked]
Subject: RE: [PMOL] Should establish working group...

Thanks Daryl for being the first to trigger the discussions on the new
list.=20

If possible I would suggest you and other folks who may express support
for the formation of a working group to also make clear which of the
two
options discussed in the BOF you would support:=20
- permanent WG for definition of Performance Metrics at Other Layers,
modeled on the IPPM WG=20
- short-lived WG to define a framework ('how to start, write, and
review
PMOL documents in the IETF') document and a couple of test cases

Also, it would be very useful if you can make a short history of your
tribulations with draft-malas-performance-metrics, maybe in a separate
mail. Many participants in the list may not be aware about the sinuous
and yet uncompleted history of this document which was one of the test
cases that let to the BOF.=20

Dan

=20
=20

> -----Original Message-----
> From: Daryl Malas [mailto:[email-masked]]=20
> Sent: Thursday, August 02, 2007 4:57 PM
> To: [email-masked]
> Subject: [PMOL] Should establish working group...
>=20
> Okay, after taking a break from the BOF and talking with=20
> others about this.  I have become further convinced a working=20
> group should be established to handle this type of work.  If=20
> we only move forward with a Directorate, the work will have a=20
> very high unlikelihood of ever making the appropriate agenda=20
> time in the existing working groups to be fully developed and=20
> prioritized.
>=20
> In the BOF, it was overwhelmingly decided this type of work=20
> is absolutely valuable to the members of the IETF. =20
> Furthermore, it was overwhelmingly decided many will commit=20
> their time to this work.  Those two decisions indicate this=20
> work will receive the necessary review, input, and expertise=20
> necessary to make it valuable to the members of the IETF and=20
> the industry as a whole.
>=20
> Summary...establish a working group for APM, PMOL, or=20
> whatever we want to call it.
>=20
> My two cents...
>=20
> --Daryl
>=20
>=20
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol
>=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Subject: RE: [PMOL] Should establish working group...
Date: Thu, 2 Aug 2007 16:31:43 +0200
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: "Daryl Malas" <[email-masked]>, <[email-masked]>
Cc: 

Thanks Daryl for being the first to trigger the discussions on the new
list.=20

If possible I would suggest you and other folks who may express support
for the formation of a working group to also make clear which of the two
options discussed in the BOF you would support:=20
- permanent WG for definition of Performance Metrics at Other Layers,
modeled on the IPPM WG=20
- short-lived WG to define a framework ('how to start, write, and review
PMOL documents in the IETF') document and a couple of test cases

Also, it would be very useful if you can make a short history of your
tribulations with draft-malas-performance-metrics, maybe in a separate
mail. Many participants in the list may not be aware about the sinuous
and yet uncompleted history of this document which was one of the test
cases that let to the BOF.=20

Dan

=20
=20

> -----Original Message-----
> From: Daryl Malas [mailto:[email-masked]]=20
> Sent: Thursday, August 02, 2007 4:57 PM
> To: [email-masked]
> Subject: [PMOL] Should establish working group...
>=20
> Okay, after taking a break from the BOF and talking with=20
> others about this.  I have become further convinced a working=20
> group should be established to handle this type of work.  If=20
> we only move forward with a Directorate, the work will have a=20
> very high unlikelihood of ever making the appropriate agenda=20
> time in the existing working groups to be fully developed and=20
> prioritized.
>=20
> In the BOF, it was overwhelmingly decided this type of work=20
> is absolutely valuable to the members of the IETF. =20
> Furthermore, it was overwhelmingly decided many will commit=20
> their time to this work.  Those two decisions indicate this=20
> work will receive the necessary review, input, and expertise=20
> necessary to make it valuable to the members of the IETF and=20
> the industry as a whole.
>=20
> Summary...establish a working group for APM, PMOL, or=20
> whatever we want to call it.
>=20
> My two cents...
>=20
> --Daryl
>=20
>=20
> _______________________________________________
> PMOL mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pmol
>=20

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

Subject: [PMOL] Should establish working group...
From: Daryl Malas <[email-masked]>
To: [email-masked]
Date: Thu, 02 Aug 2007 07:57:14 -0600

Okay, after taking a break from the BOF and talking with others about
this.  I have become further convinced a working group should be
established to handle this type of work.  If we only move forward with a
Directorate, the work will have a very high unlikelihood of ever making
the appropriate agenda time in the existing working groups to be fully
developed and prioritized.

In the BOF, it was overwhelmingly decided this type of work is
absolutely valuable to the members of the IETF.  Furthermore, it was
overwhelmingly decided many will commit their time to this work.  Those
two decisions indicate this work will receive the necessary review,
input, and expertise necessary to make it valuable to the members of the
IETF and the industry as a whole.

Summary...establish a working group for APM, PMOL, or whatever we want
to call it.

My two cents...

--Daryl

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol