Subject: Comments solicited:draft-zinin-early-review-00.txt

From: Alex Zinin <[email-masked]>
Date: Sat, 1 Nov 2003 13:10:40 -0800

Folks-

 I'd appreciate your comments on this draft. It is an attempt to come
 up with a mechanism that could be used by the WG chairs and ADs for
 early-stage (pre-IESG, pre-IETF-LC) cross-area review of the
 documents. It would also allow individual ADs to off-load some (or
 all) of the document review if needed.
 
-- 
Alex
http://www.psg.com/~zinin/

This is a forwarded message
From: [email-masked] <[email-masked]>
To: 
Cc: [email-masked]
Date: Monday, October 27, 2003, 1:52:10 PM
Subject: I-D ACTION:draft-zinin-early-review-00.txt

===8<==============Original message text===============
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Engineering Steering Group Working Group of the IETF.

        Title           : Area Review Teams for Early Cross-functional Reviews
        Author(s)       : A. Zinin
        Filename        : draft-zinin-early-review-00.txt
        Pages           : 11
        Date            : 2003-10-27
        
This document contains a proposal for cross-functional IETF review process that can be initiated at early stages of a document life cycle. The approach is based on existing experience with area directorates and other expert groups within the IETF.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-zinin-early-review-00.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
        "get draft-zinin-early-review-00.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
        [email-masked].
In the body type:
        "FILE /internet-drafts/draft-zinin-early-review-00.txt".
        
NOTE:   The mail server at ietf.org can return the document in
        MIME-encoded form by using the "mpack" utility.  To use this
        feature, insert the command "ENCODING mime" before the "FILE"
        command.  To decode the response(s), you will need "munpack" or
        a MIME-compliant mail reader.  Different MIME-compliant mail readers
        exhibit different behavior, especially when dealing with
        "multipart" MIME messages (i.e. documents which have been split
        up into multiple messages), so check your local documentation on
        how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

===8<===========End of original message text===========

ENCODING mime
FILE /internet-drafts/draft-zinin-early-review-00.txt

---

From: "Adrian Farrel" <[email-masked]>
Date: Sun, 2 Nov 2003 01:21:51 -0000

Alex,

Good start.
===
I think you are being too sensitive in trying to protect people's feelings by saying that
they mustn't think that the ART review comments are handed down from on high. The need to
track these comments out-weighs this.

Just as the ID-tracker allows us all to see the AD and IESG thoughts and comments, so we
need a mechanism to expose and track the ART comments. Where the review takes place early
in the life-cycle, the comments need to be exposed to the WG mail list for proper
discussion.

I would suggest that ART review comments to the WG mail list follow some form of
pro-forma.
- The subject always reads "ART review: draft-ietf-xxxx"
- The content always begins with a piece of set text.
   Just a couple of lines to say
   - what the ART is
   - who ART review was requested by
   - that comments are intended to be treated as normal
      input to the WG discussion.
===
You are right to be concerned about "accountability". Stacking this up behind AD
accountability is probably safe. I don't see why there would be more complaints about ART
members than there are about WG chairs.
===
One of the growing concerns is opening up the IETF process to participants who are not
fluent in English. it is clearly beneficial for the review process if drafts are well
written and comprehensible.

Is there scope for offering the services of an ERT whose purpose is solely to help bash
the text? I can't think who would have the time or inclination to do this. Maybe it is
something to hand off to Harald.
===
I would also like to see more attention given to getting cross-WG reviews. We should not
assume that all wisdom is embodied in the "expert review teams" (although this may
actually be the case). In some cases, cross-area-WG review is required.

For example, TEWG is working on multi-region TE requirements, but this may be lost in the
noise of the discussion of various DiffServ models. The multi-area requirements need
participation from those in the routing and signaling WGs that will be called upon to
generate solutions.

Maybe this is orthogonal, but I needed to say it. I feel better now :-)

Cheers,
Adrian

---

From: Alex Zinin <[email-masked]>
Date: Mon, 3 Nov 2003 17:21:40 -0800

Adrian,

Thanks a lot for reading the doc. See inline below, please.

> track these comments out-weighs this.

Aren't these two independent? I.e., comment tracking should work
equally good regardless of the status of the comments.

The reason I was being sensitive about spelling out that the reviews
are treated as normal discussion is because I didn't want the
day-to-day ART review process to override the day-to-day process
within a WG. On the other hand, if an AD receives a recommendation
from the ART saying something like "problem <foo> was pointed out, but
the WG decided to ignore it", and the AD believes that foo is
critical, [s]he may decide to put this in the AD-review and push hard
for it.

> in the life-cycle, the comments need to be exposed to the WG mail list for proper

Agreed. I am hoping we will be able to use the WG-internal issue
tracking tools and/or the current IESG ID tracker.

> - The subject always reads "ART review: draft-ietf-xxxx"

make sense

>       input to the WG discussion.

or just a piece of text referring to the final document.

I think I should include an appendix that would talk about such
procedural bits like how e-mails should look and where they should
be sent.

> members than there are about WG chairs.

OK

> something to hand off to Harald.

Sounds like an early-stage RFC-editor-like team.

Checking the ID nits should be part of the WG chair review process, as
well as part of the ART/AD review. Purely linguistical review would
likely piss off people if we introduced it as yet another required
step, but would probably be OK if done as part of technical review,
which is the way it happens now.

> actually be the case). In some cases, cross-area-WG review is required.

Agree. I'll think how this can be improved. My first thought is to cc:
the ART review requests and LC's to the hosting Area mailing list (or
a special area-review mailing list).

> participation from those in the routing and signaling WGs that will be called upon to

Agree

> Maybe this is orthogonal, but I needed to say it. I feel better now :-)

No, this is very relevant.

Thanks again.

Alex

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 5 Nov 2003 11:19:15 -0500

Alex,

Replies to this and also some comments of my own.

> > members than there are about WG chairs.

I think this somewhat speaks to the "status" of people on the ARTs.
Your document recommends assigning perhaps more "status" (sticker, 
note in the RFC, etc) than perhaps is necessary.

The purpose of the ARTs, IMO, is to let the function of the ADs scale.
The function of the ADs after all is to try to keep things a reasonably
high quality in the process.

Mentioning the ARTs in the final document in the usual "Acknowledgements"
section is fine, if the authors are willing, but institutionalizing it
seems a bit excessive.

I suspect my thoughts aren't quite well formed enough to go into more
detail yet, so I'll stop here. 

> which is the way it happens now.

I would prefer that linguistic/grammatical review be kept as generally
separate from technical review where possible.  However, I'd prefer
the linguistic/grammatical cleanup be done before technical review
is requested.  

Reading technical documents is hard enough.  Reading them when
things are grammatically unclean biases the bit of your brain that
catches logical inconsistencies.

By keeping the two elements (linguistic vs. technical) separate, we
can hopefully help the document authors produce better documents while
not appearing to bash the ideas they're trying to convey in the document.
That said, document authors should be encouraged to spell check and
grammmar check their documents before submission.  Just because we're
not something formal like an academic organization or a government
standards body doesn't mean we should expect low-quality dox to be the
norm.

> a special area-review mailing list).

I think it may be possible for us to trigger interest in particular 
documents on a greater IETF-wide basis.

One of the problems I have with IETF documents, whether it be IDs
or published RFCs, is telling how the document is meant to be relevant.
For example, if there are a chain of documents mean to deal with SMTP,
why don't we have a SMTP tag assigned to the document that tells
us that it is relevant?  To an area, like routing?  To a protocol, like
OSPF?

Sometimes this is obvious in the abstract or the draft name, but
sometimes it isn't.

By creating a standard namespace by which document authors can tag
their documents and a methdology by which we can denote this within
our documents or indices (rfc-index.txt, e.g.), we can allow people
that might say "oh, this is mpls related" and ignore something to go
"oh!  This was really about ospf!"

It seems some librarian skills would come in handy for this.

-- 
Jeff Haas 
NextHop Technologies