Subject: Quality of document review in RTG WGs

From: Alex Zinin <[email-masked]>
Date: Mon, 24 Feb 2003 22:45:30 -0800

Guys,

 I'm not sure if I'm the only one who noticed this problem,
 or even if the problem is real or not, but it seems that
 the review process at least in our area is not working
 really great. Before a WG LC is announced, people generally
 don't have time to review stuff or assume that "folks will
 take care of this", and when the WG LC is started, many think,
 that it's really not worth it, as the document is pretty much
 ready and unlikely to change, etc...

 Seems that we need to figure out some mechanism to encourage
 more diligent review throughout the process. Does anyone
 have any ideas on this topic? Some from me:

  o pull together a group of reviewers (volunteers) in each WG
    and make sure that WG chairs kick them periodically to
    review stuff in progress

  o use something like "Call for Review" instead of the "Last
    Call" while the doc is in progress

  o Have WG chairs bring docs to this directorate periodically
    AND when the LC is started.

 Thoughts?

-- 
Alex

---

From: mike shand <[email-masked]>
Date: Tue, 25 Feb 2003 10:08:30 +0000

Alex,
One COULD argue that if the drafts were sufficiently interesting, folks 
would review them without prompting. By "interesting" I mean something 
which it is in my interest to make sure it is right. I think there is no 
shortage of reviewers for controversial drafts.

That said (and I don't think there is anything we can do about that 
perception), there is always more incentive to review something if someone 
asks you to, so I think the idea of a review group MIGHT work. The 
downside, of course, is that it might encourage even more ambivalence from 
everyone else.

Maybe even something as simple as a kick from the WG chairs to everyone, 
saying "please review" would help.

         Mike

---

From: Acee Lindem <[email-masked]>
Date: Tue, 25 Feb 2003 09:08:36 -0500

> shortage of reviewers for controversial drafts.

Mike, Alex,

I also notice that new drafts are reviewed much more readily than drafts
progressing through the process. By WG last call the review seems to
be confined to WG chairs and ADs. I'm not sure what to do about this.

Thanks,
Acee

> perception), there is always more incentive to review something if 
> someone asks you to, so I think the idea of a review group MIGHT work. 
> The downside, of course, is that it might encourage even more 
> ambivalence from everyone else.
> 

-- 
Acee

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 25 Feb 2003 12:08:55 -0500

I'd have to agree to some extent with Mike.  I'd also offer
a slight extension on another viewpoint expressed:

From the relatively little time I've spent in IETF, people seem willing
to look at stuff as it initially comes out.  Once the document has
reached a certain point where most of the gross details are worked
out, the finer details are usually left behind for the small number
of people interested in them.

This seems to roughly correlate to, "We have rough consensus. We've
made *our* implementations work and interoperate.  Just declare
victory and put our efforts into the Next Great Thing."  Its only
the people who didn't go through the pain of the process of doing an initial
implementation that has to suffer if the spec is sloppy.  You have
to deduce the details.

The BGP-4 spec is a perfect example of this viewpoint.

One could more cynically argue that parties who have completed their
work have a vested interest in not documenting the "secret mix" that
makes things work and interoperate - it serves as a barrier to
entry in the market.

Part of my motiviation in becoming active in IETF was coming from
operating a small network and dealing with things that *didn't* work
and some of the problems being due to sloppy specs and poor implmenetations
of them.  Vijay Gill often makes many of the same points I'd like
to make, but he does it far better and with a much sharper tongue.

I will also admit that the finishing steps of getting something through
the process is not the interesting work.  I've been through so many
versions of the BGP FSM that I want to be finished with things.  Only
my overly anal personality keeps me from declaring it "done" until
it really is.

I'll ask a different question:  Compared to the routing area, how
active are the other areas?  Just keeping up with BGP specs is hard.
As part of taking part in the directorate, I've tried to broaden
myself a bit more than I used to, but its still a lot of stuff.

-- 
Jeff Haas 
NextHop Technologies