From jari.arkko@piuha.net Wed Nov 13 05:26:17 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0905311E819A; Wed, 13 Nov 2013 05:26:17 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -102.586 X-Spam-Level: X-Spam-Status: No, score=-102.586 tagged_above=-999 required=5 tests=[AWL=0.013, BAYES_00=-2.599, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cCD1DBEbCsJI; Wed, 13 Nov 2013 05:26:12 -0800 (PST) Received: from p130.piuha.net (p130.piuha.net [193.234.218.130]) by ietfa.amsl.com (Postfix) with ESMTP id 5965311E8199; Wed, 13 Nov 2013 05:26:12 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by p130.piuha.net (Postfix) with ESMTP id B86F52CCD0; Wed, 13 Nov 2013 15:26:11 +0200 (EET) X-Virus-Scanned: amavisd-new at piuha.net Received: from p130.piuha.net ([127.0.0.1]) by localhost (p130.piuha.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AABDiUX5ZrTi; Wed, 13 Nov 2013 15:26:10 +0200 (EET) Received: from [127.0.0.1] (p130.piuha.net [IPv6:2a00:1d50:2::130]) by p130.piuha.net (Postfix) with ESMTP id 81B082CC48; Wed, 13 Nov 2013 15:26:10 +0200 (EET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) From: Jari Arkko In-Reply-To: <076D38B1-EBDB-44C1-8013-97FA31EAF24B@piuha.net> Date: Wed, 13 Nov 2013 15:26:10 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <076D38B1-EBDB-44C1-8013-97FA31EAF24B@piuha.net> To: Working Group Chairs X-Mailer: Apple Mail (2.1510) Cc: "dir-coord@ietf.org" Subject: [dir-coord] Reminder: asking for early reviews X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Nov 2013 13:26:17 -0000 I talked to the Gen-ART team in Vancouver, and in the discussion it came = up that we have not gotten too many requests for early reviews. I wanted = to remind you chairs that you can ask for early reviews. Hopefully this = will make it easier for you to deal with any issues that might arise = (easier than having to deal with them when the reviews are raised at the = IESG stage). In addition, we need more experience with this practice to = find out if the experiment worked! So feel free to send mail to dir-coord@ietf.org if you have a document = that is coming up for an AD/IESG submission. You could ask for a review = in parallel while your AD is doing the AD review, for instance. Jari > Internet Drafts sent for approval as RFCs are reviewed by individuals = during the IETF Last Call, the Area Directors, IANA, as well as a number = of volunteers from various directorates and review teams. The reviews = from these teams has gained a significant role in ensuring that the IETF = produces high-quality, understandable and implementable RFCs. >=20 > Yet, as discussed in = http://www.ietf.org/blog/2013/05/balancing-the-process/ we have a = general problem that quite a lot of the work around IETF documents = happens at the end of the process. In particular, a number of the = reviews during IETF last call point out issues that end up being raised = by the IESG as comments. It is of course good that issues are caught, = but raising them earlier would be better. And it would be better if the = working groups - the intended focus point of work on a topic - would get = to handle them, as opposed to raising these issues with the IESG. The = IESG discussed these issues in its May 2013 retreat, and decided to = experiment with three actions designed to move more work to the = responsibility of the working groups: >=20 > (1) Perform some reviews that are now happening at IETF Last Call a = bit earlier. This will put the working group in a bigger role in = resolving cross-area and general issues. >=20 > (2) Invite document shepherds on IESG telechats when there's a = document that is likely to require discussion. This will make it = possible for the document shepherd to me directly be=20 >=20 > (3) When a document has a number of issues, hand over the process back = to the working group, as opposed to the IESG tracking the issues. Among = other things, this will ensure that changes are discussed in an open = working group list and agreed through consensus. >=20 > Some of you have seen (2) and (3) happen, more will come. For (1), = building quality and cross-area review to the process earlier is of = course a big effort. We plan to launch an experiment to make a small = change to current directorate review procedures to learn if we can move = reviews a little bit earlier. If successful, this experiment will enable = working groups to deal with issues before IETF Last Call and IESG review = and empower the working groups to be in charge of the documents = throughout their life cycle. We are also hoping that document quality = will improve and number of issues discussed in the IESG will be lower. >=20 > While the number of reviews as such is not changed, some additional = effort and care will however be required from the reviewers, directorate = coordinators/secretaries, the working group chairs, and other = participants. The experiment will show us whether this effort is = reasonable and if there are any unexpected effects. The experiment is = performed on a voluntary basis by each directorate. As early review = requests come in, the directorates can throttle workload by either = processing the requests or reverting to the existing procedures. = Initially, the Gen-ART, Security Directorate, and Applications Area = Directorate are included in this effort. Other directorates may be added = later. The experiment will be reviewed after six months. >=20 > Care must be taken to avoid a number of possible drawbacks. There may = be problematic documents that would require much re-review and effort. = Similarly, working groups often perform a number of working group last = calls on a document, and it would be undesirable to engage the reviewer = before the document was really ready to be sent forward. And when = reviewers send comments, it is important that the group listens to the = comments in the right way, like you would for a review from a security = expert or a general networking expert. E-mail practices around sending = and responding to comments have to be carefully managed, as the outside = reviewers are typically not list members. >=20 > For the working group chairs, you can request a review by sending mail = to dir-coord@ietf.org, indicating the name of the document. You can ask = for a review as soon as a working group last call has ended successfully = and the document edits are done. Initially, it would make sense to ask = for reviews on documents that you feel are likely the most stable ones. = To avoid congesting the reviewer resources, ask for this service only = for some documents, not as a wholesale service on every document you = submit for publication.=20 >=20 > You can ask for the review in parallel with ongoing AD reviews, = filling out the shepherd questionnaire, etc. The hope is that reviews = can come in in the same time frame as other tasks in this stage, and = that you can take the comments into account and revise the document = before finally sending it out for IETF Last Call. Reviews will be sent = to you and potentially the working group mailing list. You may need to = forward comments and/or approve new posters to the list. Discussion on = the mailing list is encouraged, but please try to ensure that the = reviewer is kept on the Cc line, as he or she may not be on the list = itself. Treat the reviews with respect and keep it in mind that outside = experts may have opinions that need to be taken into account, even if = the working group had not considered those aspects before. Mediate and = monitor the discussion actively. Try to keep the reviewer out of e-mail = storms and work out solutions separately and then engage the reviewer = again. >=20 > For the directorate coordinators/secretaries, please monitor the = dir-coord@ietf.list. If there is a request for a review, dispatch the = task to a reviewer. Managing overload is your task. Please acknowledge = requests and whether you can accommodate them. Directorates that today = review documents twice, at IETF Last Call and then before entering the = IESG telechat should continue this practice by doing the early review = and then the IESG telechat review. Existing practices such as retaining = the same reviewer for checking the same version is useful. Your review = tools and practices may need adjustment to accommodate reviews happening = at different times. >=20 > For the reviewers, take into account the context. Post your reviews to = the working group chairs, authors, and Cc the working group mailing list = if necessary. If your review team uses a template for these e-mails, = some changes may be necessary in the template for the early review. Some = of those changes have been discussed, e.g., in the Gen-ART list. >=20 > For everyone, please collect your experiences so that in six months we = can evaluate how you liked the experience. And thank you for your = participation in this effort! >=20 > Jari Arkko for the IESG From dthaler@microsoft.com Wed Nov 13 08:42:13 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1541A21E80CA for ; Wed, 13 Nov 2013 08:42:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.599 X-Spam-Level: X-Spam-Status: No, score=-103.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RBIJIBch7SID for ; Wed, 13 Nov 2013 08:42:08 -0800 (PST) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0236.outbound.protection.outlook.com [207.46.163.236]) by ietfa.amsl.com (Postfix) with ESMTP id 4A9D821E8116 for ; Wed, 13 Nov 2013 08:42:08 -0800 (PST) Received: from BY2PR03MB269.namprd03.prod.outlook.com (10.242.37.11) by BY2PR03MB272.namprd03.prod.outlook.com (10.242.37.24) with Microsoft SMTP Server (TLS) id 15.0.815.6; Wed, 13 Nov 2013 16:42:05 +0000 Received: from BY2PR03MB269.namprd03.prod.outlook.com ([169.254.5.176]) by BY2PR03MB269.namprd03.prod.outlook.com ([169.254.5.176]) with mapi id 15.00.0820.005; Wed, 13 Nov 2013 16:42:05 +0000 From: Dave Thaler To: "dir-coord@ietf.org" Thread-Topic: Requesting early review Thread-Index: Ac7gjvet7tVjCX/eRB+3SLT+Dx24Kw== Date: Wed, 13 Nov 2013 16:42:04 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [67.21.5.36] x-forefront-prvs: 0029F17A3F x-forefront-antispam-report: SFV:NSPM; SFS:(52044002)(377454003)(51704005)(199002)(189002)(13464003)(74316001)(76786001)(66066001)(80022001)(76176001)(74662001)(65816001)(31966008)(74706001)(63696002)(74366001)(76796001)(87936001)(69226001)(76576001)(56816003)(79102001)(81342001)(59766001)(77982001)(47446002)(77096001)(74876001)(81542001)(56776001)(54316002)(76482001)(33646001)(46102001)(47976001)(50986001)(54356001)(80976001)(47736001)(87266001)(49866001)(83322001)(74502001)(2656002)(4396001)(19580405001)(19580395003)(83072001)(85306002)(15975445006)(15202345003)(53806001)(81686001)(51856001)(81816001)(24736002); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR03MB272; H:BY2PR03MB269.namprd03.prod.outlook.com; CLIP:67.21.5.36; FPR:; RD:InfoNoRecords; A:1; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Cc: "'pcp-chairs@tools.ietf.org'" Subject: [dir-coord] Requesting early review X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Nov 2013 16:42:13 -0000 Of two drafts that were just submitted to the AD/IESG: draft-ietf-pcp-description-option-02 draft-ietf-pcp-nat64-prefix64-04 -Dave (doc shepherd for both) -----Original Message----- From: wgchairs-bounces@ietf.org [mailto:wgchairs-bounces@ietf.org] On Behal= f Of Jari Arkko Sent: Wednesday, November 13, 2013 5:26 AM To: Working Group Chairs Cc: dir-coord@ietf.org Subject: Reminder: asking for early reviews I talked to the Gen-ART team in Vancouver, and in the discussion it came up= that we have not gotten too many requests for early reviews. I wanted to r= emind you chairs that you can ask for early reviews. Hopefully this will ma= ke it easier for you to deal with any issues that might arise (easier than = having to deal with them when the reviews are raised at the IESG stage). In= addition, we need more experience with this practice to find out if the ex= periment worked! So feel free to send mail to dir-coord@ietf.org if you have a document that= is coming up for an AD/IESG submission. You could ask for a review in para= llel while your AD is doing the AD review, for instance. Jari > Internet Drafts sent for approval as RFCs are reviewed by individuals dur= ing the IETF Last Call, the Area Directors, IANA, as well as a number of vo= lunteers from various directorates and review teams. The reviews from these= teams has gained a significant role in ensuring that the IETF produces hig= h-quality, understandable and implementable RFCs. >=20 > Yet, as discussed in http://www.ietf.org/blog/2013/05/balancing-the-proce= ss/ we have a general problem that quite a lot of the work around IETF docu= ments happens at the end of the process. In particular, a number of the rev= iews during IETF last call point out issues that end up being raised by the= IESG as comments. It is of course good that issues are caught, but raising= them earlier would be better. And it would be better if the working groups= - the intended focus point of work on a topic - would get to handle them, = as opposed to raising these issues with the IESG. The IESG discussed these = issues in its May 2013 retreat, and decided to experiment with three action= s designed to move more work to the responsibility of the working groups: >=20 > (1) Perform some reviews that are now happening at IETF Last Call a bit e= arlier. This will put the working group in a bigger role in resolving cross= -area and general issues. >=20 > (2) Invite document shepherds on IESG telechats when there's a document t= hat is likely to require discussion. This will make it possible for the doc= ument shepherd to me directly be=20 >=20 > (3) When a document has a number of issues, hand over the process back to= the working group, as opposed to the IESG tracking the issues. Among other= things, this will ensure that changes are discussed in an open working gro= up list and agreed through consensus. >=20 > Some of you have seen (2) and (3) happen, more will come. For (1), buildi= ng quality and cross-area review to the process earlier is of course a big = effort. We plan to launch an experiment to make a small change to current d= irectorate review procedures to learn if we can move reviews a little bit e= arlier. If successful, this experiment will enable working groups to deal w= ith issues before IETF Last Call and IESG review and empower the working gr= oups to be in charge of the documents throughout their life cycle. We are a= lso hoping that document quality will improve and number of issues discusse= d in the IESG will be lower. >=20 > While the number of reviews as such is not changed, some additional effor= t and care will however be required from the reviewers, directorate coordin= ators/secretaries, the working group chairs, and other participants. The ex= periment will show us whether this effort is reasonable and if there are an= y unexpected effects. The experiment is performed on a voluntary basis by e= ach directorate. As early review requests come in, the directorates can thr= ottle workload by either processing the requests or reverting to the existi= ng procedures. Initially, the Gen-ART, Security Directorate, and Applicatio= ns Area Directorate are included in this effort. Other directorates may be = added later. The experiment will be reviewed after six months. >=20 > Care must be taken to avoid a number of possible drawbacks. There may be = problematic documents that would require much re-review and effort. Similar= ly, working groups often perform a number of working group last calls on a = document, and it would be undesirable to engage the reviewer before the doc= ument was really ready to be sent forward. And when reviewers send comments= , it is important that the group listens to the comments in the right way, = like you would for a review from a security expert or a general networking = expert. E-mail practices around sending and responding to comments have to = be carefully managed, as the outside reviewers are typically not list membe= rs. >=20 > For the working group chairs, you can request a review by sending mail to= dir-coord@ietf.org, indicating the name of the document. You can ask for a= review as soon as a working group last call has ended successfully and the= document edits are done. Initially, it would make sense to ask for reviews= on documents that you feel are likely the most stable ones. To avoid conge= sting the reviewer resources, ask for this service only for some documents,= not as a wholesale service on every document you submit for publication.=20 >=20 > You can ask for the review in parallel with ongoing AD reviews, filling o= ut the shepherd questionnaire, etc. The hope is that reviews can come in in= the same time frame as other tasks in this stage, and that you can take th= e comments into account and revise the document before finally sending it o= ut for IETF Last Call. Reviews will be sent to you and potentially the work= ing group mailing list. You may need to forward comments and/or approve new= posters to the list. Discussion on the mailing list is encouraged, but ple= ase try to ensure that the reviewer is kept on the Cc line, as he or she ma= y not be on the list itself. Treat the reviews with respect and keep it in = mind that outside experts may have opinions that need to be taken into acco= unt, even if the working group had not considered those aspects before. Med= iate and monitor the discussion actively. Try to keep the reviewer out of e= -mail storms and work out solutions separately and then engage the reviewer= again. >=20 > For the directorate coordinators/secretaries, please monitor the dir-coor= d@ietf.list. If there is a request for a review, dispatch the task to a rev= iewer. Managing overload is your task. Please acknowledge requests and whet= her you can accommodate them. Directorates that today review documents twic= e, at IETF Last Call and then before entering the IESG telechat should cont= inue this practice by doing the early review and then the IESG telechat rev= iew. Existing practices such as retaining the same reviewer for checking th= e same version is useful. Your review tools and practices may need adjustme= nt to accommodate reviews happening at different times. >=20 > For the reviewers, take into account the context. Post your reviews to th= e working group chairs, authors, and Cc the working group mailing list if n= ecessary. If your review team uses a template for these e-mails, some chang= es may be necessary in the template for the early review. Some of those cha= nges have been discussed, e.g., in the Gen-ART list. >=20 > For everyone, please collect your experiences so that in six months we ca= n evaluate how you liked the experience. And thank you for your participati= on in this effort! >=20 > Jari Arkko for the IESG From mahoney@nostrum.com Wed Nov 13 13:51:27 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C5F8411E8150 for ; Wed, 13 Nov 2013 13:51:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, SPF_PASS=-0.001] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ij9GUlyUwbPU for ; Wed, 13 Nov 2013 13:51:26 -0800 (PST) Received: from shaman.nostrum.com (nostrum-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:267::2]) by ietfa.amsl.com (Postfix) with ESMTP id A038311E810A for ; Wed, 13 Nov 2013 13:51:26 -0800 (PST) Received: from A-Jean-Mahoneys-MacBook-Pro.local ([4.30.77.1]) (authenticated bits=0) by shaman.nostrum.com (8.14.3/8.14.3) with ESMTP id rADLpP4C085946 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 13 Nov 2013 15:51:25 -0600 (CST) (envelope-from mahoney@nostrum.com) Message-ID: <5283F45D.7020507@nostrum.com> Date: Wed, 13 Nov 2013 15:51:25 -0600 From: "A. Jean Mahoney" User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Dave Thaler , "dir-coord@ietf.org" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Received-SPF: pass (shaman.nostrum.com: 4.30.77.1 is authenticated by a trusted mechanism) Cc: "'pcp-chairs@tools.ietf.org'" Subject: Re: [dir-coord] Requesting early review X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Nov 2013 21:51:27 -0000 Hi Dave, What's the deadline? Thanks! Jean On 11/13/13 10:42 AM, Dave Thaler wrote: > Of two drafts that were just submitted to the AD/IESG: > > draft-ietf-pcp-description-option-02 > draft-ietf-pcp-nat64-prefix64-04 > > -Dave (doc shepherd for both) > > -----Original Message----- > From: wgchairs-bounces@ietf.org [mailto:wgchairs-bounces@ietf.org] On B= ehalf Of Jari Arkko > Sent: Wednesday, November 13, 2013 5:26 AM > To: Working Group Chairs > Cc: dir-coord@ietf.org > Subject: Reminder: asking for early reviews > > I talked to the Gen-ART team in Vancouver, and in the discussion it cam= e up that we have not gotten too many requests for early reviews. I wante= d to remind you chairs that you can ask for early reviews. Hopefully this= will make it easier for you to deal with any issues that might arise (ea= sier than having to deal with them when the reviews are raised at the IES= G stage). In addition, we need more experience with this practice to find= out if the experiment worked! > > So feel free to send mail to dir-coord@ietf.org if you have a document = that is coming up for an AD/IESG submission. You could ask for a review i= n parallel while your AD is doing the AD review, for instance. > > Jari > >> Internet Drafts sent for approval as RFCs are reviewed by individuals = during the IETF Last Call, the Area Directors, IANA, as well as a number = of volunteers from various directorates and review teams. The reviews fro= m these teams has gained a significant role in ensuring that the IETF pro= duces high-quality, understandable and implementable RFCs. >> >> Yet, as discussed in http://www.ietf.org/blog/2013/05/balancing-the-pr= ocess/ we have a general problem that quite a lot of the work around IETF= documents happens at the end of the process. In particular, a number of = the reviews during IETF last call point out issues that end up being rais= ed by the IESG as comments. It is of course good that issues are caught, = but raising them earlier would be better. And it would be better if the w= orking groups - the intended focus point of work on a topic - would get t= o handle them, as opposed to raising these issues with the IESG. The IESG= discussed these issues in its May 2013 retreat, and decided to experimen= t with three actions designed to move more work to the responsibility of = the working groups: >> >> (1) Perform some reviews that are now happening at IETF Last Call a bi= t earlier. This will put the working group in a bigger role in resolving = cross-area and general issues. >> >> (2) Invite document shepherds on IESG telechats when there's a documen= t that is likely to require discussion. This will make it possible for th= e document shepherd to me directly be >> >> (3) When a document has a number of issues, hand over the process back= to the working group, as opposed to the IESG tracking the issues. Among = other things, this will ensure that changes are discussed in an open work= ing group list and agreed through consensus. >> >> Some of you have seen (2) and (3) happen, more will come. For (1), bui= lding quality and cross-area review to the process earlier is of course a= big effort. We plan to launch an experiment to make a small change to cu= rrent directorate review procedures to learn if we can move reviews a lit= tle bit earlier. If successful, this experiment will enable working group= s to deal with issues before IETF Last Call and IESG review and empower t= he working groups to be in charge of the documents throughout their life = cycle. We are also hoping that document quality will improve and number o= f issues discussed in the IESG will be lower. >> >> While the number of reviews as such is not changed, some additional ef= fort and care will however be required from the reviewers, directorate co= ordinators/secretaries, the working group chairs, and other participants.= The experiment will show us whether this effort is reasonable and if the= re are any unexpected effects. The experiment is performed on a voluntary= basis by each directorate. As early review requests come in, the directo= rates can throttle workload by either processing the requests or revertin= g to the existing procedures. Initially, the Gen-ART, Security Directorat= e, and Applications Area Directorate are included in this effort. Other d= irectorates may be added later. The experiment will be reviewed after six= months. >> >> Care must be taken to avoid a number of possible drawbacks. There may = be problematic documents that would require much re-review and effort. Si= milarly, working groups often perform a number of working group last call= s on a document, and it would be undesirable to engage the reviewer befor= e the document was really ready to be sent forward. And when reviewers se= nd comments, it is important that the group listens to the comments in th= e right way, like you would for a review from a security expert or a gene= ral networking expert. E-mail practices around sending and responding to = comments have to be carefully managed, as the outside reviewers are typic= ally not list members. >> >> For the working group chairs, you can request a review by sending mail= to dir-coord@ietf.org, indicating the name of the document. You can ask = for a review as soon as a working group last call has ended successfully = and the document edits are done. Initially, it would make sense to ask fo= r reviews on documents that you feel are likely the most stable ones. To = avoid congesting the reviewer resources, ask for this service only for so= me documents, not as a wholesale service on every document you submit for= publication. >> >> You can ask for the review in parallel with ongoing AD reviews, fillin= g out the shepherd questionnaire, etc. The hope is that reviews can come = in in the same time frame as other tasks in this stage, and that you can = take the comments into account and revise the document before finally sen= ding it out for IETF Last Call. Reviews will be sent to you and potential= ly the working group mailing list. You may need to forward comments and/o= r approve new posters to the list. Discussion on the mailing list is enco= uraged, but please try to ensure that the reviewer is kept on the Cc line= , as he or she may not be on the list itself. Treat the reviews with resp= ect and keep it in mind that outside experts may have opinions that need = to be taken into account, even if the working group had not considered th= ose aspects before. Mediate and monitor the discussion actively. Try to k= eep the reviewer out of e-mail storms and work out solutions separately a= nd then engage the reviewer again. >> >> For the directorate coordinators/secretaries, please monitor the dir-c= oord@ietf.list. If there is a request for a review, dispatch the task to = a reviewer. Managing overload is your task. Please acknowledge requests a= nd whether you can accommodate them. Directorates that today review docum= ents twice, at IETF Last Call and then before entering the IESG telechat = should continue this practice by doing the early review and then the IESG= telechat review. Existing practices such as retaining the same reviewer = for checking the same version is useful. Your review tools and practices = may need adjustment to accommodate reviews happening at different times. >> >> For the reviewers, take into account the context. Post your reviews to= the working group chairs, authors, and Cc the working group mailing list= if necessary. If your review team uses a template for these e-mails, som= e changes may be necessary in the template for the early review. Some of = those changes have been discussed, e.g., in the Gen-ART list. >> >> For everyone, please collect your experiences so that in six months we= can evaluate how you liked the experience. And thank you for your partic= ipation in this effort! >> >> Jari Arkko for the IESG > _______________________________________________ > dir-coord mailing list > dir-coord@ietf.org > https://www.ietf.org/mailman/listinfo/dir-coord From dthaler@microsoft.com Wed Nov 13 15:48:32 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C51521E80CF for ; Wed, 13 Nov 2013 15:48:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -103.599 X-Spam-Level: X-Spam-Status: No, score=-103.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kUrcNX2EYXIm for ; Wed, 13 Nov 2013 15:48:27 -0800 (PST) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0203.outbound.protection.outlook.com [207.46.163.203]) by ietfa.amsl.com (Postfix) with ESMTP id 2626D21E8094 for ; Wed, 13 Nov 2013 15:48:26 -0800 (PST) Received: from BY2PR03MB269.namprd03.prod.outlook.com (10.242.37.11) by BY2PR03MB272.namprd03.prod.outlook.com (10.242.37.24) with Microsoft SMTP Server (TLS) id 15.0.815.6; Wed, 13 Nov 2013 23:48:23 +0000 Received: from BY2PR03MB269.namprd03.prod.outlook.com ([169.254.5.176]) by BY2PR03MB269.namprd03.prod.outlook.com ([169.254.5.176]) with mapi id 15.00.0820.005; Wed, 13 Nov 2013 23:48:23 +0000 From: Dave Thaler To: "A. Jean Mahoney" , "dir-coord@ietf.org" Thread-Topic: [dir-coord] Requesting early review Thread-Index: Ac7gjvet7tVjCX/eRB+3SLT+Dx24KwAK4fyAAAQJ5FA= Date: Wed, 13 Nov 2013 23:48:23 +0000 Message-ID: <483b69b1415f46b29719ebd85ee81437@BY2PR03MB269.namprd03.prod.outlook.com> References: <5283F45D.7020507@nostrum.com> In-Reply-To: <5283F45D.7020507@nostrum.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [67.21.5.36] x-forefront-prvs: 0029F17A3F x-forefront-antispam-report: SFV:NSPM; SFS:(199002)(189002)(13464003)(24454002)(41574002)(51704005)(479174003)(377454003)(52044002)(47976001)(50986001)(33646001)(46102001)(81542001)(56776001)(74876001)(76482001)(54316002)(54356001)(15202345003)(53806001)(15975445006)(19580405001)(19580395003)(85306002)(83072001)(81816001)(81686001)(51856001)(49866001)(87266001)(80976001)(47736001)(2656002)(4396001)(74502001)(83322001)(74662001)(80022001)(66066001)(31966008)(65816001)(74316001)(76786001)(59766001)(81342001)(77982001)(56816003)(79102001)(76576001)(77096001)(47446002)(74366001)(76796001)(63696002)(74706001)(69226001)(87936001)(24736002); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR03MB272; H:BY2PR03MB269.namprd03.prod.outlook.com; CLIP:67.21.5.36; FPR:; RD:InfoNoRecords; A:1; MX:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com Cc: "'pcp-chairs@tools.ietf.org'" , Ted Lemon Subject: Re: [dir-coord] Requesting early review X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Nov 2013 23:48:32 -0000 Well it's in already in AD (Ted) review now. So I guess the deadline is by= when the IESG needs it, by which point it won't be an "early" review :) For a better answer, Ted may want to comment on his timeline. -Dave > -----Original Message----- > From: A. Jean Mahoney [mailto:mahoney@nostrum.com] > Sent: Wednesday, November 13, 2013 1:51 PM > To: Dave Thaler; dir-coord@ietf.org > Cc: 'pcp-chairs@tools.ietf.org' > Subject: Re: [dir-coord] Requesting early review >=20 > Hi Dave, >=20 > What's the deadline? >=20 > Thanks! >=20 > Jean >=20 > On 11/13/13 10:42 AM, Dave Thaler wrote: > > Of two drafts that were just submitted to the AD/IESG: > > > > draft-ietf-pcp-description-option-02 > > draft-ietf-pcp-nat64-prefix64-04 > > > > -Dave (doc shepherd for both) > > > > -----Original Message----- > > From: wgchairs-bounces@ietf.org [mailto:wgchairs-bounces@ietf.org] On > Behalf Of Jari Arkko > > Sent: Wednesday, November 13, 2013 5:26 AM > > To: Working Group Chairs > > Cc: dir-coord@ietf.org > > Subject: Reminder: asking for early reviews > > > > I talked to the Gen-ART team in Vancouver, and in the discussion it cam= e > up that we have not gotten too many requests for early reviews. I wanted = to > remind you chairs that you can ask for early reviews. Hopefully this will= make > it easier for you to deal with any issues that might arise (easier than h= aving to > deal with them when the reviews are raised at the IESG stage). In additio= n, > we need more experience with this practice to find out if the experiment > worked! > > > > So feel free to send mail to dir-coord@ietf.org if you have a document = that > is coming up for an AD/IESG submission. You could ask for a review in par= allel > while your AD is doing the AD review, for instance. > > > > Jari > > > >> Internet Drafts sent for approval as RFCs are reviewed by individuals > during the IETF Last Call, the Area Directors, IANA, as well as a number = of > volunteers from various directorates and review teams. The reviews from > these teams has gained a significant role in ensuring that the IETF produ= ces > high-quality, understandable and implementable RFCs. > >> > >> Yet, as discussed in http://www.ietf.org/blog/2013/05/balancing-the- > process/ we have a general problem that quite a lot of the work around IE= TF > documents happens at the end of the process. In particular, a number of t= he > reviews during IETF last call point out issues that end up being raised b= y the > IESG as comments. It is of course good that issues are caught, but raisin= g > them earlier would be better. And it would be better if the working group= s - > the intended focus point of work on a topic - would get to handle them, a= s > opposed to raising these issues with the IESG. The IESG discussed these > issues in its May 2013 retreat, and decided to experiment with three acti= ons > designed to move more work to the responsibility of the working groups: > >> > >> (1) Perform some reviews that are now happening at IETF Last Call a bi= t > earlier. This will put the working group in a bigger role in resolving cr= oss-area > and general issues. > >> > >> (2) Invite document shepherds on IESG telechats when there's a > document that is likely to require discussion. This will make it possible= for the > document shepherd to me directly be > >> > >> (3) When a document has a number of issues, hand over the process back > to the working group, as opposed to the IESG tracking the issues. Among > other things, this will ensure that changes are discussed in an open work= ing > group list and agreed through consensus. > >> > >> Some of you have seen (2) and (3) happen, more will come. For (1), > building quality and cross-area review to the process earlier is of cours= e a big > effort. We plan to launch an experiment to make a small change to current > directorate review procedures to learn if we can move reviews a little bi= t > earlier. If successful, this experiment will enable working groups to dea= l with > issues before IETF Last Call and IESG review and empower the working > groups to be in charge of the documents throughout their life cycle. We a= re > also hoping that document quality will improve and number of issues > discussed in the IESG will be lower. > >> > >> While the number of reviews as such is not changed, some additional > effort and care will however be required from the reviewers, directorate > coordinators/secretaries, the working group chairs, and other participant= s. > The experiment will show us whether this effort is reasonable and if ther= e > are any unexpected effects. The experiment is performed on a voluntary > basis by each directorate. As early review requests come in, the director= ates > can throttle workload by either processing the requests or reverting to t= he > existing procedures. Initially, the Gen-ART, Security Directorate, and > Applications Area Directorate are included in this effort. Other director= ates > may be added later. The experiment will be reviewed after six months. > >> > >> Care must be taken to avoid a number of possible drawbacks. There may > be problematic documents that would require much re-review and effort. > Similarly, working groups often perform a number of working group last ca= lls > on a document, and it would be undesirable to engage the reviewer before > the document was really ready to be sent forward. And when reviewers > send comments, it is important that the group listens to the comments in = the > right way, like you would for a review from a security expert or a genera= l > networking expert. E-mail practices around sending and responding to > comments have to be carefully managed, as the outside reviewers are > typically not list members. > >> > >> For the working group chairs, you can request a review by sending mail= to > dir-coord@ietf.org, indicating the name of the document. You can ask for = a > review as soon as a working group last call has ended successfully and th= e > document edits are done. Initially, it would make sense to ask for review= s on > documents that you feel are likely the most stable ones. To avoid congest= ing > the reviewer resources, ask for this service only for some documents, not= as > a wholesale service on every document you submit for publication. > >> > >> You can ask for the review in parallel with ongoing AD reviews, fillin= g out > the shepherd questionnaire, etc. The hope is that reviews can come in in = the > same time frame as other tasks in this stage, and that you can take the > comments into account and revise the document before finally sending it o= ut > for IETF Last Call. Reviews will be sent to you and potentially the worki= ng > group mailing list. You may need to forward comments and/or approve new > posters to the list. Discussion on the mailing list is encouraged, but pl= ease try > to ensure that the reviewer is kept on the Cc line, as he or she may not = be on > the list itself. Treat the reviews with respect and keep it in mind that = outside > experts may have opinions that need to be taken into account, even if the > working group had not considered those aspects before. Mediate and > monitor the discussion actively. Try to keep the reviewer out of e-mail > storms and work out solutions separately and then engage the reviewer > again. > >> > >> For the directorate coordinators/secretaries, please monitor the dir- > coord@ietf.list. If there is a request for a review, dispatch the task to= a > reviewer. Managing overload is your task. Please acknowledge requests and > whether you can accommodate them. Directorates that today review > documents twice, at IETF Last Call and then before entering the IESG tele= chat > should continue this practice by doing the early review and then the IESG > telechat review. Existing practices such as retaining the same reviewer f= or > checking the same version is useful. Your review tools and practices may > need adjustment to accommodate reviews happening at different times. > >> > >> For the reviewers, take into account the context. Post your reviews to= the > working group chairs, authors, and Cc the working group mailing list if > necessary. If your review team uses a template for these e-mails, some > changes may be necessary in the template for the early review. Some of > those changes have been discussed, e.g., in the Gen-ART list. > >> > >> For everyone, please collect your experiences so that in six months we > can evaluate how you liked the experience. And thank you for your > participation in this effort! > >> > >> Jari Arkko for the IESG > > _______________________________________________ > > dir-coord mailing list > > dir-coord@ietf.org > > https://www.ietf.org/mailman/listinfo/dir-coord >=20 From Ted.Lemon@nominum.com Wed Nov 13 17:48:26 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 426A011E8128 for ; Wed, 13 Nov 2013 17:48:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.584 X-Spam-Level: X-Spam-Status: No, score=-106.584 tagged_above=-999 required=5 tests=[AWL=0.015, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AJVrRnPD60NQ for ; Wed, 13 Nov 2013 17:48:19 -0800 (PST) Received: from exprod7og123.obsmtp.com (exprod7og123.obsmtp.com [64.18.2.24]) by ietfa.amsl.com (Postfix) with ESMTP id 79DAC11E8102 for ; Wed, 13 Nov 2013 17:48:19 -0800 (PST) Received: from shell-too.nominum.com ([64.89.228.229]) (using TLSv1) by exprod7ob123.postini.com ([64.18.6.12]) with SMTP ID DSNKUoQr44TisQPJ88kkFmBOJE1n9EfKNwk2@postini.com; Wed, 13 Nov 2013 17:48:19 PST Received: from archivist.nominum.com (archivist.nominum.com [64.89.228.108]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.nominum.com", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by shell-too.nominum.com (Postfix) with ESMTP id 088571B82AE for ; Wed, 13 Nov 2013 17:48:19 -0800 (PST) Received: from webmail.nominum.com (cas-02.win.nominum.com [64.89.228.132]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.nominum.com", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by archivist.nominum.com (Postfix) with ESMTPS id DC58D190043; Wed, 13 Nov 2013 17:48:18 -0800 (PST) (envelope-from Ted.Lemon@nominum.com) Received: from MBX-01.WIN.NOMINUM.COM ([64.89.228.133]) by CAS-02.WIN.NOMINUM.COM ([64.89.228.132]) with mapi id 14.03.0158.001; Wed, 13 Nov 2013 17:48:18 -0800 From: Ted Lemon To: Dave Thaler Thread-Topic: [dir-coord] Requesting early review Thread-Index: Ac7gjvet7tVjCX/eRB+3SLT+Dx24KwAK4fyAAAQJ5FAAFP9mgA== Date: Thu, 14 Nov 2013 01:48:17 +0000 Message-ID: <9D549D7B-2934-4D8D-8A10-453DD74978F9@nominum.com> References: <5283F45D.7020507@nostrum.com> <483b69b1415f46b29719ebd85ee81437@BY2PR03MB269.namprd03.prod.outlook.com> In-Reply-To: <483b69b1415f46b29719ebd85ee81437@BY2PR03MB269.namprd03.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.1.10] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "pcp-chairs@tools.ietf.org" , "A. Jean Mahoney" , "dir-coord@ietf.org" Subject: Re: [dir-coord] Requesting early review X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Nov 2013 01:48:26 -0000 This is really something that's up to the chairs/document shepherd. I wou= ld be a bit surprised if these were in dire need of early review. However= , if you want to request reviews early, you are certainly welcome to do so. From bclaise@cisco.com Wed Nov 13 17:59:28 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 21E5611E810C for ; Wed, 13 Nov 2013 17:59:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.498 X-Spam-Level: X-Spam-Status: No, score=-10.498 tagged_above=-999 required=5 tests=[AWL=0.101, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jL8HH7qXUvI8 for ; Wed, 13 Nov 2013 17:59:23 -0800 (PST) Received: from mtv-iport-3.cisco.com (mtv-iport-3.cisco.com [173.36.130.14]) by ietfa.amsl.com (Postfix) with ESMTP id 9873921F9D8E for ; Wed, 13 Nov 2013 17:59:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=8325; q=dns/txt; s=iport; t=1384394363; x=1385603963; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=J4edMlq87GrbkpoVpPf91qyWEpu9IOyBF/a7m1oSkKY=; b=C4/cEGfZnpvlTAf1GaaS4z6KYa9WAUpx7XWzO/tdbSXWcXzByROkU3XH mPdd1WYl/Zu2eDYNXsiQfYpuZ2A+faML4DokXwAW8YN5KGpFpgKWlzOXY r2xlvBkFJ+us+Hf6PnOLHxS4HyCDMgqcqxYLqVZYVMQ519PoZ/DC+RYqy E=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgUFAPothFKrRDoH/2dsb2JhbABZgwc4v36BJxZ0giUBAQEEAQEBNTMBAgYEAQwECw4DBAEBCgwKCAcJAwIBAgEVHwkIBgEMAQUCAQEFEAKHZQ6/d44VEYEGMwcGBIQnA4lCiiqEJIY9i06DSRuBNQ X-IronPort-AV: E=Sophos;i="4.93,696,1378857600"; d="scan'208";a="94986426" Received: from mtv-core-2.cisco.com ([171.68.58.7]) by mtv-iport-3.cisco.com with ESMTP; 14 Nov 2013 01:59:20 +0000 Received: from [10.154.209.229] ([10.154.209.229]) by mtv-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id rAE1xI0G022306; Thu, 14 Nov 2013 01:59:19 GMT Message-ID: <52842E76.6010007@cisco.com> Date: Wed, 13 Nov 2013 17:59:18 -0800 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Dave Thaler , "dir-coord@ietf.org" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Cc: "'pcp-chairs@tools.ietf.org'" Subject: Re: [dir-coord] Requesting early review X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Nov 2013 01:59:28 -0000 Hi, I thought the early reviews were more appropriate when the documents=20 were still in the WGs. The directorate reviews will anyway be triggered by the IETF LC. Regards, Benoit > Of two drafts that were just submitted to the AD/IESG: > > draft-ietf-pcp-description-option-02 > draft-ietf-pcp-nat64-prefix64-04 > > -Dave (doc shepherd for both) > > -----Original Message----- > From: wgchairs-bounces@ietf.org [mailto:wgchairs-bounces@ietf.org] On B= ehalf Of Jari Arkko > Sent: Wednesday, November 13, 2013 5:26 AM > To: Working Group Chairs > Cc: dir-coord@ietf.org > Subject: Reminder: asking for early reviews > > I talked to the Gen-ART team in Vancouver, and in the discussion it cam= e up that we have not gotten too many requests for early reviews. I wante= d to remind you chairs that you can ask for early reviews. Hopefully this= will make it easier for you to deal with any issues that might arise (ea= sier than having to deal with them when the reviews are raised at the IES= G stage). In addition, we need more experience with this practice to find= out if the experiment worked! > > So feel free to send mail to dir-coord@ietf.org if you have a document = that is coming up for an AD/IESG submission. You could ask for a review i= n parallel while your AD is doing the AD review, for instance. > > Jari > >> Internet Drafts sent for approval as RFCs are reviewed by individuals = during the IETF Last Call, the Area Directors, IANA, as well as a number = of volunteers from various directorates and review teams. The reviews fro= m these teams has gained a significant role in ensuring that the IETF pro= duces high-quality, understandable and implementable RFCs. >> >> Yet, as discussed in http://www.ietf.org/blog/2013/05/balancing-the-pr= ocess/ we have a general problem that quite a lot of the work around IETF= documents happens at the end of the process. In particular, a number of = the reviews during IETF last call point out issues that end up being rais= ed by the IESG as comments. It is of course good that issues are caught, = but raising them earlier would be better. And it would be better if the w= orking groups - the intended focus point of work on a topic - would get t= o handle them, as opposed to raising these issues with the IESG. The IESG= discussed these issues in its May 2013 retreat, and decided to experimen= t with three actions designed to move more work to the responsibility of = the working groups: >> >> (1) Perform some reviews that are now happening at IETF Last Call a bi= t earlier. This will put the working group in a bigger role in resolving = cross-area and general issues. >> >> (2) Invite document shepherds on IESG telechats when there's a documen= t that is likely to require discussion. This will make it possible for th= e document shepherd to me directly be >> >> (3) When a document has a number of issues, hand over the process back= to the working group, as opposed to the IESG tracking the issues. Among = other things, this will ensure that changes are discussed in an open work= ing group list and agreed through consensus. >> >> Some of you have seen (2) and (3) happen, more will come. For (1), bui= lding quality and cross-area review to the process earlier is of course a= big effort. We plan to launch an experiment to make a small change to cu= rrent directorate review procedures to learn if we can move reviews a lit= tle bit earlier. If successful, this experiment will enable working group= s to deal with issues before IETF Last Call and IESG review and empower t= he working groups to be in charge of the documents throughout their life = cycle. We are also hoping that document quality will improve and number o= f issues discussed in the IESG will be lower. >> >> While the number of reviews as such is not changed, some additional ef= fort and care will however be required from the reviewers, directorate co= ordinators/secretaries, the working group chairs, and other participants.= The experiment will show us whether this effort is reasonable and if the= re are any unexpected effects. The experiment is performed on a voluntary= basis by each directorate. As early review requests come in, the directo= rates can throttle workload by either processing the requests or revertin= g to the existing procedures. Initially, the Gen-ART, Security Directorat= e, and Applications Area Directorate are included in this effort. Other d= irectorates may be added later. The experiment will be reviewed after six= months. >> >> Care must be taken to avoid a number of possible drawbacks. There may = be problematic documents that would require much re-review and effort. Si= milarly, working groups often perform a number of working group last call= s on a document, and it would be undesirable to engage the reviewer befor= e the document was really ready to be sent forward. And when reviewers se= nd comments, it is important that the group listens to the comments in th= e right way, like you would for a review from a security expert or a gene= ral networking expert. E-mail practices around sending and responding to = comments have to be carefully managed, as the outside reviewers are typic= ally not list members. >> >> For the working group chairs, you can request a review by sending mail= to dir-coord@ietf.org, indicating the name of the document. You can ask = for a review as soon as a working group last call has ended successfully = and the document edits are done. Initially, it would make sense to ask fo= r reviews on documents that you feel are likely the most stable ones. To = avoid congesting the reviewer resources, ask for this service only for so= me documents, not as a wholesale service on every document you submit for= publication. >> >> You can ask for the review in parallel with ongoing AD reviews, fillin= g out the shepherd questionnaire, etc. The hope is that reviews can come = in in the same time frame as other tasks in this stage, and that you can = take the comments into account and revise the document before finally sen= ding it out for IETF Last Call. Reviews will be sent to you and potential= ly the working group mailing list. You may need to forward comments and/o= r approve new posters to the list. Discussion on the mailing list is enco= uraged, but please try to ensure that the reviewer is kept on the Cc line= , as he or she may not be on the list itself. Treat the reviews with resp= ect and keep it in mind that outside experts may have opinions that need = to be taken into account, even if the working group had not considered th= ose aspects before. Mediate and monitor the discussion actively. Try to k= eep the reviewer out of e-mail storms and work out solutions separately a= nd then engage the reviewer again. >> >> For the directorate coordinators/secretaries, please monitor the dir-c= oord@ietf.list. If there is a request for a review, dispatch the task to = a reviewer. Managing overload is your task. Please acknowledge requests a= nd whether you can accommodate them. Directorates that today review docum= ents twice, at IETF Last Call and then before entering the IESG telechat = should continue this practice by doing the early review and then the IESG= telechat review. Existing practices such as retaining the same reviewer = for checking the same version is useful. Your review tools and practices = may need adjustment to accommodate reviews happening at different times. >> >> For the reviewers, take into account the context. Post your reviews to= the working group chairs, authors, and Cc the working group mailing list= if necessary. If your review team uses a template for these e-mails, som= e changes may be necessary in the template for the early review. Some of = those changes have been discussed, e.g., in the Gen-ART list. >> >> For everyone, please collect your experiences so that in six months we= can evaluate how you liked the experience. And thank you for your partic= ipation in this effort! >> >> Jari Arkko for the IESG > _______________________________________________ > dir-coord mailing list > dir-coord@ietf.org > https://www.ietf.org/mailman/listinfo/dir-coord > . > From fred@cisco.com Wed Nov 20 23:48:39 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3A74C1AC3DD for ; Wed, 20 Nov 2013 23:48:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -114.776 X-Spam-Level: X-Spam-Status: No, score=-114.776 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, BODY_URI_ONLY=0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.525, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zGABpLjUdFN1 for ; Wed, 20 Nov 2013 23:48:37 -0800 (PST) Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) by ietfa.amsl.com (Postfix) with ESMTP id 658BD1A1F78 for ; Wed, 20 Nov 2013 23:48:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1291; q=dns/txt; s=iport; t=1385020111; x=1386229711; h=from:to:cc:subject:date:message-id:mime-version; bh=Z+ciA6ov4/KAmW+tP6Mqe+ZH3WXNOK+8zXC/EtYQMH0=; b=nAa+kmgZk5ASSBOOrB9WVeLc2hpCMfdGTy7QARHdPE5+GZ/8pNmVSJ8x HwVBblL2KljeExndWnKa3JoZp/jsmCz1LlzX4ZnJD4gMvO7B9cfQIMfK8 xGEmSyupvUA7h93qMTuojkKCEBu9Uon4U4brAfyAzIzMvP6txlfXJcjOG A=; X-Files: signature.asc : 195 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvUFAAm6jVKtJV2Z/2dsb2JhbABZgweBC71ygR0WbQeCLGUUEgGBACcEDhMNh2bAcBePa4MngRIDkDCBMYYxkhCDKIIq X-IronPort-AV: E=Sophos;i="4.93,742,1378857600"; d="asc'?scan'208";a="286555585" Received: from rcdn-core-2.cisco.com ([173.37.93.153]) by rcdn-iport-6.cisco.com with ESMTP; 21 Nov 2013 07:48:30 +0000 Received: from xhc-rcd-x05.cisco.com (xhc-rcd-x05.cisco.com [173.37.183.79]) by rcdn-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id rAL7mUMT014111 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 21 Nov 2013 07:48:30 GMT Received: from xmb-rcd-x09.cisco.com ([169.254.9.136]) by xhc-rcd-x05.cisco.com ([173.37.183.79]) with mapi id 14.03.0123.003; Thu, 21 Nov 2013 01:48:30 -0600 From: "Fred Baker (fred)" To: "dir-coord@ietf.org" Thread-Topic: Requesting SECDIR review of draft-ietf-v6ops-balanced-ipv6-security Thread-Index: AQHO5o4Rt2HUtsre+E+xEICIlT05Ow== Date: Thu, 21 Nov 2013 07:48:29 +0000 Message-ID: <16F5011B-A87C-4BE3-8AA4-F47C13E830F6@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.19.64.121] Content-Type: multipart/signed; boundary="Apple-Mail=_C74EAE95-D5CB-4DEE-8AFC-F7647EE7B1B7"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: joel jaeggli , John Brzozowski Subject: [dir-coord] Requesting SECDIR review of draft-ietf-v6ops-balanced-ipv6-security X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2013 07:48:39 -0000 --Apple-Mail=_C74EAE95-D5CB-4DEE-8AFC-F7647EE7B1B7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii This document is right now in WGLC. Some points have been raised, which = I imagine will warrant a revision. However, there are a lot of claims = being made in and about the draft to the effect that it is a security = solution that Swisscom has deployed and sees no problems with, and = therefore the IETF should bless it as a general firewall solution. I = would appreciate a timely review from the Security Directorate that = would help us know whether it is likely to pass muster with the IESG, = and if not, what types problems the directorate sees and what types of = solutions would be acceptable. --Apple-Mail=_C74EAE95-D5CB-4DEE-8AFC-F7647EE7B1B7 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iD8DBQFSjbrNbjEdbHIsm0MRAgKNAJ9DqALKbHM7udfb6ukUITiYaZ1aiwCg2jxC LqrCfL2b6ls8mND8LD4Delo= =sPXP -----END PGP SIGNATURE----- --Apple-Mail=_C74EAE95-D5CB-4DEE-8AFC-F7647EE7B1B7-- From kivinen@iki.fi Thu Nov 21 05:17:15 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 562951AE138 for ; Thu, 21 Nov 2013 05:17:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.426 X-Spam-Level: X-Spam-Status: No, score=-2.426 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-0.525, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZBKzS4Og9ezj for ; Thu, 21 Nov 2013 05:17:14 -0800 (PST) Received: from mail.kivinen.iki.fi (fireball.kivinen.iki.fi [IPv6:2001:1bc8:100d::2]) by ietfa.amsl.com (Postfix) with ESMTP id E94DF1ADC03 for ; Thu, 21 Nov 2013 05:17:13 -0800 (PST) Received: from fireball.kivinen.iki.fi (localhost [127.0.0.1]) by mail.kivinen.iki.fi (8.14.7/8.14.5) with ESMTP id rALDGrjC029064 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 21 Nov 2013 15:16:53 +0200 (EET) Received: (from kivinen@localhost) by fireball.kivinen.iki.fi (8.14.7/8.12.11) id rALDGpPa000659; Thu, 21 Nov 2013 15:16:51 +0200 (EET) X-Authentication-Warning: fireball.kivinen.iki.fi: kivinen set sender to kivinen@iki.fi using -f MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21134.1987.918612.52011@fireball.kivinen.iki.fi> Date: Thu, 21 Nov 2013 15:16:51 +0200 From: Tero Kivinen To: "Fred Baker \(fred\)" In-Reply-To: <16F5011B-A87C-4BE3-8AA4-F47C13E830F6@cisco.com> References: <16F5011B-A87C-4BE3-8AA4-F47C13E830F6@cisco.com> X-Mailer: VM 8.2.0b under 24.3.1 (x86_64--netbsd) X-Edit-Time: 1 min X-Total-Time: 1 min Cc: joel jaeggli , paul.hoffman@vpnc.org, "dir-coord@ietf.org" , John Brzozowski Subject: [dir-coord] Requesting SECDIR review of draft-ietf-v6ops-balanced-ipv6-security X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2013 13:17:15 -0000 Fred Baker (fred) writes: > This document is right now in WGLC. Some points have been raised, > which I imagine will warrant a revision. However, there are a lot of > claims being made in and about the draft to the effect that it is a > security solution that Swisscom has deployed and sees no problems > with, and therefore the IETF should bless it as a general firewall > solution. I would appreciate a timely review from the Security > Directorate that would help us know whether it is likely to pass > muster with the IESG, and if not, what types problems the > directorate sees and what types of solutions would be acceptable. This was assigned to Paul Hoffman today, I put the deadline for next week on the review. -- kivinen@iki.fi From fred@cisco.com Thu Nov 21 09:08:22 2013 Return-Path: X-Original-To: dir-coord@ietfa.amsl.com Delivered-To: dir-coord@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 821A61AE058 for ; Thu, 21 Nov 2013 09:08:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -110.026 X-Spam-Level: X-Spam-Status: No, score=-110.026 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-0.525, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fztM2XC0_WXr for ; Thu, 21 Nov 2013 09:08:20 -0800 (PST) Received: from alln-iport-8.cisco.com (alln-iport-8.cisco.com [173.37.142.95]) by ietfa.amsl.com (Postfix) with ESMTP id 9550D1AE039 for ; Thu, 21 Nov 2013 09:08:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1529; q=dns/txt; s=iport; t=1385053694; x=1386263294; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=/hp9tKkqjksLr4mWWTBK1MVb+N3wYO2BGeAQGm3fTA0=; b=klgRSX0AzSM952eP7R97g+TD2oU2XpFlYnlHeqZM96uZhtAGUalSIa3W fOddkW6OV1QHMBunijYLTKzuooB+uA8Hk29NhWDdQdhyNrtcZWk6DCB0L g29ep3lhF4YSqD998IQlK1yKlILkChLifgsD+Y/FQPkJio7ZIp13Bco2y k=; X-Files: signature.asc : 195 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhgFAL48jlKtJXG+/2dsb2JhbABZgweBC7xVgSQWdIIlAQEBAwFlFAULAgEIRjIlAgQOBQ4Nh2AGwScXj2sHgyCBEgOQMIExhjGSEIFqgT6CKg X-IronPort-AV: E=Sophos;i="4.93,745,1378857600"; d="asc'?scan'208";a="1264474" Received: from rcdn-core2-3.cisco.com ([173.37.113.190]) by alln-iport-8.cisco.com with ESMTP; 21 Nov 2013 17:08:13 +0000 Received: from xhc-aln-x14.cisco.com (xhc-aln-x14.cisco.com [173.36.12.88]) by rcdn-core2-3.cisco.com (8.14.5/8.14.5) with ESMTP id rALH8DD9026189 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 21 Nov 2013 17:08:13 GMT Received: from xmb-rcd-x09.cisco.com ([169.254.9.136]) by xhc-aln-x14.cisco.com ([173.36.12.88]) with mapi id 14.03.0123.003; Thu, 21 Nov 2013 11:08:13 -0600 From: "Fred Baker (fred)" To: Tero Kivinen Thread-Topic: [dir-coord] Requesting SECDIR review of draft-ietf-v6ops-balanced-ipv6-security Thread-Index: AQHO5rv66Cg8gqO5nU+0E3kf4ta10powT6EA Date: Thu, 21 Nov 2013 17:08:12 +0000 Message-ID: <7CEA2D2B-E583-480A-839C-5F6A959E4B30@cisco.com> References: <16F5011B-A87C-4BE3-8AA4-F47C13E830F6@cisco.com> <21134.1987.918612.52011@fireball.kivinen.iki.fi> In-Reply-To: <21134.1987.918612.52011@fireball.kivinen.iki.fi> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.19.64.121] Content-Type: multipart/signed; boundary="Apple-Mail=_0CE65264-DC3E-458E-8E06-94FCBEFB6BC7"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: joel jaeggli , "" , "dir-coord@ietf.org" , John Brzozowski Subject: Re: [dir-coord] Requesting SECDIR review of draft-ietf-v6ops-balanced-ipv6-security X-BeenThere: dir-coord@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "This is an e-mail alias for the organisers of IETF directorates." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2013 17:08:22 -0000 --Apple-Mail=_0CE65264-DC3E-458E-8E06-94FCBEFB6BC7 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On Nov 21, 2013, at 5:16 AM, Tero Kivinen wrote: > Fred Baker (fred) writes: >> This document is right now in WGLC. Some points have been raised, >> which I imagine will warrant a revision. However, there are a lot of >> claims being made in and about the draft to the effect that it is a >> security solution that Swisscom has deployed and sees no problems >> with, and therefore the IETF should bless it as a general firewall >> solution. I would appreciate a timely review from the Security >> Directorate that would help us know whether it is likely to pass >> muster with the IESG, and if not, what types problems the >> directorate sees and what types of solutions would be acceptable. > > This was assigned to Paul Hoffman today, I put the deadline for next > week on the review. Thanks > -- > kivinen@iki.fi --Apple-Mail=_0CE65264-DC3E-458E-8E06-94FCBEFB6BC7 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iD8DBQFSjj38bjEdbHIsm0MRAoDiAKDxOE5ZszSRVnLGUC8nkQUIpT3elACgyYxo CZCUBMgoAZ4aRb8FHYqgfVQ= =xKJc -----END PGP SIGNATURE----- --Apple-Mail=_0CE65264-DC3E-458E-8E06-94FCBEFB6BC7--