From acmorton@att.com Sat Feb 2 14:50:47 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3C39821F84CA for ; Sat, 2 Feb 2013 14:50:46 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -105.808 X-Spam-Level: X-Spam-Status: No, score=-105.808 tagged_above=-999 required=5 tests=[AWL=-0.449, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_LWSHORTT=1.24, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tbEVKiq3Ezct for ; Sat, 2 Feb 2013 14:50:44 -0800 (PST) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 8430621F84C9 for ; Sat, 2 Feb 2013 14:50:39 -0800 (PST) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 8C5ED12050D; Sat, 2 Feb 2013 17:52:14 -0500 (EST) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-green.research.att.com (Postfix) with ESMTP id CC870E36DA; Sat, 2 Feb 2013 17:44:55 -0500 (EST) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Sat, 2 Feb 2013 17:50:38 -0500 From: "MORTON JR., ALFRED (AL)" To: Benoit Claise , "pm-dir@ietf.org" Date: Sat, 2 Feb 2013 17:50:36 -0500 Thread-Topic: [pm-dir] Request for an RFC 6369 review of draft-ietf-xrblock-rtcp-xr-summary-stat-06 Thread-Index: Ac39SfH6KY+SZ7GIQRCRDt3RqdrptQER1seg Message-ID: References: <50F93070.7040107@ericsson.com> <51057B67.7090205@cisco.com> In-Reply-To: <51057B67.7090205@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "xrblock-chairs@tools.ietf.org" , "draft-ietf-xrblock-rtcp-xr-summary-stat@tools.ietf.org" , Gonzalo Camarillo Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-xrblock-rtcp-xr-summary-stat-06 X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Feb 2013 22:50:47 -0000 Benoit, Since there is a short term deadline=20 (the Last Call has ended on Feb 1)=20 I'll provide the PM Dir review. Al -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Since the draft describes reporting formats and (apparently) does not=20 intend to define new metrics, much of 6390 guidance is out of scope. However, see the comments on section 4.1.2 below regarding metric definitio= ns. Comments: Section 1.1 of the Intro says: This draft defines three new block types to augment those defined in [RFC3611] for use in a range of RTP applications. ok so far, but the next paragraph is hard to decipher, perhaps a simple numbered list of the new blocks would help: 1. Burst/Gap Loss Summary Statistics Metrics Block 2. Burst/Gap Discard Summary Statistics Metrics Block 3. Frame Impairment Statistics Summary Metrics Block back to the existing text: The first two block types support the reporting of burst gap loss/ discard summary statistics including packet loss/discard proportion, mean and variance and belong to the class of transport-related end system metrics defined in [RFC6792]. These two blocks are intended to be used in conjunction with information from the Burst Gap Loss Metrics Block or Burst Gap Discard Metrics Block, and on which these two block therefore depend. =20 add a reference to [RFC3611] for these two blocks. The metrics in the Burst Gap Loss block or Burst Gap Discard Metrics Block can be used independently of the metrics defined in the first two blocks. That's obvious, because anyone implementing RFC3611 is already doing it. The authors appear to have avoided using "video" as an adjective when=20 referring to video frames throughout the memo. For example: The third block supports the reporting of detailed statistics for each frame type, including the number of frames received, lost and discarded of each frame type in the Group of Pictures (GOP) and additional data allowing the calculation of statistical parameters (e.g.,the proportion of each frame type impaired by packet loss and discard). The metrics defined in this block belong to the class of application layer metrics defined in [RFC6792]. These are all video frames above, no? At least say so once. Section 2.1, on Terminology, says Picture Type Picture Types used in the different video algorithms are composed of the Key frame and Derived frames. The Key frame is also called a reference frame and used as a reference for predicting other pictures. It is coded without prediction from other pictures. Derived frames are derived from a Key frame using a prediction algorithm. At first this seems quite generic, but the definition may fail when one sli= ce of the video frame is independently coded, and other slices are predictivel= y coded (this has been done to smooth the video rate). Is any video frame tha= t includes independent or reference coding a Key frame? Section 3.1.2 says: Burst Loss Rate: 16 bits The fraction of packets lost during bursts since the beginning of reception, expressed as a fixed point number with the binary point at the left edge of the field. This value is calculated by dividing Packets Loss in Bursts by Total Packets expected in Bursts as follows: Packets Loss in Bursts / Total Packets expected in Bursts How is a burst loss ratio =3D 1.0 reported? All the digits are to the=20 right of the decimal place. Burst Duration Mean:16bits The mean burst duration is obtained as the quotient: mean =3D Sum of Burst Durations / Number of Bursts How is Divide by Zero handled? Burst Duration Variance:16bits The variance of the burst duration is obtained using the standard result: var =3D ( Sum of Squares of Burst Durations - Number of Bursts * mean^2 ) / (Number of Bursts - 1) How is Divide by Zero handled? (e.g., Number of Bursts =3D 1) Section 4.1.2 says Number of full frames lost (lost_full_frames): 32 bits If one frame is completely lost, this frame is regarded as one lost full frame. The lost_full_frames is equivalent to the number of full frames lost in the above sequence number interval. Is this a metric definition, fully lost video frames?=20 How is this event detected so it can be counted? Is there a reference for this metric available? Number of partial frames lost (lost_partial_frames): 32 bits If one frame is partially lost, this frame is regarded as one lost fractional frame. The value of the lost_partial_frames field is equivalent to the number of partial frames lost in the above sequence number interval. Is this a metric definition, partially lost video frames?=20 How is this event detected so it can be counted? Is there a reference for this metric available? -=3D-=3D-=3D-=3D-=3D-=3D- > -----Original Message----- > From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf > Of Benoit Claise > Sent: Sunday, January 27, 2013 2:09 PM > To: pm-dir@ietf.org > Cc: xrblock-chairs@tools.ietf.org; draft-ietf-xrblock-rtcp-xr-summary- > stat@tools.ietf.org; Gonzalo Camarillo > Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf- > xrblock-rtcp-xr-summary-stat-06 >=20 > Dear all, >=20 > Here is an official request for the RFC 6390 review. > Who would like to volunteer? >=20 > Regards, Benoit > > Hi Benoit, > > > > as suggested by Dan in his PROTO writeup, I would like to request the > > performance metrics directorate to perform an RFC 6390 review of the > > following draft as part of its IETF LC: > > > > https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary- > stat/ > > > > Could you please arrange such a review? > > > > Thanks, > > > > Gonzalo > > > > >=20 > _______________________________________________ > pm-dir mailing list > pm-dir@ietf.org > https://www.ietf.org/mailman/listinfo/pm-dir From bclaise@cisco.com Mon Feb 4 01:12:28 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CFE1921F842B for ; Mon, 4 Feb 2013 01:12:27 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.578 X-Spam-Level: X-Spam-Status: No, score=-10.578 tagged_above=-999 required=5 tests=[AWL=0.022, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BsNiY-kv2WSY for ; Mon, 4 Feb 2013 01:12:27 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id EED1421F841F for ; Mon, 4 Feb 2013 01:12:26 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r149CP6w006442 for ; Mon, 4 Feb 2013 10:12:25 +0100 (CET) Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r149CAk7023576 for ; Mon, 4 Feb 2013 10:12:20 +0100 (CET) Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r149C60h002615 for pm-dir@ietf.org; Mon, 4 Feb 2013 10:12:06 +0100 (CET) Date: Mon, 4 Feb 2013 10:12:06 +0100 From: Benoit Claise To: pm-dir@ietf.org Message-ID: <20130204091206.GA2586@sweet-brew-5.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 09:12:28 -0000 Dear all, This is an automatically generated email. It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. Regards, Benoit =========================================================== Normative References -------------------- draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active Informative References ---------------------- draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-concsec-03 Active draft-ietf-xrblock-rtcp-xr-decodability-07 Active draft-ietf-xrblock-rtcp-xr-discard-11 Active draft-ietf-xrblock-rtcp-xr-jb-07 Active draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active draft-ietf-xrblock-rtcp-xr-qoe-03 Active From bclaise@cisco.com Mon Feb 4 03:53:45 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 104F221F8468 for ; Mon, 4 Feb 2013 03:53:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.58 X-Spam-Level: X-Spam-Status: No, score=-10.58 tagged_above=-999 required=5 tests=[AWL=0.019, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xqllpxJB1bxS for ; Mon, 4 Feb 2013 03:53:44 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 433A421F85EB for ; Mon, 4 Feb 2013 03:53:44 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r14Brgjr025626 for ; Mon, 4 Feb 2013 12:53:43 +0100 (CET) Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r14BrRx2018697 for ; Mon, 4 Feb 2013 12:53:37 +0100 (CET) Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r14BrIBP009088 for pm-dir@ietf.org; Mon, 4 Feb 2013 12:53:18 +0100 (CET) Date: Mon, 4 Feb 2013 12:53:18 +0100 From: Benoit Claise To: pm-dir@ietf.org Message-ID: <20130204115318.GA9086@sweet-brew-5.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 11:53:45 -0000 Dear all, This is an automatically generated email. It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. It also lists all the IETF internet-drafts that contain "performance metric". Regards, Benoit =========================================================== Normative References -------------------- draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active Informative References ---------------------- draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-concsec-03 Active draft-ietf-xrblock-rtcp-xr-decodability-07 Active draft-ietf-xrblock-rtcp-xr-discard-11 Active draft-ietf-xrblock-rtcp-xr-jb-07 Active draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active draft-ietf-xrblock-rtcp-xr-qoe-03 Active drafts containing performance metric ------------------------------------ draft-ietf-alto-deployments-05 Active draft-ietf-alto-protocol-13 In IESG processing - ID Tracker state draft-ietf-ippm-rate-problem-02 Active draft-ietf-ippm-testplan-rfc2680-01 Active draft-ietf-karp-threats-reqs-07 In IESG processing - ID Tracker state draft-ietf-manet-olsrv2-mib-05 Active draft-ietf-manet-smf-mib-06 Active draft-ietf-mmusic-media-loopback-27 In IESG processing - ID Tracker state draft-ietf-nvo3-framework-01 Active draft-ietf-opsawg-oam-overview-08 In IESG processing - ID Tracker state draft-ietf-rtcweb-rtp-usage-05 Active draft-ietf-rtgwg-cl-framework-02 Active draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-concsec-03 Active draft-ietf-xrblock-rtcp-xr-decodability-07 Active draft-ietf-xrblock-rtcp-xr-discard-11 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active draft-ietf-xrblock-rtcp-xr-jb-07 Active draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active draft-ietf-xrblock-rtcp-xr-qoe-03 Active draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-synchronization-02 Active From bclaise@cisco.com Mon Feb 4 04:07:41 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F175121F846E for ; Mon, 4 Feb 2013 04:07:40 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.581 X-Spam-Level: X-Spam-Status: No, score=-10.581 tagged_above=-999 required=5 tests=[AWL=0.018, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m0l-WXjr3XQz for ; Mon, 4 Feb 2013 04:07:40 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id D147C21F8468 for ; Mon, 4 Feb 2013 04:07:39 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r14C7aNg027158; Mon, 4 Feb 2013 13:07:36 +0100 (CET) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r14C6u31000990; Mon, 4 Feb 2013 13:07:06 +0100 (CET) Message-ID: <510FA460.3060102@cisco.com> Date: Mon, 04 Feb 2013 13:06:56 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: pm-dir@ietf.org References: <20130204115318.GA9086@sweet-brew-5.cisco.com> In-Reply-To: <20130204115318.GA9086@sweet-brew-5.cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alan DeKok Subject: Re: [pm-dir] Performance metrics doctors generated email - new version. X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 12:07:41 -0000 Dear all, This is a new version of the script, provided by Alan Dekok. Many thanks Alan. The important improvement is that we also list the drafts that contain a specific keyword. In this case, "performance metric". We could add some more... With this new script, we can then engage sooner with the people who don't know about this directorate and RFC 6390. Potential small optimization for the script: remove the drafts that contain "performance metric" and that already reference RFC 6390 Regards, Benoit > Dear all, > > This is an automatically generated email. > It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. > It also lists all the IETF internet-drafts that contain "performance metric". > > Regards, Benoit > > =========================================================== > > Normative References > -------------------- > draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active > > Informative References > ---------------------- > draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-concsec-03 Active > draft-ietf-xrblock-rtcp-xr-decodability-07 Active > draft-ietf-xrblock-rtcp-xr-discard-11 Active > draft-ietf-xrblock-rtcp-xr-jb-07 Active > draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active > draft-ietf-xrblock-rtcp-xr-qoe-03 Active > > drafts containing performance metric > ------------------------------------ > draft-ietf-alto-deployments-05 Active > draft-ietf-alto-protocol-13 In IESG processing - ID Tracker state > draft-ietf-ippm-rate-problem-02 Active > draft-ietf-ippm-testplan-rfc2680-01 Active > draft-ietf-karp-threats-reqs-07 In IESG processing - ID Tracker state > draft-ietf-manet-olsrv2-mib-05 Active > draft-ietf-manet-smf-mib-06 Active > draft-ietf-mmusic-media-loopback-27 In IESG processing - ID Tracker state > draft-ietf-nvo3-framework-01 Active > draft-ietf-opsawg-oam-overview-08 In IESG processing - ID Tracker state > draft-ietf-rtcweb-rtp-usage-05 Active > draft-ietf-rtgwg-cl-framework-02 Active > draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-concsec-03 Active > draft-ietf-xrblock-rtcp-xr-decodability-07 Active > draft-ietf-xrblock-rtcp-xr-discard-11 Active > draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active > draft-ietf-xrblock-rtcp-xr-jb-07 Active > draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active > draft-ietf-xrblock-rtcp-xr-qoe-03 Active > draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-synchronization-02 Active > _______________________________________________ > pm-dir mailing list > pm-dir@ietf.org > https://www.ietf.org/mailman/listinfo/pm-dir > > From paitken@cisco.com Mon Feb 4 04:33:26 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 16FA121F86A3 for ; Mon, 4 Feb 2013 04:33:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hik7ft0MXLOx for ; Mon, 4 Feb 2013 04:33:25 -0800 (PST) Received: from ams-iport-3.cisco.com (ams-iport-3.cisco.com [144.254.224.146]) by ietfa.amsl.com (Postfix) with ESMTP id EB5B321F869F for ; Mon, 4 Feb 2013 04:33:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4209; q=dns/txt; s=iport; t=1359981205; x=1361190805; h=message-id:date:from:mime-version:to:subject:references: in-reply-to:content-transfer-encoding; bh=T6v/Y7C2dr6ploQfTnpepuyvs4HVB1OOyEL5YLoa6ZU=; b=eH7AqFL7E8Lzieu3CVHRSkr+dQPBiD+N2K3f1kHqbdWE9Iodnyn/BrQM 4KWD6CgcvdDVuAF2/RdDZxCbfSZLlwVbQIPKbKgYBHDrJGcRz8TvhL427 SFYGbQlsQcKZw+ggIBpeek3n0xBrsVphfMc8IztX/Iq1eBpEZy0uBV05u Q=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AhwFAF2qD1GQ/khL/2dsb2JhbABFgzm8AhZzgh8BAQEEAQEBJBE2ChELGAkWDwkDAgECARUwEwYCAQGIDQy9PQSNGoQ4A5YfhXCKYYJ8gW8 X-IronPort-AV: E=Sophos;i="4.84,598,1355097600"; d="scan'208";a="11570967" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-3.cisco.com with ESMTP; 04 Feb 2013 12:33:17 +0000 Received: from [10.61.85.89] (ams3-vpn-dhcp5466.cisco.com [10.61.85.89]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r14CXH0L007552 for ; Mon, 4 Feb 2013 12:33:17 GMT Message-ID: <510FAA8F.4030604@cisco.com> Date: Mon, 04 Feb 2013 12:33:19 +0000 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2 MIME-Version: 1.0 To: pm-dir@ietf.org References: <20130204115318.GA9086@sweet-brew-5.cisco.com> In-Reply-To: <20130204115318.GA9086@sweet-brew-5.cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 12:33:26 -0000 Benoit, All, How is pm-dir tracking these drafts? ie, how do we know whether someone is currently reviewing, or has already reviewed, each of these drafts? How are reviewers selected? By relevance, skill, luck, or misfortune? Also, it'd be useful to know whether the previous version was already reviewed, and what the feedback was: * issues were raised, recommendations were made, changes were expected -> some re-review should be done * no issues were raised -> a quick review of the delta may be sufficient, to check for any new issues * previous version wasn't reviewed -> a thorough review may be required. I'm thinking it'd be useful to have a pm-dir wiki listing each of the drafts below. eg, one page per draft, with one section per version, containing reviewer comments pertinent to that version, even if the comments simply say, "reviewed, no issues". Hopefully I missed something, and most (if not all?) of this is already in place? :-) Thanks, P. On 04/02/13 11:53, Benoit Claise wrote: > Dear all, > > This is an automatically generated email. > It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. > It also lists all the IETF internet-drafts that contain "performance metric". > > Regards, Benoit > > =========================================================== > > Normative References > -------------------- > draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active > > Informative References > ---------------------- > draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-concsec-03 Active > draft-ietf-xrblock-rtcp-xr-decodability-07 Active > draft-ietf-xrblock-rtcp-xr-discard-11 Active > draft-ietf-xrblock-rtcp-xr-jb-07 Active > draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active > draft-ietf-xrblock-rtcp-xr-qoe-03 Active > > drafts containing performance metric > ------------------------------------ > draft-ietf-alto-deployments-05 Active > draft-ietf-alto-protocol-13 In IESG processing - ID Tracker state > draft-ietf-ippm-rate-problem-02 Active > draft-ietf-ippm-testplan-rfc2680-01 Active > draft-ietf-karp-threats-reqs-07 In IESG processing - ID Tracker state > draft-ietf-manet-olsrv2-mib-05 Active > draft-ietf-manet-smf-mib-06 Active > draft-ietf-mmusic-media-loopback-27 In IESG processing - ID Tracker state > draft-ietf-nvo3-framework-01 Active > draft-ietf-opsawg-oam-overview-08 In IESG processing - ID Tracker state > draft-ietf-rtcweb-rtp-usage-05 Active > draft-ietf-rtgwg-cl-framework-02 Active > draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-concsec-03 Active > draft-ietf-xrblock-rtcp-xr-decodability-07 Active > draft-ietf-xrblock-rtcp-xr-discard-11 Active > draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active > draft-ietf-xrblock-rtcp-xr-jb-07 Active > draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active > draft-ietf-xrblock-rtcp-xr-qoe-03 Active > draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing - ID Tracker state > draft-ietf-xrblock-rtcp-xr-synchronization-02 Active > _______________________________________________ > pm-dir mailing list > pm-dir@ietf.org > https://www.ietf.org/mailman/listinfo/pm-dir From bclaise@cisco.com Mon Feb 4 04:39:25 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83E0C21F8625 for ; Mon, 4 Feb 2013 04:39:25 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.583 X-Spam-Level: X-Spam-Status: No, score=-10.583 tagged_above=-999 required=5 tests=[AWL=0.016, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4syxs-qprves for ; Mon, 4 Feb 2013 04:39:24 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 78FFA21F85FD for ; Mon, 4 Feb 2013 04:39:24 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r14Cbbg1000266 for ; Mon, 4 Feb 2013 13:37:37 +0100 (CET) Received: from [10.60.67.85] (ams-bclaise-8914.cisco.com [10.60.67.85]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r14CarcO024548; Mon, 4 Feb 2013 13:36:58 +0100 (CET) Message-ID: <510FAB65.4080505@cisco.com> Date: Mon, 04 Feb 2013 13:36:53 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Paul Aitken References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FAA8F.4030604@cisco.com> In-Reply-To: <510FAA8F.4030604@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pm-dir@ietf.org Subject: Re: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 12:39:25 -0000 Hi Paul, Yes, all these questions are important. Yes a WIKI is the right solution IMHO. Regards, Benoit > Benoit, All, > > How is pm-dir tracking these drafts? ie, how do we know whether > someone is currently reviewing, or has already reviewed, each of these > drafts? > > How are reviewers selected? By relevance, skill, luck, or misfortune? > > Also, it'd be useful to know whether the previous version was already > reviewed, and what the feedback was: > > * issues were raised, recommendations were made, changes were > expected -> some re-review should be done > > * no issues were raised -> a quick review of the delta may be > sufficient, to check for any new issues > > * previous version wasn't reviewed -> a thorough review may be > required. > > > I'm thinking it'd be useful to have a pm-dir wiki listing each of the > drafts below. eg, one page per draft, with one section per version, > containing reviewer comments pertinent to that version, even if the > comments simply say, "reviewed, no issues". > > Hopefully I missed something, and most (if not all?) of this is > already in place? :-) > > Thanks, > P. > > > On 04/02/13 11:53, Benoit Claise wrote: >> Dear all, >> >> This is an automatically generated email. >> It lists the IETF internet-drafts that reference the PMOL RFC 6390, >> as a normative or informative reference. >> It also lists all the IETF internet-drafts that contain "performance >> metric". >> >> Regards, Benoit >> >> =========================================================== >> >> Normative References >> -------------------- >> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >> Informative References >> ---------------------- >> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >> - ID Tracker state >> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >> - ID Tracker state >> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >> draft-ietf-xrblock-rtcp-xr-discard-11 Active >> draft-ietf-xrblock-rtcp-xr-jb-07 Active >> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >> >> drafts containing performance metric >> ------------------------------------ >> draft-ietf-alto-deployments-05 Active >> draft-ietf-alto-protocol-13 In IESG processing >> - ID Tracker state >> draft-ietf-ippm-rate-problem-02 Active >> draft-ietf-ippm-testplan-rfc2680-01 Active >> draft-ietf-karp-threats-reqs-07 In IESG processing >> - ID Tracker state >> draft-ietf-manet-olsrv2-mib-05 Active >> draft-ietf-manet-smf-mib-06 Active >> draft-ietf-mmusic-media-loopback-27 In IESG processing >> - ID Tracker state >> draft-ietf-nvo3-framework-01 Active >> draft-ietf-opsawg-oam-overview-08 In IESG processing >> - ID Tracker state >> draft-ietf-rtcweb-rtp-usage-05 Active >> draft-ietf-rtgwg-cl-framework-02 Active >> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >> - ID Tracker state >> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >> - ID Tracker state >> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >> draft-ietf-xrblock-rtcp-xr-discard-11 Active >> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >> draft-ietf-xrblock-rtcp-xr-jb-07 Active >> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >> draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing >> - ID Tracker state >> draft-ietf-xrblock-rtcp-xr-synchronization-02 Active >> _______________________________________________ >> pm-dir mailing list >> pm-dir@ietf.org >> https://www.ietf.org/mailman/listinfo/pm-dir > > _______________________________________________ > pm-dir mailing list > pm-dir@ietf.org > https://www.ietf.org/mailman/listinfo/pm-dir > > From paitken@cisco.com Mon Feb 4 04:49:55 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C6A6A21F8615 for ; Mon, 4 Feb 2013 04:49:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.599 X-Spam-Level: X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6q1n9C0M96ib for ; Mon, 4 Feb 2013 04:49:55 -0800 (PST) Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 87EA121F85EB for ; Mon, 4 Feb 2013 04:49:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4836; q=dns/txt; s=iport; t=1359982194; x=1361191794; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=wEUwK6jGTMU3Y1ARMY94RuhRLmPmaTw2h6ZNCg5j7ww=; b=GmI0XpPUE9b+Eny1dyfvtxvg9mcE/GBi1/2nxTeNrvqjdnb0R1IiWTrw GzW24LATmCCb9WcG8pchJUlooIxP82fwVqBbwNbc9NYJN5/zsz9PNh6dD 9BhaMCaMz06ZRLJ1H5sonD7EHNunNEDZ/p/yWy4zS6DM+y1SA+Qni0oDs 4=; X-IronPort-AV: E=Sophos;i="4.84,600,1355097600"; d="scan'208";a="150113497" Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-1.cisco.com with ESMTP; 04 Feb 2013 12:49:53 +0000 Received: from [10.61.85.89] (ams3-vpn-dhcp5466.cisco.com [10.61.85.89]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r14Cnrfo010581; Mon, 4 Feb 2013 12:49:53 GMT Message-ID: <510FAE73.1080901@cisco.com> Date: Mon, 04 Feb 2013 12:49:55 +0000 From: Paul Aitken User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2 MIME-Version: 1.0 To: Benoit Claise References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FAA8F.4030604@cisco.com> <510FAB65.4080505@cisco.com> In-Reply-To: <510FAB65.4080505@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pm-dir@ietf.org Subject: Re: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 12:49:55 -0000 Benoit, > Yes, all these questions are important. > Yes a WIKI is the right solution IMHO. Great. Where can it be hosted? eg, is http://trac.tools.ietf.org/wg/pmol/trac/wiki a good starting place? Or is there a pm-dir page? P. >> Benoit, All, >> >> How is pm-dir tracking these drafts? ie, how do we know whether >> someone is currently reviewing, or has already reviewed, each of >> these drafts? >> >> How are reviewers selected? By relevance, skill, luck, or misfortune? >> >> Also, it'd be useful to know whether the previous version was already >> reviewed, and what the feedback was: >> >> * issues were raised, recommendations were made, changes were >> expected -> some re-review should be done >> >> * no issues were raised -> a quick review of the delta may be >> sufficient, to check for any new issues >> >> * previous version wasn't reviewed -> a thorough review may be >> required. >> >> >> I'm thinking it'd be useful to have a pm-dir wiki listing each of the >> drafts below. eg, one page per draft, with one section per version, >> containing reviewer comments pertinent to that version, even if the >> comments simply say, "reviewed, no issues". >> >> Hopefully I missed something, and most (if not all?) of this is >> already in place? :-) >> >> Thanks, >> P. >> >> >> On 04/02/13 11:53, Benoit Claise wrote: >>> Dear all, >>> >>> This is an automatically generated email. >>> It lists the IETF internet-drafts that reference the PMOL RFC 6390, >>> as a normative or informative reference. >>> It also lists all the IETF internet-drafts that contain "performance >>> metric". >>> >>> Regards, Benoit >>> >>> =========================================================== >>> >>> Normative References >>> -------------------- >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>> Informative References >>> ---------------------- >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>> >>> drafts containing performance metric >>> ------------------------------------ >>> draft-ietf-alto-deployments-05 Active >>> draft-ietf-alto-protocol-13 In IESG processing >>> - ID Tracker state >>> draft-ietf-ippm-rate-problem-02 Active >>> draft-ietf-ippm-testplan-rfc2680-01 Active >>> draft-ietf-karp-threats-reqs-07 In IESG processing >>> - ID Tracker state >>> draft-ietf-manet-olsrv2-mib-05 Active >>> draft-ietf-manet-smf-mib-06 Active >>> draft-ietf-mmusic-media-loopback-27 In IESG processing >>> - ID Tracker state >>> draft-ietf-nvo3-framework-01 Active >>> draft-ietf-opsawg-oam-overview-08 In IESG processing >>> - ID Tracker state >>> draft-ietf-rtcweb-rtp-usage-05 Active >>> draft-ietf-rtgwg-cl-framework-02 Active >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>> draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-synchronization-02 Active >>> _______________________________________________ >>> pm-dir mailing list >>> pm-dir@ietf.org >>> https://www.ietf.org/mailman/listinfo/pm-dir >> >> _______________________________________________ >> pm-dir mailing list >> pm-dir@ietf.org >> https://www.ietf.org/mailman/listinfo/pm-dir >> >> From bill.wu@huawei.com Mon Feb 4 00:40:51 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 29BE121F855C for ; Mon, 4 Feb 2013 00:40:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -4.226 X-Spam-Level: X-Spam-Status: No, score=-4.226 tagged_above=-999 required=5 tests=[AWL=-0.620, BAYES_00=-2.599, MIME_BASE64_TEXT=1.753, RCVD_IN_DNSWL_MED=-4, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6KkFkp7w8VCu for ; Mon, 4 Feb 2013 00:40:50 -0800 (PST) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) by ietfa.amsl.com (Postfix) with ESMTP id 4DC2021F855F for ; Mon, 4 Feb 2013 00:40:49 -0800 (PST) Received: from 172.18.7.190 (EHLO lhreml204-edg.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.5-GA FastPath queued) with ESMTP id AOD28273; Mon, 04 Feb 2013 08:40:47 +0000 (GMT) Received: from LHREML402-HUB.china.huawei.com (10.201.5.241) by lhreml204-edg.china.huawei.com (172.18.7.223) with Microsoft SMTP Server (TLS) id 14.1.323.7; Mon, 4 Feb 2013 08:39:58 +0000 Received: from SZXEML403-HUB.china.huawei.com (10.82.67.35) by lhreml402-hub.china.huawei.com (10.201.5.241) with Microsoft SMTP Server (TLS) id 14.1.323.7; Mon, 4 Feb 2013 08:40:46 +0000 Received: from w53375 (10.138.41.149) by szxeml403-hub.china.huawei.com (10.82.67.35) with Microsoft SMTP Server (TLS) id 14.1.323.7; Mon, 4 Feb 2013 16:40:32 +0800 Message-ID: From: Qin Wu To: "MORTON JR., ALFRED (AL)" , Benoit Claise , References: <50F93070.7040107@ericsson.com> <51057B67.7090205@cisco.com> Date: Mon, 4 Feb 2013 16:40:31 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: base64 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 X-Originating-IP: [10.138.41.149] X-CFilter-Loop: Reflected X-Mailman-Approved-At: Mon, 04 Feb 2013 04:55:26 -0800 Cc: xrblock-chairs@tools.ietf.org, draft-ietf-xrblock-rtcp-xr-summary-stat@tools.ietf.org, Gonzalo Camarillo Subject: Re: [pm-dir] Request for an RFC 6369 review ofdraft-ietf-xrblock-rtcp-xr-summary-stat-06 X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 08:40:51 -0000 SGksQWw6DQpUaGFuayBmb3IgeW91ciB2YWx1YWJsZSByZXZpZXcuIFdlIHdpbGwgYWRkcmVzcyB5 b3VyIGNvbW1lbnRzIGluIHRoZSB1cGRhdGUuDQpQbGVhc2Ugc2VlIG15IHJlcGx5IGJlbG93Lg0K LS0tLS0gT3JpZ2luYWwgTWVzc2FnZSAtLS0tLSANCkZyb206ICJNT1JUT04gSlIuLCBBTEZSRUQg KEFMKSIgPGFjbW9ydG9uQGF0dC5jb20+DQpUbzogIkJlbm9pdCBDbGFpc2UiIDxiY2xhaXNlQGNp c2NvLmNvbT47IDxwbS1kaXJAaWV0Zi5vcmc+DQpDYzogPHhyYmxvY2stY2hhaXJzQHRvb2xzLmll dGYub3JnPjsgPGRyYWZ0LWlldGYteHJibG9jay1ydGNwLXhyLXN1bW1hcnktc3RhdEB0b29scy5p ZXRmLm9yZz47ICJHb256YWxvIENhbWFyaWxsbyIgPEdvbnphbG8uQ2FtYXJpbGxvQGVyaWNzc29u LmNvbT4NClNlbnQ6IFN1bmRheSwgRmVicnVhcnkgMDMsIDIwMTMgNjo1MCBBTQ0KU3ViamVjdDog UkU6IFtwbS1kaXJdIFJlcXVlc3QgZm9yIGFuIFJGQyA2MzY5IHJldmlldyBvZmRyYWZ0LWlldGYt eHJibG9jay1ydGNwLXhyLXN1bW1hcnktc3RhdC0wNg0KDQoNCkJlbm9pdCwNCg0KU2luY2UgdGhl cmUgaXMgYSBzaG9ydCB0ZXJtIGRlYWRsaW5lIA0KKHRoZSBMYXN0IENhbGwgaGFzIGVuZGVkIG9u IEZlYiAxKSANCkknbGwgcHJvdmlkZSB0aGUgUE0gRGlyIHJldmlldy4NCg0KQWwNCg0KLT0tPS09 LT0tPS09LT0tPS09LT0tPS09LT0tPS09LT0tPS0NClNpbmNlIHRoZSBkcmFmdCBkZXNjcmliZXMg cmVwb3J0aW5nIGZvcm1hdHMgYW5kIChhcHBhcmVudGx5KSBkb2VzIG5vdCANCmludGVuZCB0byBk ZWZpbmUgbmV3IG1ldHJpY3MsIG11Y2ggb2YgNjM5MCBndWlkYW5jZSBpcyBvdXQgb2Ygc2NvcGUu DQpIb3dldmVyLCBzZWUgdGhlIGNvbW1lbnRzIG9uIHNlY3Rpb24gNC4xLjIgYmVsb3cgcmVnYXJk aW5nIG1ldHJpYyBkZWZpbml0aW9ucy4NCg0KQ29tbWVudHM6DQoNClNlY3Rpb24gMS4xIG9mIHRo ZSBJbnRybyBzYXlzOg0KDQogICBUaGlzIGRyYWZ0IGRlZmluZXMgdGhyZWUgbmV3IGJsb2NrIHR5 cGVzIHRvIGF1Z21lbnQgdGhvc2UgZGVmaW5lZCBpbg0KICAgW1JGQzM2MTFdIGZvciB1c2UgaW4g YSByYW5nZSBvZiBSVFAgYXBwbGljYXRpb25zLg0KDQpvayBzbyBmYXIsIGJ1dCB0aGUgbmV4dCBw YXJhZ3JhcGggaXMgaGFyZCB0byBkZWNpcGhlciwgcGVyaGFwcyBhIHNpbXBsZQ0KbnVtYmVyZWQg bGlzdCBvZiB0aGUgbmV3IGJsb2NrcyB3b3VsZCBoZWxwOg0KDQogICAgMS4gQnVyc3QvR2FwIExv c3MgU3VtbWFyeSBTdGF0aXN0aWNzIE1ldHJpY3MgQmxvY2sNCiAgICAyLiBCdXJzdC9HYXAgRGlz Y2FyZCBTdW1tYXJ5IFN0YXRpc3RpY3MgTWV0cmljcyBCbG9jaw0KICAgIDMuIEZyYW1lIEltcGFp cm1lbnQgU3RhdGlzdGljcyBTdW1tYXJ5IE1ldHJpY3MgQmxvY2sNCg0KW1Fpbl06IEdvb2Qgc3Vn Z2VzdGlvbi4gVGhhbmtzLg0KDQpiYWNrIHRvIHRoZSBleGlzdGluZyB0ZXh0Og0KICAgVGhlIGZp cnN0IHR3byBibG9jayB0eXBlcyBzdXBwb3J0IHRoZSByZXBvcnRpbmcgb2YgYnVyc3QgZ2FwIGxv c3MvDQogICBkaXNjYXJkIHN1bW1hcnkgc3RhdGlzdGljcyBpbmNsdWRpbmcgcGFja2V0IGxvc3Mv ZGlzY2FyZCBwcm9wb3J0aW9uLA0KICAgbWVhbiBhbmQgdmFyaWFuY2UgYW5kIGJlbG9uZyB0byB0 aGUgY2xhc3Mgb2YgdHJhbnNwb3J0LXJlbGF0ZWQgZW5kDQogICBzeXN0ZW0gbWV0cmljcyBkZWZp bmVkIGluIFtSRkM2NzkyXS4gIFRoZXNlIHR3byBibG9ja3MgYXJlIGludGVuZGVkDQogICB0byBi ZSB1c2VkIGluIGNvbmp1bmN0aW9uIHdpdGggaW5mb3JtYXRpb24gZnJvbSB0aGUgQnVyc3QgR2Fw IExvc3MNCiAgIE1ldHJpY3MgQmxvY2sgb3IgQnVyc3QgR2FwIERpc2NhcmQgTWV0cmljcyBCbG9j aywgYW5kIG9uIHdoaWNoIHRoZXNlDQogICB0d28gYmxvY2sgdGhlcmVmb3JlIGRlcGVuZC4gIA0K YWRkIGEgcmVmZXJlbmNlIHRvIFtSRkMzNjExXSBmb3IgdGhlc2UgdHdvIGJsb2Nrcy4NCg0KW1Fp bl06IE9rYXkgYW5kIHdpbGwgYWRkIHRoYXQuDQoNCiAgIFRoZSBtZXRyaWNzIGluIHRoZSBCdXJz dCBHYXAgTG9zcyBibG9jaw0KICAgb3IgQnVyc3QgR2FwIERpc2NhcmQgTWV0cmljcyBCbG9jayBj YW4gYmUgdXNlZCBpbmRlcGVuZGVudGx5IG9mIHRoZQ0KICAgbWV0cmljcyBkZWZpbmVkIGluIHRo ZSBmaXJzdCB0d28gYmxvY2tzLg0KDQpUaGF0J3Mgb2J2aW91cywgYmVjYXVzZSBhbnlvbmUgaW1w bGVtZW50aW5nIFJGQzM2MTEgaXMgYWxyZWFkeSBkb2luZyBpdC4NCg0KW1Fpbl06IEFncmVlLg0K DQpUaGUgYXV0aG9ycyBhcHBlYXIgdG8gaGF2ZSBhdm9pZGVkIHVzaW5nICJ2aWRlbyIgYXMgYW4g YWRqZWN0aXZlIHdoZW4gDQpyZWZlcnJpbmcgdG8gdmlkZW8gZnJhbWVzIHRocm91Z2hvdXQgdGhl IG1lbW8uIEZvciBleGFtcGxlOg0KDQogICBUaGUgdGhpcmQgYmxvY2sgc3VwcG9ydHMgdGhlIHJl cG9ydGluZyBvZiBkZXRhaWxlZCBzdGF0aXN0aWNzIGZvcg0KICAgZWFjaCBmcmFtZSB0eXBlLCBp bmNsdWRpbmcgdGhlIG51bWJlciBvZiBmcmFtZXMgcmVjZWl2ZWQsIGxvc3QgYW5kDQogICBkaXNj YXJkZWQgb2YgZWFjaCBmcmFtZSB0eXBlIGluIHRoZSBHcm91cCBvZiBQaWN0dXJlcyAoR09QKSBh bmQNCiAgIGFkZGl0aW9uYWwgZGF0YSBhbGxvd2luZyB0aGUgY2FsY3VsYXRpb24gb2Ygc3RhdGlz dGljYWwgcGFyYW1ldGVycw0KICAgKGUuZy4sdGhlIHByb3BvcnRpb24gb2YgZWFjaCBmcmFtZSB0 eXBlIGltcGFpcmVkIGJ5IHBhY2tldCBsb3NzIGFuZA0KICAgZGlzY2FyZCkuICBUaGUgbWV0cmlj cyBkZWZpbmVkIGluIHRoaXMgYmxvY2sgYmVsb25nIHRvIHRoZSBjbGFzcyBvZg0KICAgYXBwbGlj YXRpb24gbGF5ZXIgbWV0cmljcyBkZWZpbmVkIGluIFtSRkM2NzkyXS4NCg0KVGhlc2UgYXJlIGFs bCB2aWRlbyBmcmFtZXMgYWJvdmUsIG5vPyAgQXQgbGVhc3Qgc2F5IHNvIG9uY2UuDQoNCltRaW5d OiBBZ3JlZSBhbmQgIHdpbGwgZml4IHRoaXMuIA0KDQpTZWN0aW9uIDIuMSwgb24gVGVybWlub2xv Z3ksIHNheXMNCg0KICAgIFBpY3R1cmUgVHlwZQ0KDQogICAgICBQaWN0dXJlIFR5cGVzIHVzZWQg aW4gdGhlIGRpZmZlcmVudCB2aWRlbyBhbGdvcml0aG1zIGFyZSBjb21wb3NlZA0KICAgICAgb2Yg dGhlIEtleSBmcmFtZSBhbmQgRGVyaXZlZCBmcmFtZXMuICBUaGUgS2V5IGZyYW1lIGlzIGFsc28g Y2FsbGVkDQogICAgICBhIHJlZmVyZW5jZSBmcmFtZSBhbmQgdXNlZCBhcyBhIHJlZmVyZW5jZSBm b3IgcHJlZGljdGluZyBvdGhlcg0KICAgICAgcGljdHVyZXMuICBJdCBpcyBjb2RlZCB3aXRob3V0 IHByZWRpY3Rpb24gZnJvbSBvdGhlciBwaWN0dXJlcy4NCiAgICAgIERlcml2ZWQgZnJhbWVzIGFy ZSBkZXJpdmVkIGZyb20gYSBLZXkgZnJhbWUgdXNpbmcgYSBwcmVkaWN0aW9uDQogICAgICBhbGdv cml0aG0uDQoNCkF0IGZpcnN0IHRoaXMgc2VlbXMgcXVpdGUgZ2VuZXJpYywgYnV0IHRoZSBkZWZp bml0aW9uIG1heSBmYWlsIHdoZW4gb25lIHNsaWNlDQpvZiB0aGUgdmlkZW8gZnJhbWUgaXMgaW5k ZXBlbmRlbnRseSBjb2RlZCwgYW5kIG90aGVyIHNsaWNlcyBhcmUgcHJlZGljdGl2ZWx5DQpjb2Rl ZCAodGhpcyBoYXMgYmVlbiBkb25lIHRvIHNtb290aCB0aGUgdmlkZW8gcmF0ZSkuIElzIGFueSB2 aWRlbyBmcmFtZSB0aGF0DQppbmNsdWRlcyBpbmRlcGVuZGVudCBvciByZWZlcmVuY2UgY29kaW5n IGEgS2V5IGZyYW1lPw0KDQpbUWluXTogT25lIGlzIGluZGVwZW5kZW50bHkgY29kZWQgYW5kIHRo ZSBvdGhlciBpcyBwcmVkaWN0aXZlbHkgY29kZWQsIHRoYXQgaXMgaG93IHRoZSBrZXkgZnJhbWUg DQpkaXN0aW5jdCBmcm9tdGhlICBkZXJpdmVkIGZyYW1lLiBPbmUgZXhhbXBsZSBvZiBLZXkgZnJh bWUgaXMgSSBmcmFtZSB3aGljaCBpcyBpbmRlcGVuZGVudGx5IGNvZGVkDQpyZWZlcmVuY2UgZnJh bWUuIElzIHRoZXJlIGFueSBleGNlcHRpb25hbCBjYXNlPw0KDQpTZWN0aW9uIDMuMS4yIHNheXM6 DQogICBCdXJzdCBMb3NzIFJhdGU6IDE2IGJpdHMNCg0KICAgICAgVGhlIGZyYWN0aW9uIG9mIHBh Y2tldHMgbG9zdCBkdXJpbmcgYnVyc3RzIHNpbmNlIHRoZSBiZWdpbm5pbmcgb2YNCiAgICAgIHJl Y2VwdGlvbiwgZXhwcmVzc2VkIGFzIGEgZml4ZWQgcG9pbnQgbnVtYmVyIHdpdGggdGhlIGJpbmFy eSBwb2ludA0KICAgICAgYXQgdGhlIGxlZnQgZWRnZSBvZiB0aGUgZmllbGQuICBUaGlzIHZhbHVl IGlzIGNhbGN1bGF0ZWQgYnkNCiAgICAgIGRpdmlkaW5nIFBhY2tldHMgTG9zcyBpbiBCdXJzdHMg YnkgVG90YWwgUGFja2V0cyBleHBlY3RlZCBpbg0KICAgICAgQnVyc3RzIGFzIGZvbGxvd3M6DQog ICAgICAgICAgUGFja2V0cyBMb3NzIGluIEJ1cnN0cyAvIFRvdGFsIFBhY2tldHMgZXhwZWN0ZWQg aW4gQnVyc3RzDQoNCkhvdyBpcyBhIGJ1cnN0IGxvc3MgcmF0aW8gPSAxLjAgcmVwb3J0ZWQ/ICBB bGwgdGhlIGRpZ2l0cyBhcmUgdG8gdGhlIA0KcmlnaHQgb2YgdGhlIGRlY2ltYWwgcGxhY2UuDQoN CltRaW5dOiBHb29kIGNvbW1lbnQsIEkgdGhpbmsgd2UgY2FuIHVzZSAxNWJpdHMgb2YgMTYtYml0 IEJ1cnN0IGdhcCBSYXRlDQp0byByZXByZXNlbnQgdGhlIHZhbHVlIG9mIGJ1cnN0IGxvc3MgcmF0 ZSA9MS4wLiBUaGUgbGVmdG1vc3QgYml0IGlzIHVzZWQgdG8gaW5kaWNhdGUgbWVhc3VyZW1lbnQN CmlzIHVuYXZhaWxhYmxlLiBUaGUgcHJvcG9zZWQgY2hhbmdlIGlzIGFzIGZvbGxvd3M6DQpPTEQg VEVYVDoNCiINCiAgIEJ1cnN0IExvc3MgUmF0ZTogMTYgYml0cw0KDQogICAgICBUaGUgZnJhY3Rp b24gb2YgcGFja2V0cyBsb3N0IGR1cmluZyBidXJzdHMgc2luY2UgdGhlIGJlZ2lubmluZyBvZg0K ICAgICAgcmVjZXB0aW9uLCBleHByZXNzZWQgYXMgYSBmaXhlZCBwb2ludCBudW1iZXIgd2l0aCB0 aGUgYmluYXJ5IHBvaW50DQogICAgICBhdCB0aGUgbGVmdCBlZGdlIG9mIHRoZSBmaWVsZC4gIFRo aXMgdmFsdWUgaXMgY2FsY3VsYXRlZCBieQ0KICAgICAgZGl2aWRpbmcgUGFja2V0cyBMb3NzIGlu IEJ1cnN0cyBieSBUb3RhbCBQYWNrZXRzIGV4cGVjdGVkIGluDQogICAgICBCdXJzdHMgYXMgZm9s bG93czoNCg0KICAgICAgUGFja2V0cyBMb3NzIGluIEJ1cnN0cyAvIFRvdGFsIFBhY2tldHMgZXhw ZWN0ZWQgaW4gQnVyc3RzDQoiDQpORVcgVEVYVDoNCiINCiAgIEJ1cnN0IExvc3MgUmF0ZTogMTYg Yml0cw0KDQogICAgICBUaGUgZnJhY3Rpb24gb2YgcGFja2V0cyBsb3N0IGR1cmluZyBidXJzdHMg c2luY2UgdGhlIGJlZ2lubmluZyBvZg0KICAgICAgcmVjZXB0aW9uLCBleHByZXNzZWQgYXMgYSBm aXhlZCBwb2ludCBudW1iZXIgd2l0aCB0aGUgYmluYXJ5IHBvaW50DQogICAgICBhdCB0aGUgbGVm dCBlZGdlIG9mIHRoZSBmaWVsZC4gIFRoaXMgdmFsdWUgaXMgY2FsY3VsYXRlZCBieQ0KICAgICAg ZGl2aWRpbmcgUGFja2V0cyBMb3NzIGluIEJ1cnN0cyBieSBUb3RhbCBQYWNrZXRzIGV4cGVjdGVk IGluDQogICAgICBCdXJzdHMsIG11bHRpcGx5aW5nIHRoZSByZXN1bHQgb2YgdGhlIGRpdmlzaW9u IGJ5IDMyNzY4ICwgYW5kIHRha2luZyB0aGUgaW50ZWdlciBwYXJ0IGFzIGZvbGxvd3M6DQoNCiAg ICAgIFBhY2tldHMgTG9zcyBpbiBCdXJzdHMgLyBUb3RhbCBQYWNrZXRzIGV4cGVjdGVkIGluIEJ1 cnN0cw0KICAgICAgSWYgdGhlIG1lYXN1cmVtZW50IGlzIHVuYXZhaWxhYmxlLCB0aGUgdmFsdWUg MHg4MDAwIE1VU1QgYmUNCiAgICAgIHJlcG9ydGVkLg0KIg0KDQogICBCdXJzdCBEdXJhdGlvbiBN ZWFuOjE2Yml0cw0KDQogICAgICBUaGUgbWVhbiBidXJzdCBkdXJhdGlvbiBpcyBvYnRhaW5lZCBh cyB0aGUgcXVvdGllbnQ6DQoNCiAgICAgIG1lYW4gPSBTdW0gb2YgQnVyc3QgRHVyYXRpb25zIC8g TnVtYmVyIG9mIEJ1cnN0cw0KDQpIb3cgaXMgRGl2aWRlIGJ5IFplcm8gaGFuZGxlZD8NCg0KW1Fp bl06IEdvb2QgY29tbWVudCwgd2Ugd2lsbCB1c2UgMHhGRkZGIHRvIGluZGljYXRlIG1lYXN1cmVt ZW50IHVuYXZhaWxhYmxlLg0KDQogICBCdXJzdCBEdXJhdGlvbiBWYXJpYW5jZToxNmJpdHMNCg0K ICAgICAgVGhlIHZhcmlhbmNlIG9mIHRoZSBidXJzdCBkdXJhdGlvbiBpcyBvYnRhaW5lZCB1c2lu ZyB0aGUgc3RhbmRhcmQNCiAgICAgIHJlc3VsdDoNCg0KICAgICAgdmFyID0gKCBTdW0gb2YgU3F1 YXJlcyBvZiBCdXJzdCBEdXJhdGlvbnMgLSBOdW1iZXIgb2YgQnVyc3RzICoNCiAgICAgIG1lYW5e MiApIC8gKE51bWJlciBvZiBCdXJzdHMgLSAxKQ0KDQpIb3cgaXMgRGl2aWRlIGJ5IFplcm8gaGFu ZGxlZD8gKGUuZy4sIE51bWJlciBvZiBCdXJzdHMgPSAxKQ0KDQpbUWluXTogR29vZCBjb21tZW50 LCB3ZSB3aWxsIHVzZSAweEZGRkYgdG8gaW5kaWNhdGUgbWVhc3VyZW1lbnQgdW5hdmFpbGFibGUu DQoNClNlY3Rpb24gNC4xLjIgc2F5cw0KDQogICBOdW1iZXIgb2YgZnVsbCBmcmFtZXMgbG9zdCAo bG9zdF9mdWxsX2ZyYW1lcyk6IDMyIGJpdHMNCg0KICAgICAgSWYgb25lIGZyYW1lIGlzIGNvbXBs ZXRlbHkgbG9zdCwgdGhpcyBmcmFtZSBpcyByZWdhcmRlZCBhcyBvbmUNCiAgICAgIGxvc3QgZnVs bCBmcmFtZS4gIFRoZSBsb3N0X2Z1bGxfZnJhbWVzIGlzIGVxdWl2YWxlbnQgdG8gdGhlIG51bWJl cg0KICAgICAgb2YgZnVsbCBmcmFtZXMgbG9zdCBpbiB0aGUgYWJvdmUgc2VxdWVuY2UgbnVtYmVy IGludGVydmFsLg0KDQoNCiAgIE51bWJlciBvZiBwYXJ0aWFsIGZyYW1lcyBsb3N0IChsb3N0X3Bh cnRpYWxfZnJhbWVzKTogMzIgYml0cw0KDQogICAgICBJZiBvbmUgZnJhbWUgaXMgcGFydGlhbGx5 IGxvc3QsIHRoaXMgZnJhbWUgaXMgcmVnYXJkZWQgYXMgb25lIGxvc3QNCiAgICAgIGZyYWN0aW9u YWwgZnJhbWUuICBUaGUgdmFsdWUgb2YgdGhlIGxvc3RfcGFydGlhbF9mcmFtZXMgZmllbGQgaXMN CiAgICAgIGVxdWl2YWxlbnQgdG8gdGhlIG51bWJlciBvZiBwYXJ0aWFsIGZyYW1lcyBsb3N0IGlu IHRoZSBhYm92ZQ0KICAgICAgc2VxdWVuY2UgbnVtYmVyIGludGVydmFsLg0KDQpJcyB0aGlzIGEg bWV0cmljIGRlZmluaXRpb24sIHBhcnRpYWxseSBsb3N0IHZpZGVvIGZyYW1lcz8gDQpJcyB0aGlz IGEgbWV0cmljIGRlZmluaXRpb24sIGZ1bGx5IGxvc3QgdmlkZW8gZnJhbWVzPyANCltRaW5dOiBZ ZXMuDQoNCkhvdyBpcyB0aGlzIGV2ZW50IGRldGVjdGVkIHNvIGl0IGNhbiBiZSBjb3VudGVkPw0K DQpbUWluXTogDQpBIGZyYW1lIGlzIGVpdGhlciBzcGxpdCBhY3Jvc3MgbXVsdGlwbGUgcGFja2V0 cyBvciBjYXJyaWVkIGluIG9ubHkgb25lIHBhY2tldCwNClN1cHBvc2UgYSBmcmFtZSBpcyBzcGxp dCBhY3Jvc3MgbXVsdGlwbGUgcGFja2V0cywNCmlmIHdlIGtub3cgaG93IG1hbnkgcGFja2V0cyBj b21wcmlzZSBhIGZyYW1lKGUuZy4sIGJ5IGxvb2tpbmcgYXQgc2VxdWVuY2UgDQpudW1iZXIgaW4g dGhlIFJUUCBoZWFkZXIgYW5kIGNvbnRpbnVlIGNvdW50ZXIgaW4gdGhlIFRTIGhlYWRlciksIHdl IHdpbGwgc2VlIHdoZXRoZXINCmEgZnJhbWUgaXMgZnVsbCBsb3N0IGZyYW1lIG9yIHBhcnRpYWxs eSBsb3N0IGZyYW1lLg0KSWYgc29tZSBwYWNrZXRzIG9mIGEgZnJhbWUgaXMgbG9zdCwgd2Ugd2ls bCByZWdhcmQgaXQgYXMgcGFydGlhbCBsb3N0IGZyYW1lLA0KSWYgYWxsIHBhY2tldHMgb2YgZnJh bWUgaXMgbG9zdCwgdGhpcyBmcmFtZSB3aWxsIGJlIHJlZ2FyZGVkIGFzIGZ1bGwgbG9zdCBmcmFt ZS4NCg0KSXMgdGhlcmUgYSByZWZlcmVuY2UgZm9yIHRoaXMgbWV0cmljIGF2YWlsYWJsZT8NCg0K W1Fpbl06IE5vdGUgYXdhcmUgb2YgaXQuIA0KLT0tPS09LT0tPS09LQ0KDQo+IC0tLS0tT3JpZ2lu YWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IHBtLWRpci1ib3VuY2VzQGlldGYub3JnIFttYWlsdG86 cG0tZGlyLWJvdW5jZXNAaWV0Zi5vcmddIE9uIEJlaGFsZg0KPiBPZiBCZW5vaXQgQ2xhaXNlDQo+ IFNlbnQ6IFN1bmRheSwgSmFudWFyeSAyNywgMjAxMyAyOjA5IFBNDQo+IFRvOiBwbS1kaXJAaWV0 Zi5vcmcNCj4gQ2M6IHhyYmxvY2stY2hhaXJzQHRvb2xzLmlldGYub3JnOyBkcmFmdC1pZXRmLXhy YmxvY2stcnRjcC14ci1zdW1tYXJ5LQ0KPiBzdGF0QHRvb2xzLmlldGYub3JnOyBHb256YWxvIENh bWFyaWxsbw0KPiBTdWJqZWN0OiBSZTogW3BtLWRpcl0gUmVxdWVzdCBmb3IgYW4gUkZDIDYzNjkg cmV2aWV3IG9mIGRyYWZ0LWlldGYtDQo+IHhyYmxvY2stcnRjcC14ci1zdW1tYXJ5LXN0YXQtMDYN Cj4gDQo+IERlYXIgYWxsLA0KPiANCj4gSGVyZSBpcyBhbiBvZmZpY2lhbCByZXF1ZXN0IGZvciB0 aGUgUkZDIDYzOTAgcmV2aWV3Lg0KPiBXaG8gd291bGQgbGlrZSB0byB2b2x1bnRlZXI/DQo+IA0K PiBSZWdhcmRzLCBCZW5vaXQNCj4gPiBIaSBCZW5vaXQsDQo+ID4NCj4gPiBhcyBzdWdnZXN0ZWQg YnkgRGFuIGluIGhpcyBQUk9UTyB3cml0ZXVwLCBJIHdvdWxkIGxpa2UgdG8gcmVxdWVzdCB0aGUN Cj4gPiBwZXJmb3JtYW5jZSBtZXRyaWNzIGRpcmVjdG9yYXRlIHRvIHBlcmZvcm0gYW4gUkZDIDYz OTAgcmV2aWV3IG9mIHRoZQ0KPiA+IGZvbGxvd2luZyBkcmFmdCBhcyBwYXJ0IG9mIGl0cyBJRVRG IExDOg0KPiA+DQo+ID4gaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQtaWV0 Zi14cmJsb2NrLXJ0Y3AteHItc3VtbWFyeS0NCj4gc3RhdC8NCj4gPg0KPiA+IENvdWxkIHlvdSBw bGVhc2UgYXJyYW5nZSBzdWNoIGEgcmV2aWV3Pw0KPiA+DQo+ID4gVGhhbmtzLA0KPiA+DQo+ID4g R29uemFsbw0KPiA+DQo+ID4NCj4gDQo+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fDQo+IHBtLWRpciBtYWlsaW5nIGxpc3QNCj4gcG0tZGlyQGlldGYub3Jn DQo+IGh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vcG0tZGly From acmorton@att.com Mon Feb 4 05:23:05 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4766921F8512 for ; Mon, 4 Feb 2013 05:23:04 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.371 X-Spam-Level: X-Spam-Status: No, score=-106.371 tagged_above=-999 required=5 tests=[AWL=0.228, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pSF9rqzciimF for ; Mon, 4 Feb 2013 05:23:03 -0800 (PST) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 88FAB21F84B9 for ; Mon, 4 Feb 2013 05:22:57 -0800 (PST) Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id C9DB21202E2; Mon, 4 Feb 2013 08:24:35 -0500 (EST) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-green.research.att.com (Postfix) with ESMTP id AC3B1E36E7; Mon, 4 Feb 2013 08:17:09 -0500 (EST) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Mon, 4 Feb 2013 08:22:56 -0500 From: "MORTON JR., ALFRED (AL)" To: Paul Aitken , Benoit Claise Date: Mon, 4 Feb 2013 08:22:54 -0500 Thread-Topic: [pm-dir] Performance metrics doctors generated email Thread-Index: Ac4C1jS3YvQQxX8oSASAdb1dq5u+6gAAqcNA Message-ID: References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FAA8F.4030604@cisco.com> <510FAB65.4080505@cisco.com> <510FAE73.1080901@cisco.com> In-Reply-To: <510FAE73.1080901@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "pm-dir@ietf.org" Subject: Re: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Feb 2013 13:23:05 -0000 Hi Paul, Our generally accepted mode of operation is in the message appended below. We need to make a few changes now that the script is finding more drafts than there are Directorate members. We could choose to expand our membership, that's a question I=20 asked the Directorate a few weeks ago, but with no feedback there was no action. When the new list was announced, several people tried to join. We should probably have a call for new members soon,=20 if the current Directorate members agree. Also, if anyone would=20 like to retire from the Performance Directorate (for example, if you didn't find time to perform reviews in 2012, then you might consider the question in that light), please let me know. I agree that we need a more accessible and now, more dynamic means of=20 organization. A Wiki will work, as long as everyone has access to it and can update draft status as they complete reviews, etc. Once we handle the initial rush of drafts from the=20 script, I would suggest that the script keep last week's list and highlight the new arrivals to facilitate assignment, rather than=20 leave this step for manual checking. regards, Al PM Dir admin -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-= =3D- At 09:47 AM 1/16/2012, Al Morton wrote: As this new year goes into full swing, I'd like to offer some thoughts on how we might operate as a Directorate and hear from others. As our web page says: "RFC 6390 is used as a reference for the technical=20 and process issues." Have your copy handy when reviewing a draft. A lot of interactions may be streamlined by pointing the authors to 6390, and mentioning that it is a BCP. I feel that our reviews should be *early* in the draft development process, and that no performance-related draft should start IETF Last Call= =20 without previous review against 6390, unless it truly slipped by us and other folks who know about the Directorate. =20 There are two ways we can collect drafts for review: - WG Chairs and ADs can/should suggest drafts to us - Directorate Members will see drafts, hear about drafts with performance work, etc. We are the eyes and ears of the Directorate. We all have friend= s working in other areas of IETF, let them know what we're looking for. I suggest to split the early review into two categories: 1. Quick Scan - If you find a draft, do a quick scan or suggest that others take a look with a message to pmol-list. Finding a draft does not mean that you will be designated to do a review (see below). 2. WG Candidate/Chartered Item - When a draft appears to have enough value and interest to become a WG item, or the WG Chairs involved recommend a=20 Performance Metrics Directorate review on their own, then we do a more complete review against 6390. =20 If these categories are useful, then I'd like to suggest that *two*=20 reviewers are assigned to WG Candidate/Chartered drafts and they follow the draft after early review. This may mean looking at the draft again in WG or IETF Last Call, assuming it goes that far. I suggest 2 reviewers while we're all new at this, and to help ensure that one is still around years later when Last Calls are done. All drafts identified, quick scans, and reviews should be sent to the pmol-list and the authors (and possibly the relevant WG chairs) let us all know what you think, Al > -----Original Message----- > From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf > Of Paul Aitken > Sent: Monday, February 04, 2013 7:50 AM > To: Benoit Claise > Cc: pm-dir@ietf.org > Subject: Re: [pm-dir] Performance metrics doctors generated email >=20 > Benoit, >=20 > > Yes, all these questions are important. > > Yes a WIKI is the right solution IMHO. >=20 > Great. Where can it be hosted? >=20 > eg, is http://trac.tools.ietf.org/wg/pmol/trac/wiki a good starting > place? Or is there a pm-dir page? >=20 > P. >=20 > >> Benoit, All, > >> > >> How is pm-dir tracking these drafts? ie, how do we know whether > >> someone is currently reviewing, or has already reviewed, each of > >> these drafts? > >> > >> How are reviewers selected? By relevance, skill, luck, or misfortune? > >> > >> Also, it'd be useful to know whether the previous version was already > >> reviewed, and what the feedback was: > >> > >> * issues were raised, recommendations were made, changes were > >> expected -> some re-review should be done > >> > >> * no issues were raised -> a quick review of the delta may be > >> sufficient, to check for any new issues > >> > >> * previous version wasn't reviewed -> a thorough review may be > >> required. > >> > >> > >> I'm thinking it'd be useful to have a pm-dir wiki listing each of the > >> drafts below. eg, one page per draft, with one section per version, > >> containing reviewer comments pertinent to that version, even if the > >> comments simply say, "reviewed, no issues". > >> > >> Hopefully I missed something, and most (if not all?) of this is > >> already in place? :-) > >> > >> Thanks, > >> P. > >> > >> > >> On 04/02/13 11:53, Benoit Claise wrote: > >>> Dear all, > >>> > >>> This is an automatically generated email. > >>> It lists the IETF internet-drafts that reference the PMOL RFC 6390, > >>> as a normative or informative reference. > >>> It also lists all the IETF internet-drafts that contain "performance > >>> metric". > >>> > >>> Regards, Benoit > >>> > >>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >>> > >>> Normative References > >>> -------------------- > >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active > >>> Informative References > >>> ---------------------- > >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active > >>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active > >>> draft-ietf-xrblock-rtcp-xr-discard-11 Active > >>> draft-ietf-xrblock-rtcp-xr-jb-07 Active > >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active > >>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active > >>> > >>> drafts containing performance metric > >>> ------------------------------------ > >>> draft-ietf-alto-deployments-05 Active > >>> draft-ietf-alto-protocol-13 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-ippm-rate-problem-02 Active > >>> draft-ietf-ippm-testplan-rfc2680-01 Active > >>> draft-ietf-karp-threats-reqs-07 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-manet-olsrv2-mib-05 Active > >>> draft-ietf-manet-smf-mib-06 Active > >>> draft-ietf-mmusic-media-loopback-27 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-nvo3-framework-01 Active > >>> draft-ietf-opsawg-oam-overview-08 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-rtcweb-rtp-usage-05 Active > >>> draft-ietf-rtgwg-cl-framework-02 Active > >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active > >>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active > >>> draft-ietf-xrblock-rtcp-xr-discard-11 Active > >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active > >>> draft-ietf-xrblock-rtcp-xr-jb-07 Active > >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active > >>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active > >>> draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing > >>> - ID Tracker state > >>> draft-ietf-xrblock-rtcp-xr-synchronization-02 Active > >>> _______________________________________________ > >>> pm-dir mailing list > >>> pm-dir@ietf.org > >>> https://www.ietf.org/mailman/listinfo/pm-dir > >> > >> _______________________________________________ > >> pm-dir mailing list > >> pm-dir@ietf.org > >> https://www.ietf.org/mailman/listinfo/pm-dir > >> > >> >=20 > _______________________________________________ > pm-dir mailing list > pm-dir@ietf.org > https://www.ietf.org/mailman/listinfo/pm-dir From bclaise@cisco.com Sun Feb 10 14:06:35 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9E9C421F8858 for ; Sun, 10 Feb 2013 14:06:35 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.572 X-Spam-Level: X-Spam-Status: No, score=-10.572 tagged_above=-999 required=5 tests=[AWL=0.027, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ct832HHzlDtC for ; Sun, 10 Feb 2013 14:06:35 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id B742D21F880B for ; Sun, 10 Feb 2013 14:06:34 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1AM6XA0011657 for ; Sun, 10 Feb 2013 23:06:33 +0100 (CET) Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1AM64YN018080 for ; Sun, 10 Feb 2013 23:06:14 +0100 (CET) Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r1AM60Ca027694 for pm-dir@ietf.org; Sun, 10 Feb 2013 23:06:00 +0100 (CET) Date: Sun, 10 Feb 2013 23:06:00 +0100 From: Benoit Claise To: pm-dir@ietf.org Message-ID: <20130210220600.GA27692@sweet-brew-5.cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2013 22:06:35 -0000 Dear all, This is an automatically generated email. It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference. It also lists all the IETF internet-drafts that contain "performance metric". Regards, Benoit =========================================================== Normative References -------------------- draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active Informative References ---------------------- draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-concsec-03 Active draft-ietf-xrblock-rtcp-xr-decodability-07 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-discard-11 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active draft-ietf-xrblock-rtcp-xr-jb-07 Active draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active draft-ietf-xrblock-rtcp-xr-qoe-03 Active drafts containing performance metric ------------------------------------ draft-ietf-alto-deployments-05 Active draft-ietf-alto-protocol-13 In IESG processing - ID Tracker state draft-ietf-bmwg-ca-bench-meth-04 Active draft-ietf-ippm-rate-problem-02 Active draft-ietf-ippm-testplan-rfc2680-01 Active draft-ietf-karp-threats-reqs-07 In IESG processing - ID Tracker state draft-ietf-manet-olsrv2-mib-05 Active draft-ietf-manet-smf-mib-06 Active draft-ietf-mmusic-media-loopback-27 In IESG processing - ID Tracker state draft-ietf-nvo3-framework-02 Active draft-ietf-opsawg-oam-overview-08 In IESG processing - ID Tracker state draft-ietf-rtcweb-rtp-usage-05 Active draft-ietf-rtgwg-cl-framework-02 Active draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-concsec-03 Active draft-ietf-xrblock-rtcp-xr-decodability-07 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-discard-11 Active draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active draft-ietf-xrblock-rtcp-xr-jb-07 Active draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active draft-ietf-xrblock-rtcp-xr-qoe-03 Active draft-ietf-xrblock-rtcp-xr-summary-stat-08 In IESG processing - ID Tracker state draft-ietf-xrblock-rtcp-xr-synchronization-02 Active From bclaise@cisco.com Wed Feb 13 05:33:29 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ECE7821F86CB for ; Wed, 13 Feb 2013 05:33:29 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.547 X-Spam-Level: X-Spam-Status: No, score=-10.547 tagged_above=-999 required=5 tests=[AWL=0.052, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h1TD4hQKd6Gy for ; Wed, 13 Feb 2013 05:33:29 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 5856E21F86C9 for ; Wed, 13 Feb 2013 05:33:28 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1DD5MID015128; Wed, 13 Feb 2013 14:05:23 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1DD4f9O013860; Wed, 13 Feb 2013 14:04:51 +0100 (CET) Message-ID: <511B8EBB.4060200@cisco.com> Date: Wed, 13 Feb 2013 14:01:47 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: "MORTON JR., ALFRED (AL)" References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FAA8F.4030604@cisco.com> <510FAB65.4080505@cisco.com> <510FAE73.1080901@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "pm-dir@ietf.org" , Paul Aitken , Alan DeKok Subject: Re: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2013 13:33:30 -0000 Al, All, > Hi Paul, > > Our generally accepted mode of operation is in the message > appended below. We need to make a few changes now that the > script is finding more drafts than there are Directorate members. > > We could choose to expand our membership, that's a question I > asked the Directorate a few weeks ago, but with no feedback there > was no action. > When the new list was announced, several people tried to join. > We should probably have a call for new members soon, > if the current Directorate members agree. Also, if anyone would > like to retire from the Performance Directorate (for example, > if you didn't find time to perform reviews in 2012, then > you might consider the question in that light), please let me know. > > I agree that we need a more accessible and now, more dynamic means of > organization. A Wiki will work, as long as everyone has access to it > and can update draft status as they complete reviews, etc. Agreed. > > Once we handle the initial rush of drafts from the > script, I would suggest that the script keep last week's list and > highlight the new arrivals to facilitate assignment, rather than > leave this step for manual checking. That would nice, but we would need a kind of database to keep track of the already assigned drafts. It starts to be a more complex script... Note: Alan (in the copy list) is improving the script so that the drafts that maps a keyword (performance metric in this case) and that have a reference to RFC 6390 doesn't appear twice. Regards, Benoit > > regards, > Al > PM Dir admin > > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > At 09:47 AM 1/16/2012, Al Morton wrote: > As this new year goes into full swing, I'd like to offer some > thoughts on how we might operate as a Directorate and > hear from others. > > As our web page says: "RFC 6390 is used as a reference for the technical > and process issues." Have your copy handy when reviewing a draft. > A lot of interactions may be streamlined by pointing the authors to 6390, > and mentioning that it is a BCP. > > I feel that our reviews should be *early* in the draft development > process, and that no performance-related draft should start IETF Last Call > without previous review against 6390, unless it truly slipped by us > and other folks who know about the Directorate. > > There are two ways we can collect drafts for review: > > - WG Chairs and ADs can/should suggest drafts to us > > - Directorate Members will see drafts, hear about drafts with performance > work, etc. We are the eyes and ears of the Directorate. We all have friends > working in other areas of IETF, let them know what we're looking for. > > I suggest to split the early review into two categories: > > 1. Quick Scan - If you find a draft, do a quick scan or suggest that > others take a look with a message to pmol-list. Finding a draft does > not mean that you will be designated to do a review (see below). > > 2. WG Candidate/Chartered Item - When a draft appears to have enough value > and interest to become a WG item, or the WG Chairs involved recommend a > Performance Metrics Directorate review on their own, then we do a more > complete review against 6390. > > If these categories are useful, then I'd like to suggest that *two* > reviewers are assigned to WG Candidate/Chartered drafts and they follow > the draft after early review. This may mean looking at the draft again > in WG or IETF Last Call, assuming it goes that far. I suggest 2 reviewers > while we're all new at this, and to help ensure that one is still around > years later when Last Calls are done. > > All drafts identified, quick scans, and reviews should be sent to the > pmol-list and the authors (and possibly the relevant WG chairs) > > let us all know what you think, > Al > >> -----Original Message----- >> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf >> Of Paul Aitken >> Sent: Monday, February 04, 2013 7:50 AM >> To: Benoit Claise >> Cc: pm-dir@ietf.org >> Subject: Re: [pm-dir] Performance metrics doctors generated email >> >> Benoit, >> >>> Yes, all these questions are important. >>> Yes a WIKI is the right solution IMHO. >> Great. Where can it be hosted? >> >> eg, is http://trac.tools.ietf.org/wg/pmol/trac/wiki a good starting >> place? Or is there a pm-dir page? >> >> P. >> >>>> Benoit, All, >>>> >>>> How is pm-dir tracking these drafts? ie, how do we know whether >>>> someone is currently reviewing, or has already reviewed, each of >>>> these drafts? >>>> >>>> How are reviewers selected? By relevance, skill, luck, or misfortune? >>>> >>>> Also, it'd be useful to know whether the previous version was already >>>> reviewed, and what the feedback was: >>>> >>>> * issues were raised, recommendations were made, changes were >>>> expected -> some re-review should be done >>>> >>>> * no issues were raised -> a quick review of the delta may be >>>> sufficient, to check for any new issues >>>> >>>> * previous version wasn't reviewed -> a thorough review may be >>>> required. >>>> >>>> >>>> I'm thinking it'd be useful to have a pm-dir wiki listing each of the >>>> drafts below. eg, one page per draft, with one section per version, >>>> containing reviewer comments pertinent to that version, even if the >>>> comments simply say, "reviewed, no issues". >>>> >>>> Hopefully I missed something, and most (if not all?) of this is >>>> already in place? :-) >>>> >>>> Thanks, >>>> P. >>>> >>>> >>>> On 04/02/13 11:53, Benoit Claise wrote: >>>>> Dear all, >>>>> >>>>> This is an automatically generated email. >>>>> It lists the IETF internet-drafts that reference the PMOL RFC 6390, >>>>> as a normative or informative reference. >>>>> It also lists all the IETF internet-drafts that contain "performance >>>>> metric". >>>>> >>>>> Regards, Benoit >>>>> >>>>> =========================================================== >>>>> >>>>> Normative References >>>>> -------------------- >>>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>>>> Informative References >>>>> ---------------------- >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>>>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>>>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>>>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>>>> >>>>> drafts containing performance metric >>>>> ------------------------------------ >>>>> draft-ietf-alto-deployments-05 Active >>>>> draft-ietf-alto-protocol-13 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-ippm-rate-problem-02 Active >>>>> draft-ietf-ippm-testplan-rfc2680-01 Active >>>>> draft-ietf-karp-threats-reqs-07 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-manet-olsrv2-mib-05 Active >>>>> draft-ietf-manet-smf-mib-06 Active >>>>> draft-ietf-mmusic-media-loopback-27 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-nvo3-framework-01 Active >>>>> draft-ietf-opsawg-oam-overview-08 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-rtcweb-rtp-usage-05 Active >>>>> draft-ietf-rtgwg-cl-framework-02 Active >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>>>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>>>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>>>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>>>> draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-synchronization-02 Active >>>>> _______________________________________________ >>>>> pm-dir mailing list >>>>> pm-dir@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/pm-dir >>>> _______________________________________________ >>>> pm-dir mailing list >>>> pm-dir@ietf.org >>>> https://www.ietf.org/mailman/listinfo/pm-dir >>>> >>>> >> _______________________________________________ >> pm-dir mailing list >> pm-dir@ietf.org >> https://www.ietf.org/mailman/listinfo/pm-dir > From acmorton@att.com Wed Feb 13 08:02:36 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 38D2521F8770 for ; Wed, 13 Feb 2013 08:02:36 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -106.447 X-Spam-Level: X-Spam-Status: No, score=-106.447 tagged_above=-999 required=5 tests=[AWL=0.151, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3ibNnUdJpgDz for ; Wed, 13 Feb 2013 08:02:34 -0800 (PST) Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 2A0C521F8758 for ; Wed, 13 Feb 2013 08:02:31 -0800 (PST) Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id ECC27120924; Wed, 13 Feb 2013 11:04:30 -0500 (EST) Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-blue.research.att.com (Postfix) with ESMTP id B6338F00F2; Wed, 13 Feb 2013 11:02:30 -0500 (EST) Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Wed, 13 Feb 2013 11:02:30 -0500 From: "MORTON JR., ALFRED (AL)" To: Benoit Claise Date: Wed, 13 Feb 2013 11:02:29 -0500 Thread-Topic: Re: [pm-dir] Performance metrics doctors generated email Thread-Index: Ac4J6t/H27z4nylnS0aJVkfWWq7q5gAF8WdQ Message-ID: References: <511B8EBB.4060200@cisco.com> <511B8EEE.8030909@cisco.com> In-Reply-To: <511B8EEE.8030909@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_F1312FAF1A1E624DA0972D1C9A91379A1BF83AFB60njfpsrvexg7re_" MIME-Version: 1.0 Cc: "pm-dir@ietf.org" , Paul Aitken , Alan DeKok Subject: Re: [pm-dir] Performance metrics doctors generated email X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2013 16:02:36 -0000 --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF83AFB60njfpsrvexg7re_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable > Once we handle the initial rush of drafts from the > script, I would suggest that the script keep last week's list and > highlight the new arrivals to facilitate assignment, rather than > leave this step for manual checking. That would nice, but we would need a kind of database to keep track of the already assigned drafts. It starts to be a more complex script... It doesn't have to keep track of assignments, just last week's list. IOW, highlight the drafts that are new this week, an automated "uniq" list at the top of the mail... Note: Alan (in the copy list) is improving the script so that the drafts that maps a keyword (performance metric in this case) and that have a reference to RFC 6390 doesn't appear twice. great, that's a good start. I'm not requesting any volunteers right now, with the draft deadlines looming... Al From: Benoit Claise [mailto:bclaise@cisco.com] Sent: Wednesday, February 13, 2013 8:03 AM To: MORTON JR., ALFRED (AL) Subject: Fwd: Re: [pm-dir] Performance metrics doctors generated email Hi Al, Can you please take care of setting up a WIKI. Thanks in advance. Regards, Benoit -------- Original Message -------- Subject: Re: [pm-dir] Performance metrics doctors generated email Date: Wed, 13 Feb 2013 14:01:47 +0100 From: Benoit Claise To: MORTON JR., ALFRED (AL) CC: Paul Aitken , "pm-dir@ietf.org= " , Alan D= eKok Al, All, > Hi Paul, > > Our generally accepted mode of operation is in the message > appended below. We need to make a few changes now that the > script is finding more drafts than there are Directorate members. > > We could choose to expand our membership, that's a question I > asked the Directorate a few weeks ago, but with no feedback there > was no action. > When the new list was announced, several people tried to join. > We should probably have a call for new members soon, > if the current Directorate members agree. Also, if anyone would > like to retire from the Performance Directorate (for example, > if you didn't find time to perform reviews in 2012, then > you might consider the question in that light), please let me know. > > I agree that we need a more accessible and now, more dynamic means of > organization. A Wiki will work, as long as everyone has access to it > and can update draft status as they complete reviews, etc. Agreed. > > Once we handle the initial rush of drafts from the > script, I would suggest that the script keep last week's list and > highlight the new arrivals to facilitate assignment, rather than > leave this step for manual checking. That would nice, but we would need a kind of database to keep track of the already assigned drafts. It starts to be a more complex script... Note: Alan (in the copy list) is improving the script so that the drafts that maps a keyword (performance metric in this case) and that have a reference to RFC 6390 doesn't appear twice. Regards, Benoit > > regards, > Al > PM Dir admin > > -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-= =3D- > At 09:47 AM 1/16/2012, Al Morton wrote: > As this new year goes into full swing, I'd like to offer some > thoughts on how we might operate as a Directorate and > hear from others. > > As our web page says: "RFC 6390 is used as a reference for the technical > and process issues." Have your copy handy when reviewing a draft. > A lot of interactions may be streamlined by pointing the authors to 6390, > and mentioning that it is a BCP. > > I feel that our reviews should be *early* in the draft development > process, and that no performance-related draft should start IETF Last Cal= l > without previous review against 6390, unless it truly slipped by us > and other folks who know about the Directorate. > > There are two ways we can collect drafts for review: > > - WG Chairs and ADs can/should suggest drafts to us > > - Directorate Members will see drafts, hear about drafts with performance > work, etc. We are the eyes and ears of the Directorate. We all have frie= nds > working in other areas of IETF, let them know what we're looking for. > > I suggest to split the early review into two categories: > > 1. Quick Scan - If you find a draft, do a quick scan or suggest that > others take a look with a message to pmol-list. Finding a draft does > not mean that you will be designated to do a review (see below). > > 2. WG Candidate/Chartered Item - When a draft appears to have enough valu= e > and interest to become a WG item, or the WG Chairs involved recommend a > Performance Metrics Directorate review on their own, then we do a more > complete review against 6390. > > If these categories are useful, then I'd like to suggest that *two* > reviewers are assigned to WG Candidate/Chartered drafts and they follow > the draft after early review. This may mean looking at the draft again > in WG or IETF Last Call, assuming it goes that far. I suggest 2 reviewers > while we're all new at this, and to help ensure that one is still around > years later when Last Calls are done. > > All drafts identified, quick scans, and reviews should be sent to the > pmol-list and the authors (and possibly the relevant WG chairs) > > let us all know what you think, > Al > >> -----Original Message----- >> From: pm-dir-bounces@ietf.org [mailto:pm= -dir-bounces@ietf.org] On Behalf >> Of Paul Aitken >> Sent: Monday, February 04, 2013 7:50 AM >> To: Benoit Claise >> Cc: pm-dir@ietf.org >> Subject: Re: [pm-dir] Performance metrics doctors generated email >> >> Benoit, >> >>> Yes, all these questions are important. >>> Yes a WIKI is the right solution IMHO. >> Great. Where can it be hosted? >> >> eg, is http://trac.tools.ietf.org/wg/pmol/trac/wiki a good starting >> place? Or is there a pm-dir page? >> >> P. >> >>>> Benoit, All, >>>> >>>> How is pm-dir tracking these drafts? ie, how do we know whether >>>> someone is currently reviewing, or has already reviewed, each of >>>> these drafts? >>>> >>>> How are reviewers selected? By relevance, skill, luck, or misfortune? >>>> >>>> Also, it'd be useful to know whether the previous version was already >>>> reviewed, and what the feedback was: >>>> >>>> * issues were raised, recommendations were made, changes were >>>> expected -> some re-review should be done >>>> >>>> * no issues were raised -> a quick review of the delta may be >>>> sufficient, to check for any new issues >>>> >>>> * previous version wasn't reviewed -> a thorough review may be >>>> required. >>>> >>>> >>>> I'm thinking it'd be useful to have a pm-dir wiki listing each of the >>>> drafts below. eg, one page per draft, with one section per version, >>>> containing reviewer comments pertinent to that version, even if the >>>> comments simply say, "reviewed, no issues". >>>> >>>> Hopefully I missed something, and most (if not all?) of this is >>>> already in place? :-) >>>> >>>> Thanks, >>>> P. >>>> >>>> >>>> On 04/02/13 11:53, Benoit Claise wrote: >>>>> Dear all, >>>>> >>>>> This is an automatically generated email. >>>>> It lists the IETF internet-drafts that reference the PMOL RFC 6390, >>>>> as a normative or informative reference. >>>>> It also lists all the IETF internet-drafts that contain "performance >>>>> metric". >>>>> >>>>> Regards, Benoit >>>>> >>>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>> >>>>> Normative References >>>>> -------------------- >>>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>>>> Informative References >>>>> ---------------------- >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>>>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>>>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>>>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>>>> >>>>> drafts containing performance metric >>>>> ------------------------------------ >>>>> draft-ietf-alto-deployments-05 Active >>>>> draft-ietf-alto-protocol-13 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-ippm-rate-problem-02 Active >>>>> draft-ietf-ippm-testplan-rfc2680-01 Active >>>>> draft-ietf-karp-threats-reqs-07 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-manet-olsrv2-mib-05 Active >>>>> draft-ietf-manet-smf-mib-06 Active >>>>> draft-ietf-mmusic-media-loopback-27 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-nvo3-framework-01 Active >>>>> draft-ietf-opsawg-oam-overview-08 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-rtcweb-rtp-usage-05 Active >>>>> draft-ietf-rtgwg-cl-framework-02 Active >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>>>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>>>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>>>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>>>> draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing >>>>> - ID Tracker state >>>>> draft-ietf-xrblock-rtcp-xr-synchronization-02 Active >>>>> _______________________________________________ >>>>> pm-dir mailing list >>>>> pm-dir@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/pm-dir >>>> _______________________________________________ >>>> pm-dir mailing list >>>> pm-dir@ietf.org >>>> https://www.ietf.org/mailman/listinfo/pm-dir >>>> >>>> >> _______________________________________________ >> pm-dir mailing list >> pm-dir@ietf.org >> https://www.ietf.org/mailman/listinfo/pm-dir > --_000_F1312FAF1A1E624DA0972D1C9A91379A1BF83AFB60njfpsrvexg7re_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable 
> Once we handle the initial rush of drafts from the
> script, I would suggest that t=
he script keep last week's list and
> highlight the new arrivals to facilitate assignment, rather t=
han
> leave this step fo=
r manual checking.
That wou=
ld nice, but we would need a kind of database to keep track of <=
/pre>
the already assigned drafts.
It starts to be a more complex scrip=
t...
 
It doesn't have to keep track of assignments, just last week's list.<=
o:p>
IOW, highlight the drafts that are new this week, an a=
utomated "uniq" list
at the top of the mail.=
..
    =
              <=
/o:p>
Note: Alan (in the copy list) is=
 improving the script so that the drafts 
that maps a keyword (performance metric in this case) and th=
at have a 
reference to RFC=
 6390 doesn't appear twice.
 
great, that's a good start.
 
<=
pre>I'm not requesting any volunteers right now, 
with=
 the draft deadlines looming...
 
Al
 
<=
span style=3D'font-size:10.0pt;font-family:"Courier New";color:windowtext'>=
 
 <=
/p>
From: Benoit Claise [mailto:bclaise@cisco.com] 
Sent: Wednesday=
, February 13, 2013 8:03 AM
To: MORTON JR., ALFRED (AL)
Sub=
ject: Fwd: Re: [pm-dir] Performance metrics doctors generated email
 
Hi Al,

Can you please take care of setting up a WI=
KI.
Thanks in advance.

Regards, Benoit


-------- Original Message -------- 
<=
td nowrap valign=3Dtop style=3D'padding:0in 0in 0in 0in'>
Subject: 
=

Re: [pm-dir=
] Performance metrics doctors generated email
<=
td nowrap valign=3Dtop style=3D'padding:0in 0in 0in 0in'>
Date: 
Wed, 13 Feb 20=
13 14:01:47 +0100
From: 
Benoit Claise <bclaise@cisco.com>
To: 
MORTON JR., ALFRED (AL)=
 <acmorton@att.com>
CC: 
Paul Aitken <paitken@cisco.com&=
gt;, "pm-dir@ietf.org"=
 <pm-dir@ietf.org>, Alan DeKok=
 <aland@freeradius.org>
 
Al, All,
> Hi Pau=
l,
> 
> Our generally =
accepted mode of operation is in the message
> appe=
nded below.  We need to make a few changes now that the
> script is finding more drafts than there are Directorate member=
s.
> 
> We could choos=
e to expand our membership, that's a question I
> a=
sked the Directorate a few weeks ago, but with no feedback there=

> was no action.
>    =
  When the new list was announced, several people tried to join.<=
/o:p>
>      We should probably have a cal=
l for new members soon,
>      =
if the current Directorate members agree.  Also, if anyone would<=
/o:p>
>      like to retire from the Perfo=
rmance Directorate (for example,
>   &nbs=
p;  if you didn't find time to perform reviews in 2012, then
>      you might consider the question i=
n that light), please let me know.
> 
> I agree that we need a more accessible and now, more dyna=
mic means of
> organization.  A Wiki will work=
, as long as everyone has access to it
> and can up=
date draft status as they complete reviews, etc.
Agree=
d.
> 
> Once we handle=
 the initial rush of drafts from the
> script, I wo=
uld suggest that the script keep last week's list and
=
> highlight the new arrivals to facilitate assignment, rather than<=
/o:p>
> leave this step for manual checking.
<=
pre>That would nice, but we would need a kind of database to keep track of =

the already assigned drafts.
It =
starts to be a more complex script...
 
Note: Alan (in the copy list) is improving the script so that t=
he drafts 
that maps a keyword (performance metric in =
this case) and that have a 
reference to RFC 6390 does=
n't appear twice.
 
Regards,=
 Benoit
> 
> regards,<=
o:p>
> Al
> PM Dir admin
> 
> -=3D-=3D-=3D-=3D-=3D-=3D=
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-
=
> At 09:47 AM 1/16/2012, Al Morton wrote:
> As t=
his new year goes into full swing, I'd like to offer some
<=
pre>> thoughts on how we might operate as a Directorate and
> hear from others.
> <=
/pre>
> As our web page says: "RFC 6390 is used as a reference =
for the technical
> and process issues." Have =
your copy handy when reviewing a draft.
> A lot of =
interactions may be streamlined by pointing the authors to 6390,=

> and mentioning that it is a BCP.
><=
o:p> 
> I feel that our reviews should be *early* i=
n the draft development
> process, and that no perf=
ormance-related draft should start IETF Last Call
>=
 without previous review against 6390, unless it truly slipped by us
> and other folks who know about the Directorate.
> 
> There are two ways we ca=
n collect drafts for review:
> 
> - WG Chairs and ADs can/should suggest drafts to us<=
/pre>
> 
> - Directorate Members will s=
ee drafts, hear about drafts with performance
> wor=
k, etc.  We are the eyes and ears of the Directorate. We all have frie=
nds
> working in other areas of IETF, let them know=
 what we're looking for.
> 
> I suggest to split the early review into two categories:=

> 
> 1. Quick Scan - If you find=
 a draft, do a quick scan or suggest that
> others =
take a look with a message to pmol-list.  Finding a draft does
> not mean that you will be designated to do a review (see=
 below).
> 
> 2. WG Ca=
ndidate/Chartered Item - When a draft appears to have enough value
> and interest to become a WG item, or the WG Chairs involv=
ed recommend a
> Performance Metrics Directorate re=
view on their own, then we do a more
> complete rev=
iew against 6390.
> 
>=
 If these categories are useful, then I'd like to suggest that *two*
> reviewers are assigned to WG Candidate/Chartered drafts=
 and they follow
> the draft after early review. Th=
is may mean looking at the draft again
> in WG or I=
ETF Last Call, assuming it goes that far. I suggest 2 reviewers<=
/pre>
> while we're all new at this, and to help ensure that one is =
still around
> years later when Last Calls are done=
.
> 
> All drafts iden=
tified, quick scans, and reviews should be sent to the
> pmol-list and the authors (and possibly the relevant WG chairs)<=
/o:p>
> 
> let us all know what y=
ou think,
> Al
> =

>> -----Original Message-----
&=
gt;> From: pm-dir-bounces@iet=
f.org [mailto:pm-dir-bounces=
@ietf.org] On Behalf
>> Of Paul Aitken<=
/o:p>
>> Sent: Monday, February 04, 2013 7:50 AM=

>> To: Benoit Claise
>> Cc: pm-dir@ietf.org
&g=
t;> Subject: Re: [pm-dir] Performance metrics doctors generated email
>> 
>> Benoit,
>> 
>>> Yes, all=
 these questions are important.
>>> Yes a WIK=
I is the right solution IMHO.
>> Great. Where ca=
n it be hosted?
>> 
&g=
t;> eg, is http=
://trac.tools.ietf.org/wg/pmol/trac/wiki a good starting
>> place? Or is there a pm-dir page?
>=
> 
>> P.
>> 
>>>> Benoit, All,
>>>> 
>>>> How is pm-di=
r tracking these drafts? ie, how do we know whether
&g=
t;>>> someone is currently reviewing, or has already reviewed, eac=
h of
>>>> these drafts?
>>>> 
>>>> How are rev=
iewers selected? By relevance, skill, luck, or misfortune?
=

>>>> 
>>>> Also, it'=
d be useful to know whether the previous version was already
>>>> reviewed, and what the feedback was:
>>>> 
>>>> &n=
bsp;    * issues were raised, recommendations were made, cha=
nges were
>>>> expected -> some re-revi=
ew should be done
>>>> 
>>>>      * no issues were rai=
sed -> a quick review of the delta may be
>>&=
gt;> sufficient, to check for any new issues
>&g=
t;>> 
>>>>   &nb=
sp;  * previous version wasn't reviewed -> a thorough review may be=

>>>> required.
>&=
gt;>> 
>>>> 
>>>> I'm thinking it'd be useful to have a pm-dir wiki =
listing each of the
>>>> drafts below. eg,=
 one page per draft, with one section per version,
>=
;>>> containing reviewer comments pertinent to that version, even =
if the
>>>> comments simply say, "rev=
iewed, no issues".
>>>> 
>>>> Hopefully I missed something, and most (if n=
ot all?) of this is
>>>> already in place?=
 :-)
>>>> 
>&=
gt;>> Thanks,
>>>> P.
>>>> 
>>>>&nb=
sp;
>>>> On 04/02/13 11:53, Benoit Claise wrote=
:
>>>>> Dear all,
=
>>>>> 
>>>>> This i=
s an automatically generated email.
>>>>&g=
t; It lists the IETF internet-drafts that reference the PMOL RFC 6390,=

>>>>> as a normative or informative referen=
ce.
>>>>> It also lists all the IETF in=
ternet-drafts that contain "performance
>>&=
gt;>> metric".
>>>>>&nb=
sp;
>>>>> Regards, Benoit
>>>>> 
>>>>> =3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D
>>>>>&nbs=
p;
>>>>> Normative References
>>>>> --------------------
>&=
gt;>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
>>>>>      =
 Informative References
>>>>> ---------=
-------------
>>>>> draft-ietf-xrblock-=
rtcp-xr-burst-gap-discard-10   In IESG processing
>>>>> - ID Tracker state <Publication Requested>=

>>>>> draft-ietf-xrblock-rtcp-xr-burst=
-gap-loss-08      In IESG processing
>>>>> - ID Tracker state <Publication Requested&g=
t;
>>>>> draft-ietf-xrblock-rtcp-xr-con=
csec-03           &n=
bsp; Active
>>>>> draft-ietf-xrblock-rt=
cp-xr-decodability-07        Active=

>>>>> draft-ietf-xrblock-rtcp-xr-discard-11=
             Ac=
tive
>>>>> draft-ietf-xrblock-rtcp-xr-j=
b-07            &nbs=
p;     Active
>>>>=
;> draft-ietf-xrblock-rtcp-xr-loss-conceal-04    &nb=
sp;   Active
>>>>> draft-ietf=
-xrblock-rtcp-xr-qoe-03        &nbs=
p;        Active
&g=
t;>>>> 
>>>>> drafts c=
ontaining performance metric
>>>>> ----=
--------------------------------
>>>>> =
draft-ietf-alto-deployments-05       &nb=
sp;            Activ=
e
>>>>> draft-ietf-alto-protocol-13&nbs=
p;            &=
nbsp;         In IESG processing
>>>>> - ID Tracker state <AD Evaluatio=
n::Revised ID Needed>
>>>>> draft-ie=
tf-ippm-rate-problem-02        &nbs=
p;          Active<=
/pre>
>>>>> draft-ietf-ippm-testplan-rfc2680-01 &nb=
sp;            =
 Active
>>>>> draft-ietf-karp-threats-r=
eqs-07           &nb=
sp;       In IESG processing
=

>>>>> - ID Tracker state <RFC Ed Queue>
>>>>> draft-ietf-manet-olsrv2-mib-05  =
            &nb=
sp;     Active
>>>>>=
; draft-ietf-manet-smf-mib-06       &nbs=
p;            &=
nbsp;  Active
>>>>> draft-ietf-mmu=
sic-media-loopback-27         =
      In IESG processing
>=
>>>> - ID Tracker state <RFC Ed Queue>
>>>>> draft-ietf-nvo3-framework-01    =
;            &n=
bsp;     Active
>>>>&g=
t; draft-ietf-opsawg-oam-overview-08       &n=
bsp;         In IESG processin=
g
>>>>> - ID Tracker state <Waiting =
for AD Go-Ahead::Revised ID Needed>
>>>>=
;> draft-ietf-rtcweb-rtp-usage-05      &nb=
sp;            =
 Active
>>>>> draft-ietf-rtgwg-cl-frame=
work-02           &n=
bsp;      Active
>>>=
>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IES=
G processing
>>>>> - ID Tracker state &=
lt;Publication Requested>
>>>>> draf=
t-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In I=
ESG processing
>>>>> - ID Tracker state=
 <Publication Requested>
>>>>> dr=
aft-ietf-xrblock-rtcp-xr-concsec-03      &nbs=
p;      Active
>>>&g=
t;> draft-ietf-xrblock-rtcp-xr-decodability-07    &n=
bsp;   Active
>>>>> draft-iet=
f-xrblock-rtcp-xr-discard-11        =
;     Active
>>>>> =
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
>>>>> draft-ietf-xrblock-rtcp-xr-jb-07   &=
nbsp;           &nbs=
p;  Active
>>>>> draft-ietf-xrbloc=
k-rtcp-xr-loss-conceal-04        Active<=
o:p>
>>>>> draft-ietf-xrblock-rtcp-xr-qoe-03=
            &nb=
sp;    Active
>>>>> draf=
t-ietf-xrblock-rtcp-xr-summary-stat-06      &=
nbsp; In IESG processing
>>>>> - ID Tra=
cker state <Waiting for AD Go-Ahead>
>>>=
;>> draft-ietf-xrblock-rtcp-xr-synchronization-02   &n=
bsp; Active
>>>>> _____________________=
__________________________
>>>>> pm-dir=
 mailing list
>>>>> pm-dir@ietf.org
>>>>&g=
t; https://www.iet=
f.org/mailman/listinfo/pm-dir
>>>> ___=
____________________________________________
>>&=
gt;> pm-dir mailing list
>>>> pm-dir@ietf.org
>&g=
t;>> https:/=
/www.ietf.org/mailman/listinfo/pm-dir
>>>=
> 
>>>> 
>> _______________________________________________
<=
pre>>> pm-dir mailing list
>> pm-dir@ietf.org
>> https://www.ietf.org=
/mailman/listinfo/pm-dir
> 
 
 
 
=

--_000_F1312FAF1A1E624DA0972D1C9A91379A1BF83AFB60njfpsrvexg7re_--

From bclaise@cisco.com  Thu Feb 14 15:19:16 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D8D8521F8959 for ; Thu, 14 Feb 2013 15:19:16 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.921
X-Spam-Level: 
X-Spam-Status: No, score=-9.921 tagged_above=-999 required=5 tests=[AWL=-0.563, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_LWSHORTT=1.24]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gkPhLmt1LbWW for ; Thu, 14 Feb 2013 15:19:14 -0800 (PST)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 5FA4121F8921 for ; Thu, 14 Feb 2013 15:19:14 -0800 (PST)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1ENJ6Fl000643; Fri, 15 Feb 2013 00:19:06 +0100 (CET)
Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1ENIU0H028709; Fri, 15 Feb 2013 00:18:40 +0100 (CET)
Message-ID: <511D70C6.2080601@cisco.com>
Date: Fri, 15 Feb 2013 00:18:30 +0100
From: Benoit Claise 
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2
MIME-Version: 1.0
To: "draft-ietf-xrblock-rtcp-xr-summary-stat@tools.ietf.org" 
References: <50F93070.7040107@ericsson.com> <51057B67.7090205@cisco.com> 
In-Reply-To: 
Content-Type: multipart/alternative; boundary="------------080403010001070800020903"
Cc: "xrblock-chairs@tools.ietf.org" , "pm-dir@ietf.org" , "MORTON JR., ALFRED \(AL\)" , Gonzalo Camarillo 
Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-xrblock-rtcp-xr-summary-stat-06
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 14 Feb 2013 23:19:17 -0000

This is a multi-part message in MIME format.
--------------080403010001070800020903
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Dear authors,

I'm reviewing version 8 for the IESG call next week.
> Benoit,
>
> Since there is a short term deadline
> (the Last Call has ended on Feb 1)
> I'll provide the PM Dir review.
>
> Al
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Since the draft describes reporting formats and (apparently) does not
> intend to define new metrics, much of 6390 guidance is out of scope.
This is the key question.
Do you define new metrics?
When I see the following (to take just one example):

      Burst Loss Rate: 16 bits

           The fraction of packets lost during bursts since the beginning of
           reception, expressed as a fixed point number with the binary point
           at the left edge of the field.  This value is calculated by
           dividing Packets Loss in Bursts by Total Packets expected in
           Bursts, multiplying the result of the division by 7FFF, with the
           maximum value 7FFF, and taking the integer part as follows:

           Packets Loss in Bursts / Total Packets expected in Bursts

           If the measurement is unavailable, the value 0x8000 MUST be
           reported.

... it seems to me that you define new metrics, and that therefore the 
RFC 6390 template should be applied.
What do I miss?

Regards, Benoit
> However, see the comments on section 4.1.2 below regarding metric definitions.
>
> Comments:
>
> Section 1.1 of the Intro says:
>
>     This draft defines three new block types to augment those defined in
>     [RFC3611] for use in a range of RTP applications.
>
> ok so far, but the next paragraph is hard to decipher, perhaps a simple
> numbered list of the new blocks would help:
>
>      1. Burst/Gap Loss Summary Statistics Metrics Block
>      2. Burst/Gap Discard Summary Statistics Metrics Block
>      3. Frame Impairment Statistics Summary Metrics Block
>
> back to the existing text:
>     The first two block types support the reporting of burst gap loss/
>     discard summary statistics including packet loss/discard proportion,
>     mean and variance and belong to the class of transport-related end
>     system metrics defined in [RFC6792].  These two blocks are intended
>     to be used in conjunction with information from the Burst Gap Loss
>     Metrics Block or Burst Gap Discard Metrics Block, and on which these
>     two block therefore depend.
> add a reference to [RFC3611] for these two blocks.
>
>     The metrics in the Burst Gap Loss block
>     or Burst Gap Discard Metrics Block can be used independently of the
>     metrics defined in the first two blocks.
>
> That's obvious, because anyone implementing RFC3611 is already doing it.
>
> The authors appear to have avoided using "video" as an adjective when
> referring to video frames throughout the memo. For example:
>
>     The third block supports the reporting of detailed statistics for
>     each frame type, including the number of frames received, lost and
>     discarded of each frame type in the Group of Pictures (GOP) and
>     additional data allowing the calculation of statistical parameters
>     (e.g.,the proportion of each frame type impaired by packet loss and
>     discard).  The metrics defined in this block belong to the class of
>     application layer metrics defined in [RFC6792].
>
> These are all video frames above, no?  At least say so once.
>
> Section 2.1, on Terminology, says
>
>      Picture Type
>
>        Picture Types used in the different video algorithms are composed
>        of the Key frame and Derived frames.  The Key frame is also called
>        a reference frame and used as a reference for predicting other
>        pictures.  It is coded without prediction from other pictures.
>        Derived frames are derived from a Key frame using a prediction
>        algorithm.
>
> At first this seems quite generic, but the definition may fail when one slice
> of the video frame is independently coded, and other slices are predictively
> coded (this has been done to smooth the video rate). Is any video frame that
> includes independent or reference coding a Key frame?
>
> Section 3.1.2 says:
>     Burst Loss Rate: 16 bits
>
>        The fraction of packets lost during bursts since the beginning of
>        reception, expressed as a fixed point number with the binary point
>        at the left edge of the field.  This value is calculated by
>        dividing Packets Loss in Bursts by Total Packets expected in
>        Bursts as follows:
>            Packets Loss in Bursts / Total Packets expected in Bursts
>
> How is a burst loss ratio = 1.0 reported?  All the digits are to the
> right of the decimal place.
>
>     Burst Duration Mean:16bits
>
>        The mean burst duration is obtained as the quotient:
>
>        mean = Sum of Burst Durations / Number of Bursts
>
> How is Divide by Zero handled?
>
>     Burst Duration Variance:16bits
>
>        The variance of the burst duration is obtained using the standard
>        result:
>
>        var = ( Sum of Squares of Burst Durations - Number of Bursts *
>        mean^2 ) / (Number of Bursts - 1)
>
> How is Divide by Zero handled? (e.g., Number of Bursts = 1)
>
> Section 4.1.2 says
>
>     Number of full frames lost (lost_full_frames): 32 bits
>
>        If one frame is completely lost, this frame is regarded as one
>        lost full frame.  The lost_full_frames is equivalent to the number
>        of full frames lost in the above sequence number interval.
>
> Is this a metric definition, fully lost video frames?
> How is this event detected so it can be counted?
> Is there a reference for this metric available?
>
>     Number of partial frames lost (lost_partial_frames): 32 bits
>
>        If one frame is partially lost, this frame is regarded as one lost
>        fractional frame.  The value of the lost_partial_frames field is
>        equivalent to the number of partial frames lost in the above
>        sequence number interval.
>
> Is this a metric definition, partially lost video frames?
> How is this event detected so it can be counted?
> Is there a reference for this metric available?
>
> -=-=-=-=-=-=-
>
>> -----Original Message-----
>> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
>> Of Benoit Claise
>> Sent: Sunday, January 27, 2013 2:09 PM
>> To: pm-dir@ietf.org
>> Cc: xrblock-chairs@tools.ietf.org; draft-ietf-xrblock-rtcp-xr-summary-
>> stat@tools.ietf.org; Gonzalo Camarillo
>> Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-
>> xrblock-rtcp-xr-summary-stat-06
>>
>> Dear all,
>>
>> Here is an official request for the RFC 6390 review.
>> Who would like to volunteer?
>>
>> Regards, Benoit
>>> Hi Benoit,
>>>
>>> as suggested by Dan in his PROTO writeup, I would like to request the
>>> performance metrics directorate to perform an RFC 6390 review of the
>>> following draft as part of its IETF LC:
>>>
>>> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-
>> stat/
>>> Could you please arrange such a review?
>>>
>>> Thanks,
>>>
>>> Gonzalo
>>>
>>>
>> _______________________________________________
>> pm-dir mailing list
>> pm-dir@ietf.org
>> https://www.ietf.org/mailman/listinfo/pm-dir
>


--------------080403010001070800020903
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  
    
  
  
    
Dear authors,

      

      I'm reviewing version 8 for the IESG call next week.

    

    

      
Benoit,

Since there is a short term deadline 
(the Last Call has ended on Feb 1) 
I'll provide the PM Dir review.

Al

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Since the draft describes reporting formats and (apparently) does not 
intend to define new metrics, much of 6390 guidance is out of scope.

    

    This is the key question.

    Do you define new metrics?

    When I see the following (to take just one example):

    

      
 Burst Loss Rate: 16 bits

      The fraction of packets lost during bursts since the beginning of
      reception, expressed as a fixed point number with the binary point
      at the left edge of the field.  This value is calculated by
      dividing Packets Loss in Bursts by Total Packets expected in
      Bursts, multiplying the result of the division by 7FFF, with the
      maximum value 7FFF, and taking the integer part as follows:

      Packets Loss in Bursts / Total Packets expected in Bursts

      If the measurement is unavailable, the value 0x8000 MUST be
      reported.

    

    ... it seems to me that you define new metrics, and that therefore
    the RFC 6390 template should be applied.

    What do I miss?

    

    Regards, Benoit

    

      
However, see the comments on section 4.1.2 below regarding metric definitions.

Comments:

Section 1.1 of the Intro says:

   This draft defines three new block types to augment those defined in
   [RFC3611] for use in a range of RTP applications.

ok so far, but the next paragraph is hard to decipher, perhaps a simple
numbered list of the new blocks would help:

    1. Burst/Gap Loss Summary Statistics Metrics Block
    2. Burst/Gap Discard Summary Statistics Metrics Block
    3. Frame Impairment Statistics Summary Metrics Block

back to the existing text:
   The first two block types support the reporting of burst gap loss/
   discard summary statistics including packet loss/discard proportion,
   mean and variance and belong to the class of transport-related end
   system metrics defined in [RFC6792].  These two blocks are intended
   to be used in conjunction with information from the Burst Gap Loss
   Metrics Block or Burst Gap Discard Metrics Block, and on which these
   two block therefore depend.  
add a reference to [RFC3611] for these two blocks.

   The metrics in the Burst Gap Loss block
   or Burst Gap Discard Metrics Block can be used independently of the
   metrics defined in the first two blocks.

That's obvious, because anyone implementing RFC3611 is already doing it.

The authors appear to have avoided using "video" as an adjective when 
referring to video frames throughout the memo. For example:

   The third block supports the reporting of detailed statistics for
   each frame type, including the number of frames received, lost and
   discarded of each frame type in the Group of Pictures (GOP) and
   additional data allowing the calculation of statistical parameters
   (e.g.,the proportion of each frame type impaired by packet loss and
   discard).  The metrics defined in this block belong to the class of
   application layer metrics defined in [RFC6792].

These are all video frames above, no?  At least say so once.

Section 2.1, on Terminology, says

    Picture Type

      Picture Types used in the different video algorithms are composed
      of the Key frame and Derived frames.  The Key frame is also called
      a reference frame and used as a reference for predicting other
      pictures.  It is coded without prediction from other pictures.
      Derived frames are derived from a Key frame using a prediction
      algorithm.

At first this seems quite generic, but the definition may fail when one slice
of the video frame is independently coded, and other slices are predictively
coded (this has been done to smooth the video rate). Is any video frame that
includes independent or reference coding a Key frame?

Section 3.1.2 says:
   Burst Loss Rate: 16 bits

      The fraction of packets lost during bursts since the beginning of
      reception, expressed as a fixed point number with the binary point
      at the left edge of the field.  This value is calculated by
      dividing Packets Loss in Bursts by Total Packets expected in
      Bursts as follows:
          Packets Loss in Bursts / Total Packets expected in Bursts

How is a burst loss ratio = 1.0 reported?  All the digits are to the 
right of the decimal place.

   Burst Duration Mean:16bits

      The mean burst duration is obtained as the quotient:

      mean = Sum of Burst Durations / Number of Bursts

How is Divide by Zero handled?

   Burst Duration Variance:16bits

      The variance of the burst duration is obtained using the standard
      result:

      var = ( Sum of Squares of Burst Durations - Number of Bursts *
      mean^2 ) / (Number of Bursts - 1)

How is Divide by Zero handled? (e.g., Number of Bursts = 1)

Section 4.1.2 says

   Number of full frames lost (lost_full_frames): 32 bits

      If one frame is completely lost, this frame is regarded as one
      lost full frame.  The lost_full_frames is equivalent to the number
      of full frames lost in the above sequence number interval.

Is this a metric definition, fully lost video frames? 
How is this event detected so it can be counted?
Is there a reference for this metric available?

   Number of partial frames lost (lost_partial_frames): 32 bits

      If one frame is partially lost, this frame is regarded as one lost
      fractional frame.  The value of the lost_partial_frames field is
      equivalent to the number of partial frames lost in the above
      sequence number interval.

Is this a metric definition, partially lost video frames? 
How is this event detected so it can be counted?
Is there a reference for this metric available?

-=-=-=-=-=-=-



      

        
-----Original Message-----
From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
Of Benoit Claise
Sent: Sunday, January 27, 2013 2:09 PM
To: pm-dir@ietf.org
Cc: xrblock-chairs@tools.ietf.org; draft-ietf-xrblock-rtcp-xr-summary-
stat@tools.ietf.org; Gonzalo Camarillo
Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-
xrblock-rtcp-xr-summary-stat-06

Dear all,

Here is an official request for the RFC 6390 review.
Who would like to volunteer?

Regards, Benoit


        

          
Hi Benoit,

as suggested by Dan in his PROTO writeup, I would like to request the
performance metrics directorate to perform an RFC 6390 review of the
following draft as part of its IETF LC:

https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-


        

        
stat/


        

          
Could you please arrange such a review?

Thanks,

Gonzalo




        

        
_______________________________________________
pm-dir mailing list
pm-dir@ietf.org
https://www.ietf.org/mailman/listinfo/pm-dir


      

      



    

    

  


--------------080403010001070800020903--

From paitken@cisco.com  Fri Feb 15 03:11:34 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 707D421F8976 for ; Fri, 15 Feb 2013 03:11:34 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level: 
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eC4EHO-NnfDZ for ; Fri, 15 Feb 2013 03:11:33 -0800 (PST)
Received: from ams-iport-4.cisco.com (ams-iport-4.cisco.com [144.254.224.147]) by ietfa.amsl.com (Postfix) with ESMTP id 5C73421F896B for ; Fri, 15 Feb 2013 03:11:33 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=4446; q=dns/txt; s=iport; t=1360926693; x=1362136293; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=QFEnHBMz4/9iCB10Wk8SpTsOYqXpT4U0CVlasR6rA4U=; b=guEEX2vnJ40TMlKPwD/5ijKsqkwYiG7bGDHAyJ/6z3BQOCV6Brmj0D0d WNX9U3DDf2IqXyaQsgq/7sw3QzkrhvBQUndpdQGFxPDuugbOX73LFQNuS eXDfzzto3moOP2mZ8Hu39oeZojTAfZqZiEaVDrmTOzECmsjnmI6aJMCg7 s=;
X-IronPort-AV: E=Sophos;i="4.84,673,1355097600"; d="scan'208";a="11823009"
Received: from ams-core-2.cisco.com ([144.254.72.75]) by ams-iport-4.cisco.com with ESMTP; 15 Feb 2013 11:11:32 +0000
Received: from [10.55.82.22] (dhcp-10-55-82-22.cisco.com [10.55.82.22]) by ams-core-2.cisco.com (8.14.5/8.14.5) with ESMTP id r1FBBV6j011883; Fri, 15 Feb 2013 11:11:31 GMT
Message-ID: <511E17E4.1020809@cisco.com>
Date: Fri, 15 Feb 2013 11:11:32 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2
MIME-Version: 1.0
To: Benoit Claise 
References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com>
In-Reply-To: <510FA460.3060102@cisco.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: pm-dir@ietf.org, Alan DeKok 
Subject: Re: [pm-dir] Performance metrics doctors generated email - new version.
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 15 Feb 2013 11:11:34 -0000

Benoit,

How often is the script run? I haven't seen any more postings these last 
10 days.

If it's run regularly - eg, weekly or nightly - then it'd be useful as 
Al said, to send a delta from the previous run so we can easily see what 
changed.

I saved the current(?) list at http://trac.tools.ietf.org/wg/pmol/trac/wiki
You could imagine additional columns, eg reviewer, comments, status, so 
the status and history can be seen at a glance.

Is this useful?

P.


On 04/02/13 12:06, Benoit Claise wrote:
> Dear all,
>
> This is a new version of the script, provided by Alan Dekok. Many 
> thanks Alan.
> The important improvement is that we also list the drafts that contain 
> a specific keyword.
> In this case, "performance metric". We could add some more...
> With this new script, we can then engage sooner with the people who 
> don't know about this directorate and RFC 6390.
>
> Potential small optimization for the script: remove the drafts that 
> contain "performance metric" and that already reference RFC 6390
>
> Regards, Benoit
>> Dear all,
>>
>> This is an automatically generated email.
>> It lists the IETF internet-drafts that reference the PMOL RFC 6390, 
>> as a normative or informative reference.
>> It also lists all the IETF internet-drafts that contain "performance 
>> metric".
>>
>> Regards, Benoit
>>
>> ===========================================================
>>
>> Normative References
>> --------------------
>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
>>      Informative References
>> ----------------------
>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing 
>> - ID Tracker state 
>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing 
>> - ID Tracker state 
>> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
>> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
>> draft-ietf-xrblock-rtcp-xr-discard-11             Active
>> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
>> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
>>
>> drafts containing performance metric
>> ------------------------------------
>> draft-ietf-alto-deployments-05                    Active
>> draft-ietf-alto-protocol-13                       In IESG processing 
>> - ID Tracker state 
>> draft-ietf-ippm-rate-problem-02                   Active
>> draft-ietf-ippm-testplan-rfc2680-01               Active
>> draft-ietf-karp-threats-reqs-07                   In IESG processing 
>> - ID Tracker state 
>> draft-ietf-manet-olsrv2-mib-05                    Active
>> draft-ietf-manet-smf-mib-06                       Active
>> draft-ietf-mmusic-media-loopback-27               In IESG processing 
>> - ID Tracker state 
>> draft-ietf-nvo3-framework-01                      Active
>> draft-ietf-opsawg-oam-overview-08                 In IESG processing 
>> - ID Tracker state 
>> draft-ietf-rtcweb-rtp-usage-05                    Active
>> draft-ietf-rtgwg-cl-framework-02                  Active
>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing 
>> - ID Tracker state 
>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing 
>> - ID Tracker state 
>> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
>> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
>> draft-ietf-xrblock-rtcp-xr-discard-11             Active
>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
>> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
>> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
>> draft-ietf-xrblock-rtcp-xr-summary-stat-06        In IESG processing 
>> - ID Tracker state 
>> draft-ietf-xrblock-rtcp-xr-synchronization-02     Active
>> _______________________________________________
>> pm-dir mailing list
>> pm-dir@ietf.org
>> https://www.ietf.org/mailman/listinfo/pm-dir
>>
>>
>
> _______________________________________________
> pm-dir mailing list
> pm-dir@ietf.org
> https://www.ietf.org/mailman/listinfo/pm-dir


From acmorton@att.com  Fri Feb 15 05:20:22 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 58FD021F8ADF for ; Fri, 15 Feb 2013 05:20:22 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.416
X-Spam-Level: 
X-Spam-Status: No, score=-106.416 tagged_above=-999 required=5 tests=[AWL=0.183, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yZ7NxAWZQoue for ; Fri, 15 Feb 2013 05:20:20 -0800 (PST)
Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 0538421F8AC3 for ; Fri, 15 Feb 2013 05:20:20 -0800 (PST)
Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 2A234121016; Fri, 15 Feb 2013 08:22:24 -0500 (EST)
Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-green.research.att.com (Postfix) with ESMTP id 66215E3734; Fri, 15 Feb 2013 08:14:07 -0500 (EST)
Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Fri, 15 Feb 2013 08:20:19 -0500
From: "MORTON JR., ALFRED  (AL)" 
To: Paul Aitken , Benoit Claise 
Date: Fri, 15 Feb 2013 08:20:17 -0500
Thread-Topic: [pm-dir] Performance metrics doctors generated email - new version.
Thread-Index: Ac4LbURckiyR+TUyTMWgfaGQhSt+MwADyqOQ
Message-ID: 
References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com>
In-Reply-To: <511E17E4.1020809@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "pm-dir@ietf.org" , Alan DeKok 
Subject: Re: [pm-dir] Performance metrics doctors generated email - new	version.
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Fri, 15 Feb 2013 13:20:22 -0000

Hi Paul,

Thanks for continuing the discussion on tracking
draft review assignments and status.  I've been discussing=20
this off-list with Benoit. Let me put my position on
the table here.

I'd prefer a solution where:
 - everyone can view the page
 - directorate reviewers can edit the page
 - all editing is WYSIWYG
 - no collisions between editors
  (if two people are editing, they can see each other and=20
   pressing save does not overwrite a nested editing session)

Unfortunately, the last two points are beyond Trac's capability.
I think that I could create a doc on Google Drive that
would meet all the above criteria.
 =20
But, all Directorate members who are active/performing reviews
would need to have an account on Google and be willing to use
it for this purpose. I suspect that many of us already have
a gmail account.

If there is another solution that meets all the points above,
we could consider that as well.

Let's hear from all Directorate members on this, it's a cross-road
that will affect how we work together in the future.

regards,
Al


> -----Original Message-----
> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
> Of Paul Aitken
> Sent: Friday, February 15, 2013 6:12 AM
> To: Benoit Claise
> Cc: pm-dir@ietf.org; Alan DeKok
> Subject: Re: [pm-dir] Performance metrics doctors generated email - new
> version.
>=20
> Benoit,
>=20
> How often is the script run? I haven't seen any more postings these last
> 10 days.
>=20
> If it's run regularly - eg, weekly or nightly - then it'd be useful as
> Al said, to send a delta from the previous run so we can easily see what
> changed.
>=20
> I saved the current(?) list at
> http://trac.tools.ietf.org/wg/pmol/trac/wiki
> You could imagine additional columns, eg reviewer, comments, status, so
> the status and history can be seen at a glance.
>=20
> Is this useful?
>=20
> P.
>=20
>=20
> On 04/02/13 12:06, Benoit Claise wrote:
> > Dear all,
> >
> > This is a new version of the script, provided by Alan Dekok. Many
> > thanks Alan.
> > The important improvement is that we also list the drafts that contain
> > a specific keyword.
> > In this case, "performance metric". We could add some more...
> > With this new script, we can then engage sooner with the people who
> > don't know about this directorate and RFC 6390.
> >
> > Potential small optimization for the script: remove the drafts that
> > contain "performance metric" and that already reference RFC 6390
> >
> > Regards, Benoit
> >> Dear all,
> >>
> >> This is an automatically generated email.
> >> It lists the IETF internet-drafts that reference the PMOL RFC 6390,
> >> as a normative or informative reference.
> >> It also lists all the IETF internet-drafts that contain "performance
> >> metric".
> >>
> >> Regards, Benoit
> >>
> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> >>
> >> Normative References
> >> --------------------
> >> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
> >>      Informative References
> >> ----------------------
> >> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
> >> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
> >> draft-ietf-xrblock-rtcp-xr-discard-11             Active
> >> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
> >> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
> >> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
> >>
> >> drafts containing performance metric
> >> ------------------------------------
> >> draft-ietf-alto-deployments-05                    Active
> >> draft-ietf-alto-protocol-13                       In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-ippm-rate-problem-02                   Active
> >> draft-ietf-ippm-testplan-rfc2680-01               Active
> >> draft-ietf-karp-threats-reqs-07                   In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-manet-olsrv2-mib-05                    Active
> >> draft-ietf-manet-smf-mib-06                       Active
> >> draft-ietf-mmusic-media-loopback-27               In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-nvo3-framework-01                      Active
> >> draft-ietf-opsawg-oam-overview-08                 In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-rtcweb-rtp-usage-05                    Active
> >> draft-ietf-rtgwg-cl-framework-02                  Active
> >> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-xrblock-rtcp-xr-concsec-03             Active
> >> draft-ietf-xrblock-rtcp-xr-decodability-07        Active
> >> draft-ietf-xrblock-rtcp-xr-discard-11             Active
> >> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active
> >> draft-ietf-xrblock-rtcp-xr-jb-07                  Active
> >> draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active
> >> draft-ietf-xrblock-rtcp-xr-qoe-03                 Active
> >> draft-ietf-xrblock-rtcp-xr-summary-stat-06        In IESG processing
> >> - ID Tracker state 
> >> draft-ietf-xrblock-rtcp-xr-synchronization-02     Active
> >> _______________________________________________
> >> pm-dir mailing list
> >> pm-dir@ietf.org
> >> https://www.ietf.org/mailman/listinfo/pm-dir
> >>
> >>
> >
> > _______________________________________________
> > pm-dir mailing list
> > pm-dir@ietf.org
> > https://www.ietf.org/mailman/listinfo/pm-dir
>=20
> _______________________________________________
> pm-dir mailing list
> pm-dir@ietf.org
> https://www.ietf.org/mailman/listinfo/pm-dir

From bclaise@cisco.com  Sun Feb 17 14:06:22 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E956C21E803A for ; Sun, 17 Feb 2013 14:06:22 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.538
X-Spam-Level: 
X-Spam-Status: No, score=-10.538 tagged_above=-999 required=5 tests=[AWL=0.061, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BjTRh7CN4-JI for ; Sun, 17 Feb 2013 14:06:22 -0800 (PST)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 260F521E8039 for ; Sun, 17 Feb 2013 14:06:21 -0800 (PST)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1HM6LrG016800 for ; Sun, 17 Feb 2013 23:06:21 +0100 (CET)
Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1HM64aI012835 for ; Sun, 17 Feb 2013 23:06:14 +0100 (CET)
Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r1HM60CM022333 for pm-dir@ietf.org; Sun, 17 Feb 2013 23:06:00 +0100 (CET)
Date: Sun, 17 Feb 2013 23:06:00 +0100
From: Benoit Claise 
To: pm-dir@ietf.org
Message-ID: <20130217220600.GA22331@sweet-brew-5.cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.3i
Subject: [pm-dir] Performance metrics doctors generated email
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sun, 17 Feb 2013 22:06:23 -0000

Dear all,

This is an automatically generated email.  
It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference.
It also lists all the IETF internet-drafts that contain "performance metric".

Regards, Benoit

===========================================================

Normative References
--------------------
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active	
    
Informative References
----------------------
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-concsec-03             Active	
draft-ietf-xrblock-rtcp-xr-decodability-07        In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-discard-11             Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active	
draft-ietf-xrblock-rtcp-xr-jb-07                  Active	
draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active	
draft-ietf-xrblock-rtcp-xr-qoe-03                 Active	

drafts containing performance metric
------------------------------------
draft-ietf-alto-deployments-05                    Active	
draft-ietf-alto-protocol-13                       In IESG processing - ID Tracker state 	
draft-ietf-bmwg-ca-bench-meth-04                  Active	
draft-ietf-ippm-rate-problem-02                   Active	
draft-ietf-ippm-testplan-rfc2680-01               Active	
draft-ietf-karp-threats-reqs-07                   In IESG processing - ID Tracker state 	
draft-ietf-manet-olsrv2-mib-05                    Active	
draft-ietf-manet-smf-mib-06                       Active	
draft-ietf-mmusic-media-loopback-27               In IESG processing - ID Tracker state 	
draft-ietf-nvo3-framework-02                      Active	
draft-ietf-opsawg-oam-overview-08                 In IESG processing - ID Tracker state 	
draft-ietf-ppsp-peer-protocol-06                  Active	
draft-ietf-rtcweb-rtp-usage-05                    Active	
draft-ietf-rtgwg-cl-framework-02                  Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-concsec-03             Active	
draft-ietf-xrblock-rtcp-xr-decodability-07        In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-discard-11             Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active	
draft-ietf-xrblock-rtcp-xr-jb-07                  Active	
draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active	
draft-ietf-xrblock-rtcp-xr-qoe-03                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-08        In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-synchronization-02     Active	

From dromasca@avaya.com  Mon Feb 18 06:10:29 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DC30421F8971; Mon, 18 Feb 2013 06:10:29 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.415
X-Spam-Level: 
X-Spam-Status: No, score=-103.415 tagged_above=-999 required=5 tests=[AWL=0.184, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1PMKaWAVKoDv; Mon, 18 Feb 2013 06:10:28 -0800 (PST)
Received: from de307622-de-outbound.net.avaya.com (de307622-de-outbound.net.avaya.com [198.152.71.100]) by ietfa.amsl.com (Postfix) with ESMTP id 3033D21F8958; Mon, 18 Feb 2013 06:10:17 -0800 (PST)
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AgEFAP0QA1HGmAcF/2dsb2JhbABFgmu7ZxZzgh4BAQEBAwEBAQkGXBcCBAEIDQQEAQELHSIMCxQHAQEFBQQBEggah20BC6FYnHAEBJBaYQOILJNuijuCd4Ik
X-IronPort-AV: E=Sophos;i="4.84,541,1355115600"; d="scan'208";a="344094165"
Received: from unknown (HELO co300216-co-erhwest.avaya.com) ([198.152.7.5]) by de307622-de-outbound.net.avaya.com with ESMTP; 18 Feb 2013 09:12:33 -0500
Received: from unknown (HELO AZ-FFEXHC04.global.avaya.com) ([135.64.58.14]) by co300216-co-erhwest-out.avaya.com with ESMTP; 18 Feb 2013 09:08:26 -0500
Received: from AZ-FFEXMB04.global.avaya.com ([fe80::6db7:b0af:8480:c126]) by AZ-FFEXHC04.global.avaya.com ([135.64.58.14]) with mapi id 14.02.0328.009; Mon, 18 Feb 2013 09:10:38 -0500
From: "Romascanu, Dan (Dan)" 
To: "xrblock@ietf.org" , "pm-dir@ietf.org" 
Thread-Topic: [rtcweb] RTP Usage: RTCP XR metrics
Thread-Index: AQHODeFroiEq9I65BEqhdI/HyMgtSJh/p0dQ
Date: Mon, 18 Feb 2013 14:10:38 +0000
Message-ID: <9904FB1B0159DA42B0B887B7FA8119CA08CFD5@AZ-FFEXMB04.global.avaya.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [135.64.58.45]
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: [pm-dir] FW: [rtcweb] RTP Usage: RTCP XR metrics
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 18 Feb 2013 14:10:30 -0000

The XRBLOCK WG and the PM-Directorate should be aware about this discussion=
 taking place in the RTCWEB WG.=20

Regards,

Dan




-----Original Message-----
From: rtcweb-bounces@ietf.org [mailto:rtcweb-bounces@ietf.org] On Behalf Of=
 Magnus Westerlund
Sent: Monday, February 18, 2013 4:08 PM
To: rtcweb@ietf.org
Cc: Colin Perkins
Subject: [rtcweb] RTP Usage: RTCP XR metrics

WG,

In Section 8 of
https://datatracker.ietf.org/doc/draft-ietf-rtcweb-rtp-usage/ there is an o=
pen issue regarding RTCP XR.

The question is if any RTCP XR metrics should be mandated or recommended to=
 be implemented in WebRTC end-points?

So far there has been some lukewarm interest in metrics, but no clear propo=
sals on what to include.

As an author I like to either get some discussion going or be able to remov=
e this open issue. So people, either make a proposal for what to include or=
 we will remove the TBD and consider the question dealt with.
Please note that there do exist signalling support for negotiating RTCP XR =
metrics to use in a particular session.

Submit any proposal no later than the 17th of March.

cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
F=E4r=F6gatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: magnus.westerlund@ericsson.com
----------------------------------------------------------------------

_______________________________________________
rtcweb mailing list
rtcweb@ietf.org
https://www.ietf.org/mailman/listinfo/rtcweb

From bclaise@cisco.com  Mon Feb 18 09:30:42 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AF73021F8C57; Mon, 18 Feb 2013 09:30:42 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.648
X-Spam-Level: 
X-Spam-Status: No, score=-9.648 tagged_above=-999 required=5 tests=[AWL=-0.834, BAYES_00=-2.599, HTML_FONT_FACE_BAD=0.884, HTML_MESSAGE=0.001, J_CHICKENPOX_15=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T1DMLd+wX8jY; Mon, 18 Feb 2013 09:30:40 -0800 (PST)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 866F421F8C3D; Mon, 18 Feb 2013 09:30:40 -0800 (PST)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1IHUcD1013261; Mon, 18 Feb 2013 18:30:38 +0100 (CET)
Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1IHU7kx012605; Mon, 18 Feb 2013 18:30:17 +0100 (CET)
Message-ID: <5122651F.6050405@cisco.com>
Date: Mon, 18 Feb 2013 18:30:07 +0100
From: Benoit Claise 
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2
MIME-Version: 1.0
To: "pm-dir@ietf.org" 
References: <5F7BCCF5541B7444830A2288ABBEBC9625FBD154A3@FRMRSSXCHMBSD2.dc-m.alcatel-lucent.com>
In-Reply-To: <5F7BCCF5541B7444830A2288ABBEBC9625FBD154A3@FRMRSSXCHMBSD2.dc-m.alcatel-lucent.com>
X-Forwarded-Message-Id: <5F7BCCF5541B7444830A2288ABBEBC9625FBD154A3@FRMRSSXCHMBSD2.dc-m.alcatel-lucent.com>
Content-Type: multipart/alternative; boundary="------------040307050005010706040509"
Cc: IESG IESG , Eliot Lear 
Subject: [pm-dir] =?iso-8859-1?q?ITU-T_using_the_RFC_6390_Template=3A_=5BT?= =?iso-8859-1?q?13Q17=5D_=5Bnew_Q=2E7/13=5D=3A_Y=2Edpifr=2C_=A7_8_Performa?= =?iso-8859-1?q?nce_Indicators?=
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 18 Feb 2013 17:30:42 -0000

This is a multi-part message in MIME format.
--------------040307050005010706040509
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

FYI, the ITU-T (actually SG13 Q 17) planning on using the RFC 6390 template.
See in there.

    We've agreed that any performance indicator definition shall follow
    a *template* style:
    => IETF *RFC 6390*, § 5.4.4
    (_http://tools.ietf.org/html/rfc6390#page-14_); or e.g.

Regards, Benoit

-------- Original Message --------
Subject: 	[T13Q17] [new Q.7/13]: Y.dpifr, § 8 Performance Indicators
Date: 	Mon, 18 Feb 2013 11:52:18 +0100
From: 	Schwarz, Albrecht (Albrecht) 
To: 	t09sg13q17@lists.itu.int 



This discussion is associated to C-6, 7 & 37 and the attribution of a 
performance indicator (PI) as KPI or non-KPI:
*8.**2.1 **Overview**--Performance **i**ndicators for DPI Nodes *
/{Editor's note (2012-06 meeting): there are two open questions:/
/Q1: what are the criteria for classifying a performance indicator as 
KPI or non-KPI?/
/Q2: is there any good definition for KPI which may be reused in Y.dpifr?/
/Contributions are solicited.}/
We've agreed that any performance indicator definition shall follow a 
*template* style:
=> IETF *RFC 6390*, § 5.4.4 
(_http://tools.ietf.org/html/rfc6390#page-14_); or e.g.
=> *3GPP 32.410*, § 5 (_http://www.3gpp.org/ftp/Specs/html-info/32410.htm_)
To Editor's note Q2: the 3GPP 32.410 KPI usage might be used as baseline 
for a KPI definition in Y.dpifr.
Something like:
KPIs: are primary metrics to evaluate process performance as indicators 
of the core function of the network element. A KPI in scope of this 
Recommendation would therefore characterize the performance of the /DPI 
packet processing path/ (also known as DPI engine).
NOTE -- The notion of /performance indicator/ (PI) is synonym to 
/performance metric/ in this Recommendation.
To Editor's note Q1: above definition proposal provides an inherent, 
rough criteria for classification.
We got arround 25 PI proposals so far in Y.dpifr, which may be 
categorized as:
PI:
|-KPI:
|  |- e.g. § 8.2.1.1
|-non-KPI: e.g.
   |- Protocol-specific PIs (such as TCP relates PIs)
   |- Energy-related PIs (because energy consumption is implementation 
specific!)
   |- Non-DPI engine related PIs (such as associated to DPI-PIB)
   |- ...
It's clear that every PI might be a "KPI" in a particular context and 
instantiation of a DPI node (-> § 6.1/Y.dpifr), - and that the 
differentiation between KPI and non-KPI is always debatable ("and should 
be actually a minor aspect in Y.dpifr"). But when we want to use the 
term KPI in Y.dpifr, then we need to describe the difference to non-KPI ...
Comments?
Albrecht



--------------040307050005010706040509
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit


  

    
  
  
    FYI, the ITU-T (actually SG13 Q 17) planning on using the RFC 6390
    template.

    See in there.

    

        
We’ve agreed that any performance
          indicator definition shall follow a template style:

        
=> IETF RFC 6390, §
          5.4.4 (http://tools.ietf.org/html/rfc6390#page-14);
          or e.g.

      

    
Regards, Benoit

      

      -------- Original Message --------
      

        
          

            Subject:
            
            [T13Q17] [new Q.7/13]: Y.dpifr, § 8 Performance
              Indicators
          

          

            Date: 
            Mon, 18 Feb 2013 11:52:18 +0100
          

          

            From: 
            Schwarz, Albrecht (Albrecht)
              <albrecht.schwarz@alcatel-lucent.com>
          

          

            To: 
            t09sg13q17@lists.itu.int
              <t09sg13q17@lists.itu.int>
          

        
      

      

      

      
      
      
      
      
        
This discussion is associated to
          C-6, 7 & 37 and the attribution of a performance indicator
          (PI) as KPI or non-KPI:

        
        
8.2.1  
            Overview –Performance indicators
              for DPI Nodes 

        
{Editor’s note
                (2012-06 meeting): there are two open questions:

        
Q1: what are the
                criteria for classifying a performance indicator as KPI
                or non-KPI?

        
Q2: is there any good
                definition for KPI which may be reused in Y.dpifr?

        
Contributions
                are solicited.}

        
 

        
We’ve agreed that any performance
          indicator definition shall follow a template style:

        
=> IETF RFC 6390, §
          5.4.4 (http://tools.ietf.org/html/rfc6390#page-14);
          or e.g.

        
        
 

        
To Editor’s note Q2: the 3GPP
          32.410 KPI usage might be used as baseline for a KPI
          definition in Y.dpifr.

        
Something like:

        
KPIs: are
          primary metrics to evaluate process performance as indicators
          of the core function of the network element. A KPI in scope of
          this Recommendation would therefore characterize the
          performance of the
          DPI packet processing path (also known as DPI engine).

        
NOTE – The
          notion of performance indicator (PI) is synonym to performance
            metric in this Recommendation.

        
 

        
To Editor’s note Q1: above
          definition proposal provides an inherent, rough criteria for
          classification.

        
We got arround 25 PI proposals so
          far in Y.dpifr, which may be categorized as:

        
PI:

        
|-KPI: 

        
|  |- e.g. § 8.2.1.1

        
|-non-KPI: e.g.

        
   |- Protocol-specific PIs (such
          as TCP relates PIs)

        
   |- Energy-related PIs (because
          energy consumption is implementation specific!)

        
   |- Non-DPI engine related PIs
          (such as associated to DPI-PIB)

        
   |- …

        
 

        
It’s clear that every PI might be
          a “KPI” in a particular context and instantiation of a DPI
          node (-> § 6.1/Y.dpifr), - and that the differentiation
          between KPI and non-KPI is always debatable (“and should be
          actually a minor
          aspect in Y.dpifr”). But when we want to use the term KPI in
          Y.dpifr, then we need to describe the difference to non-KPI …

        
Comments?

        
Albrecht

        
 

        
 

        
 

        
 

      
      

    

    

  


--------------040307050005010706040509--

From ron.even.tlv@gmail.com  Mon Feb 18 09:27:28 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3124121F8607; Mon, 18 Feb 2013 09:27:28 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.099
X-Spam-Level: 
X-Spam-Status: No, score=-3.099 tagged_above=-999 required=5 tests=[AWL=0.500,  BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hp71KtBjKAzX; Mon, 18 Feb 2013 09:27:27 -0800 (PST)
Received: from mail-ea0-f178.google.com (mail-ea0-f178.google.com [209.85.215.178]) by ietfa.amsl.com (Postfix) with ESMTP id B3B1B21F8C2F; Mon, 18 Feb 2013 09:27:26 -0800 (PST)
Received: by mail-ea0-f178.google.com with SMTP id a14so2504726eaa.9 for ; Mon, 18 Feb 2013 09:27:25 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:to:references:in-reply-to:subject:date:message-id :mime-version:content-type:content-transfer-encoding:x-mailer :thread-index:content-language; bh=Joq3aKCS6hTP+ShBOpQCZN7NEd09Rl8brElDZlV76+4=; b=AMg8lKA8S3bi/ADRJ2x8CN4p0rptB2TDapXJoDcWNCUojknJsbGzZtgLsKnSVM9c/C jRQ2CtOdgPvKRf1wGBfCKiT6HGnESjyzc2FVi011Eb4W6cj+6r4wEP1uWeltlr9w78e3 hk0mRc2u088M9+Q648/WgqhKEQxaLh4QR55EhzVyqzzJyClYPmWTF1XTVGzPqGxr46xC XN7tpOuCx3l6IML1HQvIRvk2T+i+AYRmP2wulU654fIIFEUJ8PWwOgz4Mqx+nwu0BTUQ 6Sf2+6uJeF1g2UQQLRTtW3UzSk7UmKbEkzO2btszFpBLA9C+kQsp91B2hwWLlY7XN5D0 HISg==
X-Received: by 10.14.3.133 with SMTP id 5mr46314239eeh.43.1361208445760; Mon, 18 Feb 2013 09:27:25 -0800 (PST)
Received: from RoniE (bzq-79-181-179-229.red.bezeqint.net. [79.181.179.229]) by mx.google.com with ESMTPS id r4sm55211591eeo.12.2013.02.18.09.27.23 (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 18 Feb 2013 09:27:24 -0800 (PST)
From: "Roni Even" 
To: "'Romascanu, Dan \(Dan\)'" , , 
References: <9904FB1B0159DA42B0B887B7FA8119CA08CFD5@AZ-FFEXMB04.global.avaya.com>
In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA08CFD5@AZ-FFEXMB04.global.avaya.com>
Date: Mon, 18 Feb 2013 19:24:15 +0200
Message-ID: <007801ce0dfc$c82aa240$587fe6c0$@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable
X-Mailer: Microsoft Outlook 14.0
Thread-Index: AQIN0/U3T43aJp1LUi5Fof91gdwX9pgAUVOw
Content-Language: en-us
X-Mailman-Approved-At: Mon, 18 Feb 2013 10:22:04 -0800
Subject: Re: [pm-dir] [xrblock] FW: [rtcweb] RTP Usage: RTCP XR metrics
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 18 Feb 2013 17:27:28 -0000

Hi,
I responded on the rtcweb list.
http://tools.ietf.org/html/draft-huang-rtcweb-monitoring-00 was =
submitted a
year ago and at the time did not progress since this was low priprity
Roni Even

-----Original Message-----
From: xrblock-bounces@ietf.org [mailto:xrblock-bounces@ietf.org] On =
Behalf
Of Romascanu, Dan (Dan)
Sent: 18 February, 2013 4:11 PM
To: xrblock@ietf.org; pm-dir@ietf.org
Subject: [xrblock] FW: [rtcweb] RTP Usage: RTCP XR metrics

The XRBLOCK WG and the PM-Directorate should be aware about this =
discussion
taking place in the RTCWEB WG.=20

Regards,

Dan




-----Original Message-----
From: rtcweb-bounces@ietf.org [mailto:rtcweb-bounces@ietf.org] On Behalf =
Of
Magnus Westerlund
Sent: Monday, February 18, 2013 4:08 PM
To: rtcweb@ietf.org
Cc: Colin Perkins
Subject: [rtcweb] RTP Usage: RTCP XR metrics

WG,

In Section 8 of
https://datatracker.ietf.org/doc/draft-ietf-rtcweb-rtp-usage/ there is =
an
open issue regarding RTCP XR.

The question is if any RTCP XR metrics should be mandated or recommended =
to
be implemented in WebRTC end-points?

So far there has been some lukewarm interest in metrics, but no clear
proposals on what to include.

As an author I like to either get some discussion going or be able to =
remove
this open issue. So people, either make a proposal for what to include =
or we
will remove the TBD and consider the question dealt with.
Please note that there do exist signalling support for negotiating RTCP =
XR
metrics to use in a particular session.

Submit any proposal no later than the 17th of March.

cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
F=E4r=F6gatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: magnus.westerlund@ericsson.com
----------------------------------------------------------------------

_______________________________________________
rtcweb mailing list
rtcweb@ietf.org
https://www.ietf.org/mailman/listinfo/rtcweb
_______________________________________________
xrblock mailing list
xrblock@ietf.org
https://www.ietf.org/mailman/listinfo/xrblock


From rlb@ipv.sx  Mon Feb 18 09:35:10 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DC71B21F8C84 for ; Mon, 18 Feb 2013 09:35:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.604
X-Spam-Level: 
X-Spam-Status: No, score=-1.604 tagged_above=-999 required=5 tests=[AWL=-0.412, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_FONT_FACE_BAD=0.884, HTML_MESSAGE=0.001, J_CHICKENPOX_15=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id akFarrgzPQRj for ; Mon, 18 Feb 2013 09:35:09 -0800 (PST)
Received: from mail-oa0-f42.google.com (mail-oa0-f42.google.com [209.85.219.42]) by ietfa.amsl.com (Postfix) with ESMTP id 955EA21F8C83 for ; Mon, 18 Feb 2013 09:35:09 -0800 (PST)
Received: by mail-oa0-f42.google.com with SMTP id i18so6164152oag.29 for ; Mon, 18 Feb 2013 09:35:09 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:x-originating-ip:in-reply-to:references :date:message-id:subject:from:to:cc:content-type:x-gm-message-state; bh=RqSj6YXRYVcUR0Ty4hTVP6x5vfAeOz3HcVd+fwRopOI=; b=QhTNjjeNihK+CSgg/B9K4Lh0f5tI2F/ZN5JDu81qyOUfBTP87FwQScWNg7ryqCSjiV HWoM24aVWWLdX5Q76mT1WiERzA0LeJYTVjt0OesBH4oKNSnyeE3WBOJls3a2pyK2v6C1 HzeklH3KnvPBIDLf181uUt2XTlxfgOl+aJs8pPgfyYJkZwhqbN7RIDXsDuAltUoVAL7s 3eIW0281omQ5wLxj868lNokMxzD4SnyVgPhJUUjD9NfzYo3H1OY5G2HGchxlF+k5g7Sh 6mN06BdUTNzK3Kc+6raPKSX0UIm4Zpg4rxXRL0hFBnvWK/evaF0W4UPAp0ectt7UDau3 033g==
MIME-Version: 1.0
X-Received: by 10.60.21.38 with SMTP id s6mr6417096oee.3.1361208909036; Mon, 18 Feb 2013 09:35:09 -0800 (PST)
Received: by 10.60.7.132 with HTTP; Mon, 18 Feb 2013 09:35:08 -0800 (PST)
X-Originating-IP: [2606:4100:3880:2520:f035:6a9f:8a04:1595]
In-Reply-To: <5122651F.6050405@cisco.com>
References: <5F7BCCF5541B7444830A2288ABBEBC9625FBD154A3@FRMRSSXCHMBSD2.dc-m.alcatel-lucent.com> <5122651F.6050405@cisco.com>
Date: Mon, 18 Feb 2013 12:35:08 -0500
Message-ID: 
From: Richard Barnes 
To: Benoit Claise 
Content-Type: multipart/alternative; boundary=e89a8fb2068a7cfdcf04d60325ce
X-Gm-Message-State: ALoCoQnrinp8bAvCifQUNSeV09GoOOHGhJFQdq/8F+2eze/SsNSu+as1FM/HO1CN+iXi5vnW0M19
X-Mailman-Approved-At: Mon, 18 Feb 2013 10:22:04 -0800
Cc: "pm-dir@ietf.org" , IESG IESG , Eliot Lear 
Subject: Re: [pm-dir] =?iso-8859-1?q?ITU-T_using_the_RFC_6390_Template=3A_=5BT?= =?iso-8859-1?q?13Q17=5D_=5Bnew_Q=2E7/13=5D=3A_Y=2Edpifr=2C_=A7_8_P?= =?iso-8859-1?q?erformance_Indicators?=
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Mon, 18 Feb 2013 17:35:11 -0000

--e89a8fb2068a7cfdcf04d60325ce
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Just for context (due to partial acronym collision), this would be for
setting metrics for Deep Packet Inspection nodes?  That is, the nodes
discussed in Y.2770?


Thanks,
--Richard


On Mon, Feb 18, 2013 at 12:30 PM, Benoit Claise  wrote:

>  FYI, the ITU-T (actually SG13 Q 17) planning on using the RFC 6390
> template.
> See in there.
>
> We=92ve agreed that any performance indicator definition shall follow a *
> template* style:
> =3D> IETF *RFC 6390*, =A7 5.4.4 (*http://tools.ietf.org/html/rfc6390#page=
-14*);
> or e.g.
>
> Regards, Benoit
>
> -------- Original Message --------  Subject: [T13Q17] [new Q.7/13]:
> Y.dpifr, =A7 8 Performance Indicators  Date: Mon, 18 Feb 2013 11:52:18 +0=
100  From:
> Schwarz, Albrecht (Albrecht)   To:
> t09sg13q17@lists.itu.int 
>
>  This discussion is associated to C-6, 7 & 37 and the attribution of a
> performance indicator (PI) as KPI or non-KPI:
>  *8.**2.1   **Overview** =96Performance **i**ndicators for DPI Nodes *
> *{Editor=92s note (2012-06 meeting): there are two open questions:*
> *Q1: what are the criteria for classifying a performance indicator as KPI
> or non-KPI?*
> *Q2: is there any good definition for KPI which may be reused in Y.dpifr?=
*
> *Contributions are solicited.}*
>
> We=92ve agreed that any performance indicator definition shall follow a *
> template* style:
> =3D> IETF *RFC 6390*, =A7 5.4.4 (*http://tools.ietf.org/html/rfc6390#page=
-14*);
> or e.g.
> =3D> *3GPP 32.410*, =A7 5 (*http://www.3gpp.org/ftp/Specs/html-info/32410=
.htm*
> )
>
> To Editor=92s note Q2: the 3GPP 32.410 KPI usage might be used as baselin=
e
> for a KPI definition in Y.dpifr.
> Something like:
> KPIs: are primary metrics to evaluate process performance as indicators o=
f
> the core function of the network element. A KPI in scope of this
> Recommendation would therefore characterize the performance of the *DPI
> packet processing path* (also known as DPI engine).
> NOTE =96 The notion of *performance indicator* (PI) is synonym to *perfor=
mance
> metric* in this Recommendation.
>
> To Editor=92s note Q1: above definition proposal provides an inherent, ro=
ugh
> criteria for classification.
> We got arround 25 PI proposals so far in Y.dpifr, which may be categorize=
d
> as:
> PI:
> |-KPI:
> |  |- e.g. =A7 8.2.1.1
> |-non-KPI: e.g.
>    |- Protocol-specific PIs (such as TCP relates PIs)
>    |- Energy-related PIs (because energy consumption is implementation
> specific!)
>    |- Non-DPI engine related PIs (such as associated to DPI-PIB)
>    |- =85
>
> It=92s clear that every PI might be a =93KPI=94 in a particular context a=
nd
> instantiation of a DPI node (-> =A7 6.1/Y.dpifr), - and that the
> differentiation between KPI and non-KPI is always debatable (=93and shoul=
d be
> actually a minor aspect in Y.dpifr=94). But when we want to use the term =
KPI
> in Y.dpifr, then we need to describe the difference to non-KPI =85
> Comments?
> Albrecht
>
>
>
>
>
>
>

--e89a8fb2068a7cfdcf04d60325ce
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable


Just for context (due to partial acronym collision), this =
would be for setting metrics for Deep Packet Inspection nodes? =A0That is, =
the nodes discussed in Y.2770?


Thanks,
--Richard


On Mon, Feb 18, 2=
013 at 12:30 PM, Benoit Claise <bclaise@cisco.com> wrote:


 =20

   =20
 =20
  

    FYI, the ITU-T (actually SG13 Q 17) planning on using the RFC 6390
    template.

    See in there.

    

        
We=92ve agreed that any performance
          indicator definition shall follow a template style:

        
=3D> IETF RFC 6390, =A7
          5.4.4 (http://tools.ietf.org/html/rfc639=
0#page-14);
          or e.g.

      

    
Regards, Benoit

      

      -------- Original Message --------
      

        
          

            Subject:
            
            [T13Q17] [new Q.7/13]: Y.dpifr, =A7 8 Performance
              Indicators
          

          

            Date: 
            Mon, 18 Feb 2013 11:52:18 +0100
          

          

            From: 
            Schwarz, Albrecht (Albrecht)
              <albrecht.schwarz@alcatel-lucent.com>
          

          

            To: 
            t09sg13q17@lists.itu.int
              <t09sg13q17@lists.itu.int>
          

        
      

      

      

     =20
     =20
     =20
     =20
      
        
This discussion is associated to
          C-6, 7 & 37 and the attribution of a performance indicator
          (PI) as KPI or non-KPI:

        
        
8.2.1=A0=A0
            Overview =96Performance indicators
              for DPI Nodes 

        
=
{Editor=92s note
                (2012-06 meeting): there are two open questions:=


        
=
Q1: what are the
                criteria for classifying a performance indicator as KPI
                or non-KPI?

        
=
Q2: is there any good
                definition for KPI which may be reused in Y.dpifr?

        
C=
ontributions
                are solicited.}

        
=A0

        
We=92ve agreed that any performance
          indicator definition shall follow a template style:

        
=3D> IETF RFC 6390, =A7
          5.4.4 (http://tools.ietf.org/html/rfc639=
0#page-14);
          or e.g.

        
=3D> 3GPP 32.410, =A7 5 (http://www.3gpp.org/ftp/Specs/html-info/32410=
.htm)


        
=A0

        
To Editor=92s note Q2: the 3GPP
          32.410 KPI usage might be used as baseline for a KPI
          definition in Y.dpifr.

        
Something like:

        
KPIs: are
          primary metrics to evaluate process performance as indicators
          of the core function of the network element. A KPI in scope of
          this Recommendation would therefore characterize the
          performance of the
          DPI packet processing path (also known as DPI engine).
        
NOTE =96 The
          notion of performance indicator (PI) is synonym to perf=
ormance
            metric in this Recommendation.

        
=A0

        
To Editor=92s note Q1: above
          definition proposal provides an inherent, rough criteria for
          classification.

        
We got arround 25 PI proposals so
          far in Y.dpifr, which may be categorized as:

        
PI:

        
|-KPI: 

        
|=A0 |- e.g. =A7 8.2.1.1

        
|-non-KPI: e.g.

        
 =A0 |- Protocol-specific PIs (such
          as TCP relates PIs)

        
 =A0 |- Energy-related PIs (because
          energy consumption is implementation specific!)

        
 =A0 |- Non-DPI engine related PIs
          (such as associated to DPI-PIB)

        
 =A0 |- =85

        
=A0

        
It=92s clear that every PI might be
          a =93KPI=94 in a particular context and instantiation of a DPI
          node (-> =A7 6.1/Y.dpifr), - and that the differentiation
          between KPI and non-KPI is always debatable (=93and should be
          actually a minor
          aspect in Y.dpifr=94). But when we want to use the term KPI in
          Y.dpifr, then we need to describe the difference to non-KPI =85
        
Comments?

        
Albrecht

        
=A0

        
=A0

        
=A0

        
=A0

--e89a8fb2068a7cfdcf04d60325ce-- From bclaise@cisco.com Tue Feb 19 03:49:51 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 41CAB21F8B69; Tue, 19 Feb 2013 03:49:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.642 X-Spam-Level: X-Spam-Status: No, score=-9.642 tagged_above=-999 required=5 tests=[AWL=-0.828, BAYES_00=-2.599, HTML_FONT_FACE_BAD=0.884, HTML_MESSAGE=0.001, J_CHICKENPOX_15=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Iaf2puIAK00t; Tue, 19 Feb 2013 03:49:50 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 6B7CF21F8830; Tue, 19 Feb 2013 03:49:44 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JBXAoD025373; Tue, 19 Feb 2013 12:33:11 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JBWYKS000098; Tue, 19 Feb 2013 12:32:45 +0100 (CET) Message-ID: <512362D2.70108@cisco.com> Date: Tue, 19 Feb 2013 12:32:34 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Richard Barnes References: <5F7BCCF5541B7444830A2288ABBEBC9625FBD154A3@FRMRSSXCHMBSD2.dc-m.alcatel-lucent.com> <5122651F.6050405@cisco.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------080801030104000803040703" Cc: "pm-dir@ietf.org" , IESG IESG , Eliot Lear Subject: Re: [pm-dir] =?windows-1252?q?ITU-T_using_the_RFC_6390_Template=3A_?= =?windows-1252?q?=5BT13Q17=5D_=5Bnew_Q=2E7/13=5D=3A_Y=2Edpifr=2C_=A7_8_Pe?= =?windows-1252?q?rformance_Indicators?= X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Feb 2013 11:49:51 -0000 This is a multi-part message in MIME format. --------------080801030104000803040703 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Richard, > Just for context (due to partial acronym collision), this would be for > setting metrics for Deep Packet Inspection nodes? That is, the nodes > discussed in Y.2770? > Either this one (Y.DPIreq) or the framework Y.DPIfr Note: I was involved in this in the past (not any longer) and I pushed for those performance metrics to be in an appendix. Regards, Benoit > > Thanks, > --Richard > > > On Mon, Feb 18, 2013 at 12:30 PM, Benoit Claise > wrote: > > FYI, the ITU-T (actually SG13 Q 17) planning on using the RFC 6390 > template. > See in there. > > We’ve agreed that any performance indicator definition shall > follow a *template* style: > => IETF *RFC 6390*, § 5.4.4 > (_http://tools.ietf.org/html/rfc6390#page-14_); or e.g. > > Regards, Benoit > > -------- Original Message -------- > Subject: [T13Q17] [new Q.7/13]: Y.dpifr, § 8 Performance Indicators > Date: Mon, 18 Feb 2013 11:52:18 +0100 > From: Schwarz, Albrecht (Albrecht) > > > To: t09sg13q17@lists.itu.int > > > > > This discussion is associated to C-6, 7 & 37 and the attribution > of a performance indicator (PI) as KPI or non-KPI: > *8.**2.1 **Overview**–Performance **i**ndicators for DPI Nodes * > /{Editor’s note (2012-06 meeting): there are two open questions:/ > /Q1: what are the criteria for classifying a performance indicator > as KPI or non-KPI?/ > /Q2: is there any good definition for KPI which may be reused in > Y.dpifr?/ > /Contributions are solicited.}/ > We’ve agreed that any performance indicator definition shall > follow a *template* style: > => IETF *RFC 6390*, § 5.4.4 > (_http://tools.ietf.org/html/rfc6390#page-14_); or e.g. > => *3GPP 32.410*, § 5 > (_http://www.3gpp.org/ftp/Specs/html-info/32410.htm_) > To Editor’s note Q2: the 3GPP 32.410 KPI usage might be used as > baseline for a KPI definition in Y.dpifr. > Something like: > KPIs: are primary metrics to evaluate process performance as > indicators of the core function of the network element. A KPI in > scope of this Recommendation would therefore characterize the > performance of the /DPI packet processing path/ (also known as DPI > engine). > NOTE – The notion of /performance indicator/ (PI) is synonym to > /performance metric/ in this Recommendation. > To Editor’s note Q1: above definition proposal provides an > inherent, rough criteria for classification. > We got arround 25 PI proposals so far in Y.dpifr, which may be > categorized as: > PI: > |-KPI: > | |- e.g. § 8.2.1.1 > |-non-KPI: e.g. > |- Protocol-specific PIs (such as TCP relates PIs) > |- Energy-related PIs (because energy consumption is > implementation specific!) > |- Non-DPI engine related PIs (such as associated to DPI-PIB) > |- … > It’s clear that every PI might be a “KPI” in a particular context > and instantiation of a DPI node (-> § 6.1/Y.dpifr), - and that the > differentiation between KPI and non-KPI is always debatable (“and > should be actually a minor aspect in Y.dpifr”). But when we want > to use the term KPI in Y.dpifr, then we need to describe the > difference to non-KPI … > Comments? > Albrecht > > > --------------080801030104000803040703 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit
Richard,

Just for context (due to partial acronym collision), this would be for setting metrics for Deep Packet Inspection nodes?  That is, the nodes discussed in Y.2770?
Either this one (Y.DPIreq) or the framework Y.DPIfr
Note: I was involved in this in the past (not any longer) and I pushed for those performance metrics to be in an appendix.

Regards, Benoit

Thanks,
--Richard


On Mon, Feb 18, 2013 at 12:30 PM, Benoit Claise <bclaise@cisco.com> wrote:
FYI, the ITU-T (actually SG13 Q 17) planning on using the RFC 6390 template.
See in there.
We’ve agreed that any performance indicator definition shall follow a template style:
=> IETF RFC 6390, § 5.4.4 (http://tools.ietf.org/html/rfc6390#page-14); or e.g.
Regards, Benoit

-------- Original Message --------
Subject: [T13Q17] [new Q.7/13]: Y.dpifr, § 8 Performance Indicators
Date: Mon, 18 Feb 2013 11:52:18 +0100
From: Schwarz, Albrecht (Albrecht) <albrecht.schwarz@alcatel-lucent.com>
To: t09sg13q17@lists.itu.int <t09sg13q17@lists.itu.int>


This discussion is associated to C-6, 7 & 37 and the attribution of a performance indicator (PI) as KPI or non-KPI:
8.2.1   Overview –Performance indicators for DPI Nodes
{Editor’s note (2012-06 meeting): there are two open questions:
Q1: what are the criteria for classifying a performance indicator as KPI or non-KPI?
Q2: is there any good definition for KPI which may be reused in Y.dpifr?
Contributions are solicited.}
 
We’ve agreed that any performance indicator definition shall follow a template style:
=> IETF RFC 6390, § 5.4.4 (http://tools.ietf.org/html/rfc6390#page-14); or e.g.
 
To Editor’s note Q2: the 3GPP 32.410 KPI usage might be used as baseline for a KPI definition in Y.dpifr.
Something like:
KPIs: are primary metrics to evaluate process performance as indicators of the core function of the network element. A KPI in scope of this Recommendation would therefore characterize the performance of the DPI packet processing path (also known as DPI engine).
NOTE – The notion of performance indicator (PI) is synonym to performance metric in this Recommendation.
 
To Editor’s note Q1: above definition proposal provides an inherent, rough criteria for classification.
We got arround 25 PI proposals so far in Y.dpifr, which may be categorized as:
PI:
|-KPI:
|  |- e.g. § 8.2.1.1
|-non-KPI: e.g.
  |- Protocol-specific PIs (such as TCP relates PIs)
  |- Energy-related PIs (because energy consumption is implementation specific!)
  |- Non-DPI engine related PIs (such as associated to DPI-PIB)
  |- …
 
It’s clear that every PI might be a “KPI” in a particular context and instantiation of a DPI node (-> § 6.1/Y.dpifr), - and that the differentiation between KPI and non-KPI is always debatable (“and should be actually a minor aspect in Y.dpifr”). But when we want to use the term KPI in Y.dpifr, then we need to describe the difference to non-KPI …
Comments?
Albrecht
 
 
 
 



--------------080801030104000803040703-- From bill.wu@huawei.com Mon Feb 18 21:11:58 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2DD6921F8DCB; Mon, 18 Feb 2013 21:11:58 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -5.013 X-Spam-Level: X-Spam-Status: No, score=-5.013 tagged_above=-999 required=5 tests=[AWL=-0.167, BAYES_00=-2.599, MIME_BASE64_TEXT=1.753, RCVD_IN_DNSWL_MED=-4] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KicUT-5P7PPd; Mon, 18 Feb 2013 21:11:57 -0800 (PST) Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) by ietfa.amsl.com (Postfix) with ESMTP id CBD1B21F8D46; Mon, 18 Feb 2013 21:11:56 -0800 (PST) Received: from 172.18.7.190 (EHLO lhreml204-edg.china.huawei.com) ([172.18.7.190]) by lhrrg01-dlp.huawei.com (MOS 4.3.5-GA FastPath queued) with ESMTP id APV87011; Tue, 19 Feb 2013 05:11:10 +0000 (GMT) Received: from LHREML403-HUB.china.huawei.com (10.201.5.217) by lhreml204-edg.china.huawei.com (172.18.7.223) with Microsoft SMTP Server (TLS) id 14.1.323.7; Tue, 19 Feb 2013 05:10:38 +0000 Received: from SZXEML402-HUB.china.huawei.com (10.82.67.32) by lhreml403-hub.china.huawei.com (10.201.5.217) with Microsoft SMTP Server (TLS) id 14.1.323.7; Tue, 19 Feb 2013 05:11:08 +0000 Received: from w53375 (10.138.41.149) by szxeml402-hub.china.huawei.com (10.82.67.32) with Microsoft SMTP Server (TLS) id 14.1.323.7; Tue, 19 Feb 2013 13:11:01 +0800 Message-ID: <257EC0A6090546078E82EDF2DF0BC750@china.huawei.com> From: Qin Wu To: "Romascanu, Dan (Dan)" , , References: <9904FB1B0159DA42B0B887B7FA8119CA08CFD5@AZ-FFEXMB04.global.avaya.com> Date: Tue, 19 Feb 2013 13:11:00 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: base64 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109 X-Originating-IP: [10.138.41.149] X-CFilter-Loop: Reflected X-Mailman-Approved-At: Tue, 19 Feb 2013 04:56:35 -0800 Subject: Re: [pm-dir] [xrblock] FW: [rtcweb] RTP Usage: RTCP XR metrics X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Feb 2013 05:11:58 -0000 SSBoYXZlIHNlbnQgbXkgcHJvcG9zYWwgdG8gUlRDd2ViIGxpc3QuDQoNClJlZ2FyZHMhDQotUWlu DQotLS0tLSBPcmlnaW5hbCBNZXNzYWdlIC0tLS0tIA0KRnJvbTogIlJvbWFzY2FudSwgRGFuIChE YW4pIiA8ZHJvbWFzY2FAYXZheWEuY29tPg0KVG86IDx4cmJsb2NrQGlldGYub3JnPjsgPHBtLWRp ckBpZXRmLm9yZz4NClNlbnQ6IE1vbmRheSwgRmVicnVhcnkgMTgsIDIwMTMgMTA6MTAgUE0NClN1 YmplY3Q6IFt4cmJsb2NrXSBGVzogW3J0Y3dlYl0gUlRQIFVzYWdlOiBSVENQIFhSIG1ldHJpY3MN Cg0KDQpUaGUgWFJCTE9DSyBXRyBhbmQgdGhlIFBNLURpcmVjdG9yYXRlIHNob3VsZCBiZSBhd2Fy ZSBhYm91dCB0aGlzIGRpc2N1c3Npb24gdGFraW5nIHBsYWNlIGluIHRoZSBSVENXRUIgV0cuIA0K DQpSZWdhcmRzLA0KDQpEYW4NCg0KDQoNCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZy b206IHJ0Y3dlYi1ib3VuY2VzQGlldGYub3JnIFttYWlsdG86cnRjd2ViLWJvdW5jZXNAaWV0Zi5v cmddIE9uIEJlaGFsZiBPZiBNYWdudXMgV2VzdGVybHVuZA0KU2VudDogTW9uZGF5LCBGZWJydWFy eSAxOCwgMjAxMyA0OjA4IFBNDQpUbzogcnRjd2ViQGlldGYub3JnDQpDYzogQ29saW4gUGVya2lu cw0KU3ViamVjdDogW3J0Y3dlYl0gUlRQIFVzYWdlOiBSVENQIFhSIG1ldHJpY3MNCg0KV0csDQoN CkluIFNlY3Rpb24gOCBvZg0KaHR0cHM6Ly9kYXRhdHJhY2tlci5pZXRmLm9yZy9kb2MvZHJhZnQt aWV0Zi1ydGN3ZWItcnRwLXVzYWdlLyB0aGVyZSBpcyBhbiBvcGVuIGlzc3VlIHJlZ2FyZGluZyBS VENQIFhSLg0KDQpUaGUgcXVlc3Rpb24gaXMgaWYgYW55IFJUQ1AgWFIgbWV0cmljcyBzaG91bGQg YmUgbWFuZGF0ZWQgb3IgcmVjb21tZW5kZWQgdG8gYmUgaW1wbGVtZW50ZWQgaW4gV2ViUlRDIGVu ZC1wb2ludHM/DQoNClNvIGZhciB0aGVyZSBoYXMgYmVlbiBzb21lIGx1a2V3YXJtIGludGVyZXN0 IGluIG1ldHJpY3MsIGJ1dCBubyBjbGVhciBwcm9wb3NhbHMgb24gd2hhdCB0byBpbmNsdWRlLg0K DQpBcyBhbiBhdXRob3IgSSBsaWtlIHRvIGVpdGhlciBnZXQgc29tZSBkaXNjdXNzaW9uIGdvaW5n IG9yIGJlIGFibGUgdG8gcmVtb3ZlIHRoaXMgb3BlbiBpc3N1ZS4gU28gcGVvcGxlLCBlaXRoZXIg bWFrZSBhIHByb3Bvc2FsIGZvciB3aGF0IHRvIGluY2x1ZGUgb3Igd2Ugd2lsbCByZW1vdmUgdGhl IFRCRCBhbmQgY29uc2lkZXIgdGhlIHF1ZXN0aW9uIGRlYWx0IHdpdGguDQpQbGVhc2Ugbm90ZSB0 aGF0IHRoZXJlIGRvIGV4aXN0IHNpZ25hbGxpbmcgc3VwcG9ydCBmb3IgbmVnb3RpYXRpbmcgUlRD UCBYUiBtZXRyaWNzIHRvIHVzZSBpbiBhIHBhcnRpY3VsYXIgc2Vzc2lvbi4NCg0KU3VibWl0IGFu eSBwcm9wb3NhbCBubyBsYXRlciB0aGFuIHRoZSAxN3RoIG9mIE1hcmNoLg0KDQpjaGVlcnMNCg0K TWFnbnVzIFdlc3Rlcmx1bmQNCg0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KTXVsdGltZWRpYSBUZWNobm9sb2dp ZXMsIEVyaWNzc29uIFJlc2VhcmNoIEVBQi9UVk0NCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCkVyaWNzc29uIEFC ICAgICAgICAgICAgICAgIHwgUGhvbmUgICs0NiAxMCA3MTQ4Mjg3DQpG5HL2Z2F0YW4gNiAgICAg ICAgICAgICAgICB8IE1vYmlsZSArNDYgNzMgMDk0OTA3OQ0KU0UtMTY0IDgwIFN0b2NraG9sbSwg U3dlZGVufCBtYWlsdG86IG1hZ251cy53ZXN0ZXJsdW5kQGVyaWNzc29uLmNvbQ0KLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLQ0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0K cnRjd2ViIG1haWxpbmcgbGlzdA0KcnRjd2ViQGlldGYub3JnDQpodHRwczovL3d3dy5pZXRmLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL3J0Y3dlYg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18NCnhyYmxvY2sgbWFpbGluZyBsaXN0DQp4cmJsb2NrQGlldGYub3Jn DQpodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3hyYmxvY2s= From bclaise@cisco.com Tue Feb 19 07:22:24 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0520221F880F for ; Tue, 19 Feb 2013 07:22:22 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.529 X-Spam-Level: X-Spam-Status: No, score=-10.529 tagged_above=-999 required=5 tests=[AWL=0.070, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N21WaH45mBMw for ; Tue, 19 Feb 2013 07:22:17 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 8A6B621F8996 for ; Tue, 19 Feb 2013 07:22:14 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JFAqRS020900; Tue, 19 Feb 2013 16:10:52 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JFABNL014956; Tue, 19 Feb 2013 16:10:22 +0100 (CET) Message-ID: <512395D3.7000002@cisco.com> Date: Tue, 19 Feb 2013 16:10:11 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: "MORTON JR., ALFRED (AL)" References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "pm-dir@ietf.org" , Paul Aitken , Alan DeKok Subject: Re: [pm-dir] Performance metrics doctors generated email - new version. X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Feb 2013 15:22:24 -0000 Hi Al, > Hi Paul, > > Thanks for continuing the discussion on tracking > draft review assignments and status. I've been discussing > this off-list with Benoit. Let me put my position on > the table here. > > I'd prefer a solution where: > - everyone can view the page > - directorate reviewers can edit the page > - all editing is WYSIWYG > - no collisions between editors > (if two people are editing, they can see each other and > pressing save does not overwrite a nested editing session) > > Unfortunately, the last two points are beyond Trac's capability. > I think that I could create a doc on Google Drive that > would meet all the above criteria. My experience with google doc is not great, but I'm ready to give it a try... Regards, Benoit > > But, all Directorate members who are active/performing reviews > would need to have an account on Google and be willing to use > it for this purpose. I suspect that many of us already have > a gmail account. > > If there is another solution that meets all the points above, > we could consider that as well. > > Let's hear from all Directorate members on this, it's a cross-road > that will affect how we work together in the future. > > regards, > Al > > >> -----Original Message----- >> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf >> Of Paul Aitken >> Sent: Friday, February 15, 2013 6:12 AM >> To: Benoit Claise >> Cc: pm-dir@ietf.org; Alan DeKok >> Subject: Re: [pm-dir] Performance metrics doctors generated email - new >> version. >> >> Benoit, >> >> How often is the script run? I haven't seen any more postings these last >> 10 days. >> >> If it's run regularly - eg, weekly or nightly - then it'd be useful as >> Al said, to send a delta from the previous run so we can easily see what >> changed. >> >> I saved the current(?) list at >> http://trac.tools.ietf.org/wg/pmol/trac/wiki >> You could imagine additional columns, eg reviewer, comments, status, so >> the status and history can be seen at a glance. >> >> Is this useful? >> >> P. >> >> >> On 04/02/13 12:06, Benoit Claise wrote: >>> Dear all, >>> >>> This is a new version of the script, provided by Alan Dekok. Many >>> thanks Alan. >>> The important improvement is that we also list the drafts that contain >>> a specific keyword. >>> In this case, "performance metric". We could add some more... >>> With this new script, we can then engage sooner with the people who >>> don't know about this directorate and RFC 6390. >>> >>> Potential small optimization for the script: remove the drafts that >>> contain "performance metric" and that already reference RFC 6390 >>> >>> Regards, Benoit >>>> Dear all, >>>> >>>> This is an automatically generated email. >>>> It lists the IETF internet-drafts that reference the PMOL RFC 6390, >>>> as a normative or informative reference. >>>> It also lists all the IETF internet-drafts that contain "performance >>>> metric". >>>> >>>> Regards, Benoit >>>> >>>> =========================================================== >>>> >>>> Normative References >>>> -------------------- >>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>>> Informative References >>>> ---------------------- >>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>>> >>>> drafts containing performance metric >>>> ------------------------------------ >>>> draft-ietf-alto-deployments-05 Active >>>> draft-ietf-alto-protocol-13 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-ippm-rate-problem-02 Active >>>> draft-ietf-ippm-testplan-rfc2680-01 Active >>>> draft-ietf-karp-threats-reqs-07 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-manet-olsrv2-mib-05 Active >>>> draft-ietf-manet-smf-mib-06 Active >>>> draft-ietf-mmusic-media-loopback-27 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-nvo3-framework-01 Active >>>> draft-ietf-opsawg-oam-overview-08 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-rtcweb-rtp-usage-05 Active >>>> draft-ietf-rtgwg-cl-framework-02 Active >>>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>>> draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing >>>> - ID Tracker state >>>> draft-ietf-xrblock-rtcp-xr-synchronization-02 Active >>>> _______________________________________________ >>>> pm-dir mailing list >>>> pm-dir@ietf.org >>>> https://www.ietf.org/mailman/listinfo/pm-dir >>>> >>>> >>> _______________________________________________ >>> pm-dir mailing list >>> pm-dir@ietf.org >>> https://www.ietf.org/mailman/listinfo/pm-dir >> _______________________________________________ >> pm-dir mailing list >> pm-dir@ietf.org >> https://www.ietf.org/mailman/listinfo/pm-dir > From bclaise@cisco.com Tue Feb 19 07:27:20 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B6BDA21F8C22 for ; Tue, 19 Feb 2013 07:27:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.529 X-Spam-Level: X-Spam-Status: No, score=-10.529 tagged_above=-999 required=5 tests=[AWL=0.070, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dLpQ9loiou7d for ; Tue, 19 Feb 2013 07:27:15 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 23DF521F8C17 for ; Tue, 19 Feb 2013 07:27:03 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JF9kvo020752; Tue, 19 Feb 2013 16:09:47 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JF99qE012974; Tue, 19 Feb 2013 16:09:14 +0100 (CET) Message-ID: <51239595.8030906@cisco.com> Date: Tue, 19 Feb 2013 16:09:09 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Paul Aitken References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com> In-Reply-To: <511E17E4.1020809@cisco.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pm-dir@ietf.org, Alan DeKok Subject: Re: [pm-dir] Performance metrics doctors generated email - new version. X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Feb 2013 15:27:21 -0000 Paul, > Benoit, > > How often is the script run? Weekly, on Sundays > I haven't seen any more postings these last 10 days. One was sent on Feb 10th > > If it's run regularly - eg, weekly or nightly - then it'd be useful as > Al said, to send a delta from the previous run so we can easily see > what changed. > > I saved the current(?) list at > http://trac.tools.ietf.org/wg/pmol/trac/wiki > You could imagine additional columns, eg reviewer, comments, status, > so the status and history can be seen at a glance. > > Is this useful? Yes, if we keep updating it. Regards, Benoit > > P. > > > On 04/02/13 12:06, Benoit Claise wrote: >> Dear all, >> >> This is a new version of the script, provided by Alan Dekok. Many >> thanks Alan. >> The important improvement is that we also list the drafts that >> contain a specific keyword. >> In this case, "performance metric". We could add some more... >> With this new script, we can then engage sooner with the people who >> don't know about this directorate and RFC 6390. >> >> Potential small optimization for the script: remove the drafts that >> contain "performance metric" and that already reference RFC 6390 >> >> Regards, Benoit >>> Dear all, >>> >>> This is an automatically generated email. >>> It lists the IETF internet-drafts that reference the PMOL RFC 6390, >>> as a normative or informative reference. >>> It also lists all the IETF internet-drafts that contain "performance >>> metric". >>> >>> Regards, Benoit >>> >>> =========================================================== >>> >>> Normative References >>> -------------------- >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>> Informative References >>> ---------------------- >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>> >>> drafts containing performance metric >>> ------------------------------------ >>> draft-ietf-alto-deployments-05 Active >>> draft-ietf-alto-protocol-13 In IESG processing >>> - ID Tracker state >>> draft-ietf-ippm-rate-problem-02 Active >>> draft-ietf-ippm-testplan-rfc2680-01 Active >>> draft-ietf-karp-threats-reqs-07 In IESG processing >>> - ID Tracker state >>> draft-ietf-manet-olsrv2-mib-05 Active >>> draft-ietf-manet-smf-mib-06 Active >>> draft-ietf-mmusic-media-loopback-27 In IESG processing >>> - ID Tracker state >>> draft-ietf-nvo3-framework-01 Active >>> draft-ietf-opsawg-oam-overview-08 In IESG processing >>> - ID Tracker state >>> draft-ietf-rtcweb-rtp-usage-05 Active >>> draft-ietf-rtgwg-cl-framework-02 Active >>> draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-concsec-03 Active >>> draft-ietf-xrblock-rtcp-xr-decodability-07 Active >>> draft-ietf-xrblock-rtcp-xr-discard-11 Active >>> draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active >>> draft-ietf-xrblock-rtcp-xr-jb-07 Active >>> draft-ietf-xrblock-rtcp-xr-loss-conceal-04 Active >>> draft-ietf-xrblock-rtcp-xr-qoe-03 Active >>> draft-ietf-xrblock-rtcp-xr-summary-stat-06 In IESG processing >>> - ID Tracker state >>> draft-ietf-xrblock-rtcp-xr-synchronization-02 Active >>> _______________________________________________ >>> pm-dir mailing list >>> pm-dir@ietf.org >>> https://www.ietf.org/mailman/listinfo/pm-dir >>> >>> >> >> _______________________________________________ >> pm-dir mailing list >> pm-dir@ietf.org >> https://www.ietf.org/mailman/listinfo/pm-dir > > From bclaise@cisco.com Tue Feb 19 16:23:56 2013 Return-Path: X-Original-To: pm-dir@ietfa.amsl.com Delivered-To: pm-dir@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ABE1721F86D3 for ; Tue, 19 Feb 2013 16:23:56 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -9.921 X-Spam-Level: X-Spam-Status: No, score=-9.921 tagged_above=-999 required=5 tests=[AWL=-0.563, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SARE_LWSHORTT=1.24] Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i15qAtBf6DKD for ; Tue, 19 Feb 2013 16:23:55 -0800 (PST) Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 254D821F85ED for ; Tue, 19 Feb 2013 16:23:43 -0800 (PST) X-TACSUNS: Virus Scanned Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JNxXkH008098; Wed, 20 Feb 2013 00:59:33 +0100 (CET) Received: from [10.60.67.84] (ams-bclaise-8913.cisco.com [10.60.67.84]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1JNwrd6018606; Wed, 20 Feb 2013 00:59:03 +0100 (CET) Message-ID: <512411BD.2020609@cisco.com> Date: Wed, 20 Feb 2013 00:58:53 +0100 From: Benoit Claise User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: "pm-dir@ietf.org" References: <50F93070.7040107@ericsson.com> <51057B67.7090205@cisco.com> <511D70C6.2080601@cisco.com> In-Reply-To: <511D70C6.2080601@cisco.com> Content-Type: multipart/alternative; boundary="------------040101010906030104000603" Cc: Gonzalo Camarillo Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-xrblock-rtcp-xr-summary-stat-06 X-BeenThere: pm-dir@ietf.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Performance Metrics Directorate Discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Feb 2013 00:23:56 -0000 This is a multi-part message in MIME format. --------------040101010906030104000603 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Dear pm-dir, [reducing the list to pm-dir and Gonzallo, XRBLOCK AD] I filed a DISCUSS on this document. See https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-stat/ballot/#benoit-claise for the details. We should give the right guidelines to performance metrics related draft authors. Maybe we should even mention http://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/ballot/#benoit-claise in our WIKI? Regards, Benoit > Dear authors, > > I'm reviewing version 8 for the IESG call next week. >> Benoit, >> >> Since there is a short term deadline >> (the Last Call has ended on Feb 1) >> I'll provide the PM Dir review. >> >> Al >> >> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- >> Since the draft describes reporting formats and (apparently) does not >> intend to define new metrics, much of 6390 guidance is out of scope. > This is the key question. > Do you define new metrics? > When I see the following (to take just one example): > > Burst Loss Rate: 16 bits > > The fraction of packets lost during bursts since the beginning of > reception, expressed as a fixed point number with the binary point > at the left edge of the field. This value is calculated by > dividing Packets Loss in Bursts by Total Packets expected in > Bursts, multiplying the result of the division by 7FFF, with the > maximum value 7FFF, and taking the integer part as follows: > > Packets Loss in Bursts / Total Packets expected in Bursts > > If the measurement is unavailable, the value 0x8000 MUST be > reported. > > ... it seems to me that you define new metrics, and that therefore the > RFC 6390 template should be applied. > What do I miss? > > Regards, Benoit >> However, see the comments on section 4.1.2 below regarding metric definitions. >> >> Comments: >> >> Section 1.1 of the Intro says: >> >> This draft defines three new block types to augment those defined in >> [RFC3611] for use in a range of RTP applications. >> >> ok so far, but the next paragraph is hard to decipher, perhaps a simple >> numbered list of the new blocks would help: >> >> 1. Burst/Gap Loss Summary Statistics Metrics Block >> 2. Burst/Gap Discard Summary Statistics Metrics Block >> 3. Frame Impairment Statistics Summary Metrics Block >> >> back to the existing text: >> The first two block types support the reporting of burst gap loss/ >> discard summary statistics including packet loss/discard proportion, >> mean and variance and belong to the class of transport-related end >> system metrics defined in [RFC6792]. These two blocks are intended >> to be used in conjunction with information from the Burst Gap Loss >> Metrics Block or Burst Gap Discard Metrics Block, and on which these >> two block therefore depend. >> add a reference to [RFC3611] for these two blocks. >> >> The metrics in the Burst Gap Loss block >> or Burst Gap Discard Metrics Block can be used independently of the >> metrics defined in the first two blocks. >> >> That's obvious, because anyone implementing RFC3611 is already doing it. >> >> The authors appear to have avoided using "video" as an adjective when >> referring to video frames throughout the memo. For example: >> >> The third block supports the reporting of detailed statistics for >> each frame type, including the number of frames received, lost and >> discarded of each frame type in the Group of Pictures (GOP) and >> additional data allowing the calculation of statistical parameters >> (e.g.,the proportion of each frame type impaired by packet loss and >> discard). The metrics defined in this block belong to the class of >> application layer metrics defined in [RFC6792]. >> >> These are all video frames above, no? At least say so once. >> >> Section 2.1, on Terminology, says >> >> Picture Type >> >> Picture Types used in the different video algorithms are composed >> of the Key frame and Derived frames. The Key frame is also called >> a reference frame and used as a reference for predicting other >> pictures. It is coded without prediction from other pictures. >> Derived frames are derived from a Key frame using a prediction >> algorithm. >> >> At first this seems quite generic, but the definition may fail when one slice >> of the video frame is independently coded, and other slices are predictively >> coded (this has been done to smooth the video rate). Is any video frame that >> includes independent or reference coding a Key frame? >> >> Section 3.1.2 says: >> Burst Loss Rate: 16 bits >> >> The fraction of packets lost during bursts since the beginning of >> reception, expressed as a fixed point number with the binary point >> at the left edge of the field. This value is calculated by >> dividing Packets Loss in Bursts by Total Packets expected in >> Bursts as follows: >> Packets Loss in Bursts / Total Packets expected in Bursts >> >> How is a burst loss ratio = 1.0 reported? All the digits are to the >> right of the decimal place. >> >> Burst Duration Mean:16bits >> >> The mean burst duration is obtained as the quotient: >> >> mean = Sum of Burst Durations / Number of Bursts >> >> How is Divide by Zero handled? >> >> Burst Duration Variance:16bits >> >> The variance of the burst duration is obtained using the standard >> result: >> >> var = ( Sum of Squares of Burst Durations - Number of Bursts * >> mean^2 ) / (Number of Bursts - 1) >> >> How is Divide by Zero handled? (e.g., Number of Bursts = 1) >> >> Section 4.1.2 says >> >> Number of full frames lost (lost_full_frames): 32 bits >> >> If one frame is completely lost, this frame is regarded as one >> lost full frame. The lost_full_frames is equivalent to the number >> of full frames lost in the above sequence number interval. >> >> Is this a metric definition, fully lost video frames? >> How is this event detected so it can be counted? >> Is there a reference for this metric available? >> >> Number of partial frames lost (lost_partial_frames): 32 bits >> >> If one frame is partially lost, this frame is regarded as one lost >> fractional frame. The value of the lost_partial_frames field is >> equivalent to the number of partial frames lost in the above >> sequence number interval. >> >> Is this a metric definition, partially lost video frames? >> How is this event detected so it can be counted? >> Is there a reference for this metric available? >> >> -=-=-=-=-=-=- >> >>> -----Original Message----- >>> From:pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf >>> Of Benoit Claise >>> Sent: Sunday, January 27, 2013 2:09 PM >>> To:pm-dir@ietf.org >>> Cc:xrblock-chairs@tools.ietf.org; draft-ietf-xrblock-rtcp-xr-summary- >>> stat@tools.ietf.org; Gonzalo Camarillo >>> Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf- >>> xrblock-rtcp-xr-summary-stat-06 >>> >>> Dear all, >>> >>> Here is an official request for the RFC 6390 review. >>> Who would like to volunteer? >>> >>> Regards, Benoit >>>> Hi Benoit, >>>> >>>> as suggested by Dan in his PROTO writeup, I would like to request the >>>> performance metrics directorate to perform an RFC 6390 review of the >>>> following draft as part of its IETF LC: >>>> >>>> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary- >>> stat/ >>>> Could you please arrange such a review? >>>> >>>> Thanks, >>>> >>>> Gonzalo >>>> >>>> >>> _______________________________________________ >>> pm-dir mailing list >>> pm-dir@ietf.org >>> https://www.ietf.org/mailman/listinfo/pm-dir > > > > _______________________________________________ > pm-dir mailing list > pm-dir@ietf.org > https://www.ietf.org/mailman/listinfo/pm-dir --------------040101010906030104000603 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Dear pm-dir,

[reducing the list to pm-dir and Gonzallo, XRBLOCK AD]
I filed a DISCUSS on this document.
See https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-stat/ballot/#benoit-claise for the details.
We should give the right guidelines to performance metrics related draft authors.
Maybe we should even mention http://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/ballot/#benoit-claise in our WIKI?

Regards, Benoit
Dear authors,

I'm reviewing version 8 for the IESG call next week.
Benoit,

Since there is a short term deadline 
(the Last Call has ended on Feb 1) 
I'll provide the PM Dir review.

Al

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Since the draft describes reporting formats and (apparently) does not 
intend to define new metrics, much of 6390 guidance is out of scope.
This is the key question.
Do you define new metrics?
When I see the following (to take just one example):
 Burst Loss Rate: 16 bits

      The fraction of packets lost during bursts since the beginning of
      reception, expressed as a fixed point number with the binary point
      at the left edge of the field.  This value is calculated by
      dividing Packets Loss in Bursts by Total Packets expected in
      Bursts, multiplying the result of the division by 7FFF, with the
      maximum value 7FFF, and taking the integer part as follows:

      Packets Loss in Bursts / Total Packets expected in Bursts

      If the measurement is unavailable, the value 0x8000 MUST be
      reported.
... it seems to me that you define new metrics, and that therefore the RFC 6390 template should be applied.
What do I miss?

Regards, Benoit
However, see the comments on section 4.1.2 below regarding metric definitions.

Comments:

Section 1.1 of the Intro says:

   This draft defines three new block types to augment those defined in
   [RFC3611] for use in a range of RTP applications.

ok so far, but the next paragraph is hard to decipher, perhaps a simple
numbered list of the new blocks would help:

    1. Burst/Gap Loss Summary Statistics Metrics Block
    2. Burst/Gap Discard Summary Statistics Metrics Block
    3. Frame Impairment Statistics Summary Metrics Block

back to the existing text:
   The first two block types support the reporting of burst gap loss/
   discard summary statistics including packet loss/discard proportion,
   mean and variance and belong to the class of transport-related end
   system metrics defined in [RFC6792].  These two blocks are intended
   to be used in conjunction with information from the Burst Gap Loss
   Metrics Block or Burst Gap Discard Metrics Block, and on which these
   two block therefore depend.  
add a reference to [RFC3611] for these two blocks.

   The metrics in the Burst Gap Loss block
   or Burst Gap Discard Metrics Block can be used independently of the
   metrics defined in the first two blocks.

That's obvious, because anyone implementing RFC3611 is already doing it.

The authors appear to have avoided using "video" as an adjective when 
referring to video frames throughout the memo. For example:

   The third block supports the reporting of detailed statistics for
   each frame type, including the number of frames received, lost and
   discarded of each frame type in the Group of Pictures (GOP) and
   additional data allowing the calculation of statistical parameters
   (e.g.,the proportion of each frame type impaired by packet loss and
   discard).  The metrics defined in this block belong to the class of
   application layer metrics defined in [RFC6792].

These are all video frames above, no?  At least say so once.

Section 2.1, on Terminology, says

    Picture Type

      Picture Types used in the different video algorithms are composed
      of the Key frame and Derived frames.  The Key frame is also called
      a reference frame and used as a reference for predicting other
      pictures.  It is coded without prediction from other pictures.
      Derived frames are derived from a Key frame using a prediction
      algorithm.

At first this seems quite generic, but the definition may fail when one slice
of the video frame is independently coded, and other slices are predictively
coded (this has been done to smooth the video rate). Is any video frame that
includes independent or reference coding a Key frame?

Section 3.1.2 says:
   Burst Loss Rate: 16 bits

      The fraction of packets lost during bursts since the beginning of
      reception, expressed as a fixed point number with the binary point
      at the left edge of the field.  This value is calculated by
      dividing Packets Loss in Bursts by Total Packets expected in
      Bursts as follows:
          Packets Loss in Bursts / Total Packets expected in Bursts

How is a burst loss ratio = 1.0 reported?  All the digits are to the 
right of the decimal place.

   Burst Duration Mean:16bits

      The mean burst duration is obtained as the quotient:

      mean = Sum of Burst Durations / Number of Bursts

How is Divide by Zero handled?

   Burst Duration Variance:16bits

      The variance of the burst duration is obtained using the standard
      result:

      var = ( Sum of Squares of Burst Durations - Number of Bursts *
      mean^2 ) / (Number of Bursts - 1)

How is Divide by Zero handled? (e.g., Number of Bursts = 1)

Section 4.1.2 says

   Number of full frames lost (lost_full_frames): 32 bits

      If one frame is completely lost, this frame is regarded as one
      lost full frame.  The lost_full_frames is equivalent to the number
      of full frames lost in the above sequence number interval.

Is this a metric definition, fully lost video frames? 
How is this event detected so it can be counted?
Is there a reference for this metric available?

   Number of partial frames lost (lost_partial_frames): 32 bits

      If one frame is partially lost, this frame is regarded as one lost
      fractional frame.  The value of the lost_partial_frames field is
      equivalent to the number of partial frames lost in the above
      sequence number interval.

Is this a metric definition, partially lost video frames? 
How is this event detected so it can be counted?
Is there a reference for this metric available?

-=-=-=-=-=-=-


-----Original Message-----
From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
Of Benoit Claise
Sent: Sunday, January 27, 2013 2:09 PM
To: pm-dir@ietf.org
Cc: xrblock-chairs@tools.ietf.org; draft-ietf-xrblock-rtcp-xr-summary-
stat@tools.ietf.org; Gonzalo Camarillo
Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-
xrblock-rtcp-xr-summary-stat-06

Dear all,

Here is an official request for the RFC 6390 review.
Who would like to volunteer?

Regards, Benoit

Hi Benoit,

as suggested by Dan in his PROTO writeup, I would like to request the
performance metrics directorate to perform an RFC 6390 review of the
following draft as part of its IETF LC:

https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-

stat/

Could you please arrange such a review?

Thanks,

Gonzalo



_______________________________________________
pm-dir mailing list
pm-dir@ietf.org
https://www.ietf.org/mailman/listinfo/pm-dir


      

      

      

      

      

      
_______________________________________________
pm-dir mailing list
pm-dir@ietf.org
https://www.ietf.org/mailman/listinfo/pm-dir


    

    

  


--------------040101010906030104000603--

From gonzalo.camarillo@ericsson.com  Wed Feb 20 00:23:04 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3CCE021F8AE8 for ; Wed, 20 Feb 2013 00:23:04 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -105.559
X-Spam-Level: 
X-Spam-Status: No, score=-105.559 tagged_above=-999 required=5 tests=[AWL=-0.550, BAYES_00=-2.599, HELO_EQ_SE=0.35, RCVD_IN_DNSWL_MED=-4, SARE_LWSHORTT=1.24, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eLwPP-Oh0AHd for ; Wed, 20 Feb 2013 00:23:03 -0800 (PST)
Received: from mailgw1.ericsson.se (mailgw1.ericsson.se [193.180.251.45]) by ietfa.amsl.com (Postfix) with ESMTP id A562B21F883A for ; Wed, 20 Feb 2013 00:23:02 -0800 (PST)
X-AuditID: c1b4fb2d-b7f316d0000028db-49-512487e55e9d
Received: from esessmw0197.eemea.ericsson.se (Unknown_Domain [153.88.253.124]) by mailgw1.ericsson.se (Symantec Mail Security) with SMTP id 28.22.10459.5E784215; Wed, 20 Feb 2013 09:23:01 +0100 (CET)
Received: from [131.160.126.44] (153.88.115.8) by esessmw0197.eemea.ericsson.se (153.88.115.88) with Microsoft SMTP Server id 8.3.279.1; Wed, 20 Feb 2013 09:23:01 +0100
Message-ID: <512487E4.7010604@ericsson.com>
Date: Wed, 20 Feb 2013 10:23:00 +0200
From: Gonzalo Camarillo 
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2
MIME-Version: 1.0
To: Benoit Claise 
References: <50F93070.7040107@ericsson.com> <51057B67.7090205@cisco.com>  <511D70C6.2080601@cisco.com> <512411BD.2020609@cisco.com>
In-Reply-To: <512411BD.2020609@cisco.com>
X-Enigmail-Version: 1.4.6
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupjluLIzCtJLcpLzFFi42KZGfG3Rvdpu0qgwYQvOhZHH0tYHP1g6cDk MeX3RlaPJUt+MgUwRXHZpKTmZJalFunbJXBl9G4QKpjkXDHjeD97A+Nsky5GTg4JAROJm89/ MkHYYhIX7q1nA7GFBE4ySmw57d/FyAVkr2GUeHx4FQtIgldAW2J2xzYwm0VAVeL8xrdgDWwC FhJbbt0Hi4sKREm8v9rEDFEvKHFy5hOwuAhQff/WLWA2s4C6xK9/axhBbGGBJIln1/ezQSw7 wyhx4MN1sASngKbEyRt3mCGuk5RYNK0TqllPYsrVFkYIW15i+9s5zBBXa0ssf9bCMoFRaBaS 3bOQtMxC0rKAkXkVI3tuYmZOernhJkZgmB7c8lt3B+OpcyKHGKU5WJTEecNcLwQICaQnlqRm p6YWpBbFF5XmpBYfYmTi4JRqYOQS/HldUUJm3y+PE9e5vzK+WjFxKsN35dXP7E9N3rXiZGnh ETZr3++GlrUqzTpLPln2/Nw2/cB0Pb+MoI/L33z5d+66VnmiLuP1HJn8Lt7otQqewROfdXYE R7TuyJrTXLn5TqN8unWlU+7uttMqu2M3XrA4JcRxQ09xz9zX9redtQumMR0LilRiKc5INNRi LipOBAACiQ+dIQIAAA==
Cc: "pm-dir@ietf.org" 
Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-xrblock-rtcp-xr-summary-stat-06
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 20 Feb 2013 08:23:04 -0000

Hi Benoit,

thanks for following this up. I have already told Dan, who is this
document's shepherd, to get involved in the discussions so that we can
move forward.

Cheers,

Gonzalo

On 20/02/2013 1:58 AM, Benoit Claise wrote:
> Dear pm-dir,
> 
> [reducing the list to pm-dir and Gonzallo, XRBLOCK AD]
> I filed a DISCUSS on this document.
> See
> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-stat/ballot/#benoit-claise
> for the details.
> We should give the right guidelines to performance metrics related draft
> authors.
> Maybe we should even mention
> http://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/ballot/#benoit-claise
> in our WIKI?
> 
> Regards, Benoit
>> Dear authors,
>>
>> I'm reviewing version 8 for the IESG call next week.
>>> Benoit,
>>>
>>> Since there is a short term deadline 
>>> (the Last Call has ended on Feb 1) 
>>> I'll provide the PM Dir review.
>>>
>>> Al
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>>> Since the draft describes reporting formats and (apparently) does not 
>>> intend to define new metrics, much of 6390 guidance is out of scope.
>> This is the key question.
>> Do you define new metrics?
>> When I see the following (to take just one example):
>>
>>      Burst Loss Rate: 16 bits
>>
>>           The fraction of packets lost during bursts since the beginning of
>>           reception, expressed as a fixed point number with the binary point
>>           at the left edge of the field.  This value is calculated by
>>           dividing Packets Loss in Bursts by Total Packets expected in
>>           Bursts, multiplying the result of the division by 7FFF, with the
>>           maximum value 7FFF, and taking the integer part as follows:
>>
>>           Packets Loss in Bursts / Total Packets expected in Bursts
>>
>>           If the measurement is unavailable, the value 0x8000 MUST be
>>           reported.
>>
>> ... it seems to me that you define new metrics, and that therefore the
>> RFC 6390 template should be applied.
>> What do I miss?
>>
>> Regards, Benoit
>>> However, see the comments on section 4.1.2 below regarding metric definitions.
>>>
>>> Comments:
>>>
>>> Section 1.1 of the Intro says:
>>>
>>>    This draft defines three new block types to augment those defined in
>>>    [RFC3611] for use in a range of RTP applications.
>>>
>>> ok so far, but the next paragraph is hard to decipher, perhaps a simple
>>> numbered list of the new blocks would help:
>>>
>>>     1. Burst/Gap Loss Summary Statistics Metrics Block
>>>     2. Burst/Gap Discard Summary Statistics Metrics Block
>>>     3. Frame Impairment Statistics Summary Metrics Block
>>>
>>> back to the existing text:
>>>    The first two block types support the reporting of burst gap loss/
>>>    discard summary statistics including packet loss/discard proportion,
>>>    mean and variance and belong to the class of transport-related end
>>>    system metrics defined in [RFC6792].  These two blocks are intended
>>>    to be used in conjunction with information from the Burst Gap Loss
>>>    Metrics Block or Burst Gap Discard Metrics Block, and on which these
>>>    two block therefore depend.  
>>> add a reference to [RFC3611] for these two blocks.
>>>
>>>    The metrics in the Burst Gap Loss block
>>>    or Burst Gap Discard Metrics Block can be used independently of the
>>>    metrics defined in the first two blocks.
>>>
>>> That's obvious, because anyone implementing RFC3611 is already doing it.
>>>
>>> The authors appear to have avoided using "video" as an adjective when 
>>> referring to video frames throughout the memo. For example:
>>>
>>>    The third block supports the reporting of detailed statistics for
>>>    each frame type, including the number of frames received, lost and
>>>    discarded of each frame type in the Group of Pictures (GOP) and
>>>    additional data allowing the calculation of statistical parameters
>>>    (e.g.,the proportion of each frame type impaired by packet loss and
>>>    discard).  The metrics defined in this block belong to the class of
>>>    application layer metrics defined in [RFC6792].
>>>
>>> These are all video frames above, no?  At least say so once.
>>>
>>> Section 2.1, on Terminology, says
>>>
>>>     Picture Type
>>>
>>>       Picture Types used in the different video algorithms are composed
>>>       of the Key frame and Derived frames.  The Key frame is also called
>>>       a reference frame and used as a reference for predicting other
>>>       pictures.  It is coded without prediction from other pictures.
>>>       Derived frames are derived from a Key frame using a prediction
>>>       algorithm.
>>>
>>> At first this seems quite generic, but the definition may fail when one slice
>>> of the video frame is independently coded, and other slices are predictively
>>> coded (this has been done to smooth the video rate). Is any video frame that
>>> includes independent or reference coding a Key frame?
>>>
>>> Section 3.1.2 says:
>>>    Burst Loss Rate: 16 bits
>>>
>>>       The fraction of packets lost during bursts since the beginning of
>>>       reception, expressed as a fixed point number with the binary point
>>>       at the left edge of the field.  This value is calculated by
>>>       dividing Packets Loss in Bursts by Total Packets expected in
>>>       Bursts as follows:
>>>           Packets Loss in Bursts / Total Packets expected in Bursts
>>>
>>> How is a burst loss ratio = 1.0 reported?  All the digits are to the 
>>> right of the decimal place.
>>>
>>>    Burst Duration Mean:16bits
>>>
>>>       The mean burst duration is obtained as the quotient:
>>>
>>>       mean = Sum of Burst Durations / Number of Bursts
>>>
>>> How is Divide by Zero handled?
>>>
>>>    Burst Duration Variance:16bits
>>>
>>>       The variance of the burst duration is obtained using the standard
>>>       result:
>>>
>>>       var = ( Sum of Squares of Burst Durations - Number of Bursts *
>>>       mean^2 ) / (Number of Bursts - 1)
>>>
>>> How is Divide by Zero handled? (e.g., Number of Bursts = 1)
>>>
>>> Section 4.1.2 says
>>>
>>>    Number of full frames lost (lost_full_frames): 32 bits
>>>
>>>       If one frame is completely lost, this frame is regarded as one
>>>       lost full frame.  The lost_full_frames is equivalent to the number
>>>       of full frames lost in the above sequence number interval.
>>>
>>> Is this a metric definition, fully lost video frames? 
>>> How is this event detected so it can be counted?
>>> Is there a reference for this metric available?
>>>
>>>    Number of partial frames lost (lost_partial_frames): 32 bits
>>>
>>>       If one frame is partially lost, this frame is regarded as one lost
>>>       fractional frame.  The value of the lost_partial_frames field is
>>>       equivalent to the number of partial frames lost in the above
>>>       sequence number interval.
>>>
>>> Is this a metric definition, partially lost video frames? 
>>> How is this event detected so it can be counted?
>>> Is there a reference for this metric available?
>>>
>>> -=-=-=-=-=-=-
>>>
>>>> -----Original Message-----
>>>> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
>>>> Of Benoit Claise
>>>> Sent: Sunday, January 27, 2013 2:09 PM
>>>> To: pm-dir@ietf.org
>>>> Cc: xrblock-chairs@tools.ietf.org; draft-ietf-xrblock-rtcp-xr-summary-
>>>> stat@tools.ietf.org; Gonzalo Camarillo
>>>> Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-
>>>> xrblock-rtcp-xr-summary-stat-06
>>>>
>>>> Dear all,
>>>>
>>>> Here is an official request for the RFC 6390 review.
>>>> Who would like to volunteer?
>>>>
>>>> Regards, Benoit
>>>>> Hi Benoit,
>>>>>
>>>>> as suggested by Dan in his PROTO writeup, I would like to request the
>>>>> performance metrics directorate to perform an RFC 6390 review of the
>>>>> following draft as part of its IETF LC:
>>>>>
>>>>> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-
>>>> stat/
>>>>> Could you please arrange such a review?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Gonzalo
>>>>>
>>>>>
>>>> _______________________________________________
>>>> pm-dir mailing list
>>>> pm-dir@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/pm-dir
>>
>>
>>
>> _______________________________________________
>> pm-dir mailing list
>> pm-dir@ietf.org
>> https://www.ietf.org/mailman/listinfo/pm-dir
> 


From acmorton@att.com  Wed Feb 20 06:36:14 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3F16E21F87D1 for ; Wed, 20 Feb 2013 06:36:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.474
X-Spam-Level: 
X-Spam-Status: No, score=-106.474 tagged_above=-999 required=5 tests=[AWL=0.125, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w8FuQmaINMjM for ; Wed, 20 Feb 2013 06:36:13 -0800 (PST)
Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 8AA5D21F875F for ; Wed, 20 Feb 2013 06:36:12 -0800 (PST)
Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id 2F6FF120826; Wed, 20 Feb 2013 09:38:28 -0500 (EST)
Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-blue.research.att.com (Postfix) with ESMTP id 08423EFC58; Wed, 20 Feb 2013 09:36:12 -0500 (EST)
Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Wed, 20 Feb 2013 09:36:11 -0500
From: "MORTON JR., ALFRED  (AL)" 
To: Benoit Claise 
Date: Wed, 20 Feb 2013 09:36:11 -0500
Thread-Topic: [pm-dir] Performance metrics doctors generated email - new version.
Thread-Index: Ac4OtPiepddzzK1yQFqFrs40W64IegAwiniQ
Message-ID: 
References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com>  <512395D3.7000002@cisco.com>
In-Reply-To: <512395D3.7000002@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "pm-dir@ietf.org" , Paul Aitken , Alan DeKok 
Subject: Re: [pm-dir] Performance metrics doctors generated email - new	version.
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 20 Feb 2013 14:36:14 -0000

Ok Benoit,=20
I have a version of the doc going, but probably won't finish
till after the revised deadline expires...

Before going further, it would be useful to know if anyone
in the directorate objects to using or establishing a gmail
account in order to register their review assignments and status.

regards,
Al

> -----Original Message-----
> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
> Of Benoit Claise
> Sent: Tuesday, February 19, 2013 10:10 AM
> To: MORTON JR., ALFRED (AL)
> Cc: pm-dir@ietf.org; Paul Aitken; Alan DeKok
...
> Hi Al,
> > Hi Paul,
> >
> > Thanks for continuing the discussion on tracking
> > draft review assignments and status.  I've been discussing
> > this off-list with Benoit. Let me put my position on
> > the table here.
> >
> > I'd prefer a solution where:
> >   - everyone can view the page
> >   - directorate reviewers can edit the page
> >   - all editing is WYSIWYG
> >   - no collisions between editors
> >    (if two people are editing, they can see each other and
> >     pressing save does not overwrite a nested editing session)
> >
> > Unfortunately, the last two points are beyond Trac's capability.
> > I think that I could create a doc on Google Drive that
> > would meet all the above criteria.
> My experience with google doc is not great, but I'm ready to give it a
> try...
>=20
> Regards, Benoit


From cpignata@cisco.com  Wed Feb 20 06:59:38 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 40A4B21F8802 for ; Wed, 20 Feb 2013 06:59:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -110.562
X-Spam-Level: 
X-Spam-Status: No, score=-110.562 tagged_above=-999 required=5 tests=[AWL=0.038, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V7IS6Fa7XcBP for ; Wed, 20 Feb 2013 06:59:37 -0800 (PST)
Received: from rcdn-iport-1.cisco.com (rcdn-iport-1.cisco.com [173.37.86.72]) by ietfa.amsl.com (Postfix) with ESMTP id 4D49321F87AF for ; Wed, 20 Feb 2013 06:59:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1782; q=dns/txt; s=iport; t=1361372377; x=1362581977; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=plXIkTli8dmvhFTPFERmrLECa7D4bxkv8ISPxUantr8=; b=UyTUZ4G4DhpJGOHi18I0IWzm2BieScU9lpqUo7oj7qKEyiVaZfyZUNhA m3TnD3CobW98gmDgypmaNhadJmKpyS8j67eNnSYA876gJ/p0Fh3iMp5io Seb65m1OL+vnxWtHR/d829gzXAsAT1S37/Zum1JwZi9910jtNWOAl31ps 4=;
X-IronPort-AV: E=Sophos;i="4.84,702,1355097600"; d="scan'208";a="178973546"
Received: from rcdn-core2-5.cisco.com ([173.37.113.192]) by rcdn-iport-1.cisco.com with ESMTP; 20 Feb 2013 14:59:37 +0000
Received: from xhc-aln-x09.cisco.com (xhc-aln-x09.cisco.com [173.36.12.83]) by rcdn-core2-5.cisco.com (8.14.5/8.14.5) with ESMTP id r1KExaPH030304 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 20 Feb 2013 14:59:36 GMT
Received: from xmb-aln-x02.cisco.com ([169.254.5.155]) by xhc-aln-x09.cisco.com ([173.36.12.83]) with mapi id 14.02.0318.004; Wed, 20 Feb 2013 08:59:36 -0600
From: "Carlos Pignataro (cpignata)" 
To: "MORTON JR., ALFRED (AL)" 
Thread-Topic: [pm-dir] Performance metrics doctors generated email -	new version.
Thread-Index: AQHOD3enqu4DJS35tkOiaMWGobBWsZiDO3CA
Date: Wed, 20 Feb 2013 14:59:35 +0000
Message-ID: <95067C434CE250468B77282634C96ED3229935B7@xmb-aln-x02.cisco.com>
References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com>  <512395D3.7000002@cisco.com> 
In-Reply-To: 
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.150.44.213]
Content-Type: text/plain; charset="us-ascii"
Content-ID: 
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "Benoit Claise \(bclaise\)" , "pm-dir@ietf.org" , "Paul Aitken \(paitken\)" , Alan DeKok 
Subject: Re: [pm-dir] Performance metrics doctors generated email -	new	version.
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 20 Feb 2013 14:59:38 -0000

On Feb 20, 2013, at 9:36 AM, "MORTON JR., ALFRED (AL)"  w=
rote:

> Ok Benoit,=20
> I have a version of the doc going, but probably won't finish
> till after the revised deadline expires...
>=20
> Before going further, it would be useful to know if anyone
> in the directorate objects to using or establishing a gmail
> account in order to register their review assignments and status.
>=20

No objection. I think it's a great solution.

Thanks,

-- Carlos.

> regards,
> Al
>=20
>> -----Original Message-----
>> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
>> Of Benoit Claise
>> Sent: Tuesday, February 19, 2013 10:10 AM
>> To: MORTON JR., ALFRED (AL)
>> Cc: pm-dir@ietf.org; Paul Aitken; Alan DeKok
> ...
>> Hi Al,
>>> Hi Paul,
>>>=20
>>> Thanks for continuing the discussion on tracking
>>> draft review assignments and status.  I've been discussing
>>> this off-list with Benoit. Let me put my position on
>>> the table here.
>>>=20
>>> I'd prefer a solution where:
>>>  - everyone can view the page
>>>  - directorate reviewers can edit the page
>>>  - all editing is WYSIWYG
>>>  - no collisions between editors
>>>   (if two people are editing, they can see each other and
>>>    pressing save does not overwrite a nested editing session)
>>>=20
>>> Unfortunately, the last two points are beyond Trac's capability.
>>> I think that I could create a doc on Google Drive that
>>> would meet all the above criteria.
>> My experience with google doc is not great, but I'm ready to give it a
>> try...
>>=20
>> Regards, Benoit
>=20
> _______________________________________________
> pm-dir mailing list
> pm-dir@ietf.org
> https://www.ietf.org/mailman/listinfo/pm-dir
>=20


From paitken@cisco.com  Wed Feb 20 07:46:51 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E2A7E21F883E for ; Wed, 20 Feb 2013 07:46:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level: 
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0UMOWPhqeWQf for ; Wed, 20 Feb 2013 07:46:51 -0800 (PST)
Received: from ams-iport-1.cisco.com (ams-iport-1.cisco.com [144.254.224.140]) by ietfa.amsl.com (Postfix) with ESMTP id 31C8C21F883C for ; Wed, 20 Feb 2013 07:46:51 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=60; q=dns/txt; s=iport; t=1361375211; x=1362584811; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; bh=Wbhm9mXinBLEl5n0zrZIbKZ/+MQEApGV1d7Q8EVPKcQ=; b=Vuv9CmcfeXFNbcSJhlkn9xRkW6SBxnqCu9kbjCIdQjWnBnjeAKld2oY+ klhigK9sdBMrRmzOVzASR2lFC07cFM6pQuKjaxh1IHC6UCmadEEbsZxGc T05X/nuigebdlwPw656ZPiUi00zBunSFjKt6vbt7h1aoUuKojYhnxJpAo g=;
X-IronPort-AV: E=Sophos;i="4.84,702,1355097600"; d="scan'208";a="150683299"
Received: from ams-core-3.cisco.com ([144.254.72.76]) by ams-iport-1.cisco.com with ESMTP; 20 Feb 2013 15:46:50 +0000
Received: from [144.254.153.46] (dhcp-144-254-153-46.cisco.com [144.254.153.46]) by ams-core-3.cisco.com (8.14.5/8.14.5) with ESMTP id r1KFkoHB006642; Wed, 20 Feb 2013 15:46:50 GMT
Message-ID: <5124EFEB.7070107@cisco.com>
Date: Wed, 20 Feb 2013 15:46:51 +0000
From: Paul Aitken 
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130106 Thunderbird/17.0.2
MIME-Version: 1.0
To: "Carlos Pignataro (cpignata)" , "MORTON JR., ALFRED (AL)" 
References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com>  <512395D3.7000002@cisco.com>  <95067C434CE250468B77282634C96ED3229935B7@xmb-aln-x02.cisco.com>
In-Reply-To: <95067C434CE250468B77282634C96ED3229935B7@xmb-aln-x02.cisco.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: "Benoit Claise \(bclaise\)" , "pm-dir@ietf.org" , Alan DeKok 
Subject: Re: [pm-dir] Performance metrics doctors generated email -	new version.
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Wed, 20 Feb 2013 15:46:52 -0000

> No objection. I think it's a great solution.

+1

P.

From yaakov_s@rad.com  Wed Feb 20 23:06:47 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D41E121F84C6 for ; Wed, 20 Feb 2013 23:06:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.098
X-Spam-Level: 
X-Spam-Status: No, score=-103.098 tagged_above=-999 required=5 tests=[AWL=-0.500, BAYES_00=-2.599, UNPARSEABLE_RELAY=0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2yIgjFYd9I1F for ; Wed, 20 Feb 2013 23:06:47 -0800 (PST)
Received: from rad.co.il (mailrelay01.rad.co.il [62.0.23.252]) by ietfa.amsl.com (Postfix) with ESMTP id 4C83921F8E01 for ; Wed, 20 Feb 2013 23:06:44 -0800 (PST)
Received: from Internal Mail-Server by MailRelay01 (envelope-from yaakov?s@rad.com) with AES128-SHA encrypted SMTP; 21 Feb 2013 09:08:31 +0200
Received: from EXRAD5.ad.rad.co.il ([192.114.24.28]) by EXRAD5.ad.rad.co.il ([192.114.24.28]) with mapi id 14.02.0298.004; Thu, 21 Feb 2013 09:06:40 +0200
From: Yaakov Stein 
To: "MORTON JR., ALFRED (AL)" 
Thread-Topic: [pm-dir] Performance metrics doctors generated email -	new version.
Thread-Index: AQHOD3rskk7cOBCP20yAGfWoyAqM85iD4t1Q
Date: Thu, 21 Feb 2013 07:06:39 +0000
Message-ID: <07F7D7DED63154409F13298786A2ADC904C8C95E@EXRAD5.ad.rad.co.il>
References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com>  <512395D3.7000002@cisco.com>  <95067C434CE250468B77282634C96ED3229935B7@xmb-aln-x02.cisco.com>
In-Reply-To: <95067C434CE250468B77282634C96ED3229935B7@xmb-aln-x02.cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [207.232.33.112]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Commtouch-Refid: str=0001.0A0C0205.5125C781.006C,ss=1,fgs=0
Cc: "pm-dir@ietf.org" 
Subject: Re: [pm-dir] Performance metrics doctors generated email -	new	version.
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 21 Feb 2013 07:06:47 -0000

Sorry for perhaps not catching on,=20
but are you suggesting setting up a single account for all of the directora=
te
and using "Google Tasks" to aggregate the review statuses ?

While this could work, I think there are perhaps better options.

The simplest would be just maintaining a table with draft-name + reviewer +=
 status + date
on a wiki.

As a reviewer for several academic journals I know they have tools
(much too advanced for our present needs)
that select the most appropriate reviewer(s), send them emails,
receive their acceptance/rejection of the task,=20
send email reminders when the due date approaches,=20
input the review, etc.

Y(J)S=20

-----Original Message-----
From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf Of=
 Carlos Pignataro (cpignata)
Sent: Wednesday, February 20, 2013 17:00
To: MORTON JR., ALFRED (AL)
Cc: Benoit Claise (bclaise); pm-dir@ietf.org; Paul Aitken (paitken); Alan D=
eKok
Subject: Re: [pm-dir] Performance metrics doctors generated email - new ver=
sion.


On Feb 20, 2013, at 9:36 AM, "MORTON JR., ALFRED (AL)"  w=
rote:

> Ok Benoit,=20
> I have a version of the doc going, but probably won't finish
> till after the revised deadline expires...
>=20
> Before going further, it would be useful to know if anyone
> in the directorate objects to using or establishing a gmail
> account in order to register their review assignments and status.
>=20

No objection. I think it's a great solution.

Thanks,

-- Carlos.

> regards,
> Al
>=20

From acmorton@att.com  Thu Feb 21 04:36:25 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C4C7421F8DA2 for ; Thu, 21 Feb 2013 04:36:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.484
X-Spam-Level: 
X-Spam-Status: No, score=-106.484 tagged_above=-999 required=5 tests=[AWL=0.115, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9AlScWuhSO1w for ; Thu, 21 Feb 2013 04:36:25 -0800 (PST)
Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id ECE8021F8BE0 for ; Thu, 21 Feb 2013 04:36:24 -0800 (PST)
Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id DE3FD120CA6; Thu, 21 Feb 2013 07:38:42 -0500 (EST)
Received: from njfpsrvexg7.research.att.com (njfpsrvexg7.research.att.com [135.207.177.33]) by mail-blue.research.att.com (Postfix) with ESMTP id 9F06DF031E; Thu, 21 Feb 2013 07:36:24 -0500 (EST)
Received: from njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299]) by njfpsrvexg7.research.att.com ([fe80::3598:75fe:b400:9299%11]) with mapi; Thu, 21 Feb 2013 07:36:24 -0500
From: "MORTON JR., ALFRED  (AL)" 
To: Yaakov Stein 
Date: Thu, 21 Feb 2013 07:36:22 -0500
Thread-Topic: [pm-dir] Performance metrics doctors generated email -	new version.
Thread-Index: AQHOD3rskk7cOBCP20yAGfWoyAqM85iD4t1QgABdGmA=
Message-ID: 
References: <20130204115318.GA9086@sweet-brew-5.cisco.com> <510FA460.3060102@cisco.com> <511E17E4.1020809@cisco.com>  <512395D3.7000002@cisco.com>  <95067C434CE250468B77282634C96ED3229935B7@xmb-aln-x02.cisco.com> <07F7D7DED63154409F13298786A2ADC904C8C95E@EXRAD5.ad.rad.co.il>
In-Reply-To: <07F7D7DED63154409F13298786A2ADC904C8C95E@EXRAD5.ad.rad.co.il>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "pm-dir@ietf.org" 
Subject: Re: [pm-dir] Performance metrics doctors generated email -	new	version.
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Thu, 21 Feb 2013 12:36:26 -0000

Not Google Tasks (which I haven't investigated),
just a Google doc with the table:
draft-name + reviewer + status + date

Which all reviewers can edit WYSIWYG
I'll own the doc and share it with the everyone in the=20
directorate (who actually does reviews :-) ).

And, that doc won't be ready till after the draft deadline!

Al

> -----Original Message-----
> From: Yaakov Stein [mailto:yaakov_s@rad.com]
> Sent: Thursday, February 21, 2013 2:07 AM
> To: MORTON JR., ALFRED (AL)
> Cc: pm-dir@ietf.org
> Subject: RE: [pm-dir] Performance metrics doctors generated email - new
> version.
>=20
> Sorry for perhaps not catching on,
> but are you suggesting setting up a single account for all of the
> directorate
> and using "Google Tasks" to aggregate the review statuses ?
>=20
> While this could work, I think there are perhaps better options.
>=20
> The simplest would be just maintaining a table with draft-name + reviewer
> + status + date
> on a wiki.
>=20
> As a reviewer for several academic journals I know they have tools
> (much too advanced for our present needs)
> that select the most appropriate reviewer(s), send them emails,
> receive their acceptance/rejection of the task,
> send email reminders when the due date approaches,
> input the review, etc.
>=20
> Y(J)S
>=20
> -----Original Message-----
> From: pm-dir-bounces@ietf.org [mailto:pm-dir-bounces@ietf.org] On Behalf
> Of Carlos Pignataro (cpignata)
> Sent: Wednesday, February 20, 2013 17:00
> To: MORTON JR., ALFRED (AL)
> Cc: Benoit Claise (bclaise); pm-dir@ietf.org; Paul Aitken (paitken); Alan
> DeKok
> Subject: Re: [pm-dir] Performance metrics doctors generated email - new
> version.
>=20
>=20
> On Feb 20, 2013, at 9:36 AM, "MORTON JR., ALFRED (AL)" 
> wrote:
>=20
> > Ok Benoit,
> > I have a version of the doc going, but probably won't finish
> > till after the revised deadline expires...
> >
> > Before going further, it would be useful to know if anyone
> > in the directorate objects to using or establishing a gmail
> > account in order to register their review assignments and status.
> >
>=20
> No objection. I think it's a great solution.
>=20
> Thanks,
>=20
> -- Carlos.
>=20
> > regards,
> > Al
> >

From bclaise@cisco.com  Sun Feb 24 14:06:34 2013
Return-Path: 
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0452B21F910B for ; Sun, 24 Feb 2013 14:06:34 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.545
X-Spam-Level: 
X-Spam-Status: No, score=-10.545 tagged_above=-999 required=5 tests=[AWL=0.054, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xPmklaiFYwGA for ; Sun, 24 Feb 2013 14:06:33 -0800 (PST)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 37AA921F911F for ; Sun, 24 Feb 2013 14:06:33 -0800 (PST)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1OM6WsZ023698 for ; Sun, 24 Feb 2013 23:06:32 +0100 (CET)
Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r1OM62r4001106 for ; Sun, 24 Feb 2013 23:06:12 +0100 (CET)
Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r1OM62xQ024343 for pm-dir@ietf.org; Sun, 24 Feb 2013 23:06:02 +0100 (CET)
Date: Sun, 24 Feb 2013 23:06:02 +0100
From: Benoit Claise 
To: pm-dir@ietf.org
Message-ID: <20130224220602.GA24339@sweet-brew-5.cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.3i
Subject: [pm-dir] Performance metrics doctors generated email
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list 
List-Unsubscribe: , 
List-Archive: 
List-Post: 
List-Help: 
List-Subscribe: , 
X-List-Received-Date: Sun, 24 Feb 2013 22:06:34 -0000

Dear all,

This is an automatically generated email.  
It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference.
It also lists all the IETF internet-drafts that contain "performance metric".

Regards, Benoit

===========================================================

Normative References
--------------------
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active	
    
Informative References
----------------------
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-concsec-03             Active	
draft-ietf-xrblock-rtcp-xr-decodability-07        In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-discard-11             Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active	
draft-ietf-xrblock-rtcp-xr-jb-08                  Active	
draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active	
draft-ietf-xrblock-rtcp-xr-qoe-03                 Active	

drafts containing performance metric
------------------------------------
draft-ietf-alto-deployments-05                    Active	
draft-ietf-alto-protocol-13                       In IESG processing - ID Tracker state 	
draft-ietf-bmwg-ca-bench-meth-04                  Active	
draft-ietf-ippm-rate-problem-02                   Active	
draft-ietf-ippm-testplan-rfc2680-02               Active	
draft-ietf-karp-threats-reqs-07                   In IESG processing - ID Tracker state 	
draft-ietf-manet-olsrv2-mib-05                    Active	
draft-ietf-manet-smf-mib-06                       Active	
draft-ietf-nvo3-framework-02                      Active	
draft-ietf-opsawg-oam-overview-08                 In IESG processing - ID Tracker state 	
draft-ietf-ppsp-peer-protocol-06                  Active	
draft-ietf-rtcweb-rtp-usage-05                    Active	
draft-ietf-rtgwg-cl-framework-02                  Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-10   In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-burst-gap-loss-08      In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-concsec-03             Active	
draft-ietf-xrblock-rtcp-xr-decodability-07        In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-discard-11             Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-05 Active	
draft-ietf-xrblock-rtcp-xr-jb-08                  Active	
draft-ietf-xrblock-rtcp-xr-loss-conceal-04        Active	
draft-ietf-xrblock-rtcp-xr-qoe-03                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-09        In IESG processing - ID Tracker state 	
draft-ietf-xrblock-rtcp-xr-synchronization-02     Active