From bclaise@cisco.com  Sun Aug  4 14:06:42 2013
Return-Path: <bclaise@cisco.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F16CF21E8092 for <pm-dir@ietfa.amsl.com>; Sun,  4 Aug 2013 14:06:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.571
X-Spam-Level: 
X-Spam-Status: No, score=-10.571 tagged_above=-999 required=5 tests=[AWL=0.028, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bxLM+yR9Bjr9 for <pm-dir@ietfa.amsl.com>; Sun,  4 Aug 2013 14:06:37 -0700 (PDT)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 5147E21E8093 for <pm-dir@ietf.org>; Sun,  4 Aug 2013 14:06:33 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r74L6V5G002839 for <pm-dir@ietf.org>; Sun, 4 Aug 2013 23:06:31 +0200 (CEST)
Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r74L6DUs019016 for <pm-dir@ietf.org>; Sun, 4 Aug 2013 23:06:23 +0200 (CEST)
Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r74L6AGm008919 for pm-dir@ietf.org; Sun, 4 Aug 2013 23:06:10 +0200 (CEST)
Date: Sun, 4 Aug 2013 23:06:10 +0200
From: Benoit Claise <bclaise@cisco.com>
To: pm-dir@ietf.org
Message-ID: <20130804210610.GA8917@sweet-brew-5.cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.3i
Subject: [pm-dir] Performance metrics doctors generated email
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 04 Aug 2013 21:06:43 -0000

Dear all,

This is an automatically generated email.  
It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference.
It also lists all the IETF internet-drafts that contain "performance metric".

Regards, Benoit

===========================================================

Normative References
--------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
    
Informative References
----------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-decodability-12        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

drafts containing performance metric
------------------------------------
draft-ietf-alto-deployments-07                    Active	
draft-ietf-bmwg-ca-bench-meth-04                  Active	
draft-ietf-cdni-footprint-capabilities-semantics-00Active	
draft-ietf-ippm-2330-update-00                    Active	
draft-ietf-ippm-lmap-path-00                      Active	
draft-ietf-ippm-model-based-metrics-00            Active	
draft-ietf-ippm-rate-problem-03                   Active	
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-manet-smf-mib-07                       In IESG processing - ID Tracker state <AD Evaluation::Revised I-D Needed>	
draft-ietf-nvo3-framework-03                      Active	
draft-ietf-opsawg-oam-overview-09                 In IESG processing - ID Tracker state <AD Evaluation>	
draft-ietf-ppsp-base-tracker-protocol-01          Active	
draft-ietf-ppsp-peer-protocol-07                  Active	
draft-ietf-rtcweb-rtp-usage-07                    Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-decodability-12        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

From bclaise@cisco.com  Sun Aug 11 14:14:25 2013
Return-Path: <bclaise@cisco.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7E5D411E8122 for <pm-dir@ietfa.amsl.com>; Sun, 11 Aug 2013 14:14:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.578
X-Spam-Level: 
X-Spam-Status: No, score=-10.578 tagged_above=-999 required=5 tests=[AWL=0.021, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M1bAOx71gfQa for <pm-dir@ietfa.amsl.com>; Sun, 11 Aug 2013 14:14:19 -0700 (PDT)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 0638A21F8FB4 for <pm-dir@ietf.org>; Sun, 11 Aug 2013 14:06:27 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r7BL6Rok027397 for <pm-dir@ietf.org>; Sun, 11 Aug 2013 23:06:27 +0200 (CEST)
Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r7BL6BrY023496 for <pm-dir@ietf.org>; Sun, 11 Aug 2013 23:06:21 +0200 (CEST)
Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r7BL69MM021905 for pm-dir@ietf.org; Sun, 11 Aug 2013 23:06:09 +0200 (CEST)
Date: Sun, 11 Aug 2013 23:06:09 +0200
From: Benoit Claise <bclaise@cisco.com>
To: pm-dir@ietf.org
Message-ID: <20130811210609.GA21903@sweet-brew-5.cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.3i
Subject: [pm-dir] Performance metrics doctors generated email
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 11 Aug 2013 21:14:25 -0000

Dear all,

This is an automatically generated email.  
It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference.
It also lists all the IETF internet-drafts that contain "performance metric".

Regards, Benoit

===========================================================

Normative References
--------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
    
Informative References
----------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-decodability-12        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

drafts containing performance metric
------------------------------------
draft-ietf-alto-deployments-07                    Active	
draft-ietf-cdni-footprint-capabilities-semantics-00Active	
draft-ietf-ippm-2330-update-00                    Active	
draft-ietf-ippm-lmap-path-00                      Active	
draft-ietf-ippm-model-based-metrics-00            Active	
draft-ietf-ippm-rate-problem-03                   Active	
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-manet-smf-mib-07                       In IESG processing - ID Tracker state <AD Evaluation::Revised I-D Needed>	
draft-ietf-nvo3-framework-03                      Active	
draft-ietf-opsawg-oam-overview-09                 In IESG processing - ID Tracker state <AD Evaluation>	
draft-ietf-ppsp-base-tracker-protocol-01          Active	
draft-ietf-ppsp-peer-protocol-07                  Active	
draft-ietf-rtcweb-rtp-usage-07                    Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-decodability-12        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

From janovak@cisco.com  Tue Aug 13 04:15:58 2013
Return-Path: <janovak@cisco.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CB74A21F871B for <pm-dir@ietfa.amsl.com>; Tue, 13 Aug 2013 04:15:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level: 
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KV-UOv1YXg8e for <pm-dir@ietfa.amsl.com>; Tue, 13 Aug 2013 04:15:53 -0700 (PDT)
Received: from rcdn-iport-3.cisco.com (rcdn-iport-3.cisco.com [173.37.86.74]) by ietfa.amsl.com (Postfix) with ESMTP id A83CC21F86DD for <pm-dir@ietf.org>; Tue, 13 Aug 2013 04:15:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=495; q=dns/txt; s=iport; t=1376392538; x=1377602138; h=from:to:cc:subject:date:message-id: content-transfer-encoding:mime-version; bh=1AkARV200Usb7EUddtIWLqtQLYHoCQwA/CO+9WVcCaA=; b=d2EUMByInQh3Ao4iD6k+R9hyan/l7aSPFacx3NetbTRp61uJH4RKKmlB 51TKZgz116PDm0ZazvnqNwQUUHbaiq6aYHhO9ADQP6b1CpGIgUGulZxB7 cpScM0pojBRLd0qwWqviBrRTeuMavox72tbvRmO3UMpTdZAkunVVP0evR M=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AmgGAFQUClKtJXG8/2dsb2JhbABbgwaBBb5bgR8WbQeCJgEEOj8SASoUQiYBBA4NiAi3XJALMYMidgOpNYMbgio
X-IronPort-AV: E=Sophos;i="4.89,868,1367971200"; d="scan'208";a="246695368"
Received: from rcdn-core2-1.cisco.com ([173.37.113.188]) by rcdn-iport-3.cisco.com with ESMTP; 13 Aug 2013 11:15:38 +0000
Received: from xhc-aln-x11.cisco.com (xhc-aln-x11.cisco.com [173.36.12.85]) by rcdn-core2-1.cisco.com (8.14.5/8.14.5) with ESMTP id r7DBFc1N030333 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 13 Aug 2013 11:15:38 GMT
Received: from xmb-aln-x03.cisco.com ([169.254.6.31]) by xhc-aln-x11.cisco.com ([173.36.12.85]) with mapi id 14.02.0318.004; Tue, 13 Aug 2013 06:15:37 -0500
From: "Jan Novak (janovak)" <janovak@cisco.com>
To: "pm-dir@ietf.org" <pm-dir@ietf.org>
Thread-Topic: RFC6390 Review of draft-ietf-ppsp-peer-protocol-07
Thread-Index: Ac6YFmYrZ6bZ5+zRRbav+G7FP7Ve3Q==
Date: Tue, 13 Aug 2013 11:15:37 +0000
Message-ID: <F45DBC0B6261374F8F8D3AF620413DFED93F5B@xmb-aln-x03.cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.147.1.16]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: "MORTON JR., ALFRED C \(AL\)" <acmorton@att.com>
Subject: [pm-dir] RFC6390 Review of draft-ietf-ppsp-peer-protocol-07
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Aug 2013 11:15:58 -0000

Hi Al,

I have read through draft-ietf-ppsp-peer-protocol-07 and the draft
Itself does not define any performance metrics so RFC6390 isn't
Applicable.
The script must have matched on the "Performance Management" section
12.2.5 which only references other work the application performance
Area (RFC3729 and 4150) and a future PPSP work.

Jan

The climate of Edinburgh is such that the weak succumb young, and the stron=
g envy them ....
                                 Dr. Johnson




From acmorton@att.com  Tue Aug 13 04:27:58 2013
Return-Path: <acmorton@att.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8DBE511E8119 for <pm-dir@ietfa.amsl.com>; Tue, 13 Aug 2013 04:27:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -106.599
X-Spam-Level: 
X-Spam-Status: No, score=-106.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BLOYrA7h-Tvd for <pm-dir@ietfa.amsl.com>; Tue, 13 Aug 2013 04:27:52 -0700 (PDT)
Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id CEFD911E80EF for <pm-dir@ietf.org>; Tue, 13 Aug 2013 04:27:51 -0700 (PDT)
Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 35A42120595; Tue, 13 Aug 2013 07:27:50 -0400 (EDT)
Received: from njfpsrvexg2.research.att.com (njfpsrvexg2.research.att.com [135.207.177.29]) by mail-green.research.att.com (Postfix) with ESMTP id C9F0FE2533; Tue, 13 Aug 2013 07:27:29 -0400 (EDT)
Received: from concierge.research.att.com (135.207.179.20) by njfpsrvexg2.research.att.com (135.207.177.29) with Microsoft SMTP Server (TLS) id 8.3.298.1; Tue, 13 Aug 2013 07:27:50 -0400
Received: from njfpsrvexg7.research.att.com ([fe80:0000:0000:0000:3598:75fe:180.0.146.153]) by concierge.research.att.com ([135.207.24.83]) with mapi; Tue, 13 Aug 2013 07:27:50 -0400
From: "MORTON JR., ALFRED C (AL)" <acmorton@att.com>
To: "Jan Novak (janovak)" <janovak@cisco.com>, "pm-dir@ietf.org" <pm-dir@ietf.org>
Date: Tue, 13 Aug 2013 07:27:49 -0400
Thread-Topic: RFC6390 Review of draft-ietf-ppsp-peer-protocol-07
Thread-Index: Ac6YFmYrZ6bZ5+zRRbav+G7FP7Ve3QAAaOBg
Message-ID: <F1312FAF1A1E624DA0972D1C9A91379A1CA52C98E5@njfpsrvexg7.research.att.com>
References: <F45DBC0B6261374F8F8D3AF620413DFED93F5B@xmb-aln-x03.cisco.com>
In-Reply-To: <F45DBC0B6261374F8F8D3AF620413DFED93F5B@xmb-aln-x03.cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [pm-dir] RFC6390 Review of draft-ietf-ppsp-peer-protocol-07
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 13 Aug 2013 11:27:58 -0000

Thanks for making this known, Jan.
I'll update the status sheet,
Al


> -----Original Message-----
> From: Jan Novak (janovak) [mailto:janovak@cisco.com]
> Sent: Tuesday, August 13, 2013 7:16 AM
> To: pm-dir@ietf.org
> Cc: MORTON JR., ALFRED C (AL)
> Subject: RFC6390 Review of draft-ietf-ppsp-peer-protocol-07
>=20
> Hi Al,
>=20
> I have read through draft-ietf-ppsp-peer-protocol-07 and the draft
> Itself does not define any performance metrics so RFC6390 isn't
> Applicable.
> The script must have matched on the "Performance Management" section
> 12.2.5 which only references other work the application performance
> Area (RFC3729 and 4150) and a future PPSP work.
>=20
> Jan
>=20
> The climate of Edinburgh is such that the weak succumb young, and the
> strong envy them ....
>                                  Dr. Johnson
>=20
>=20


From bclaise@cisco.com  Sun Aug 18 14:06:49 2013
Return-Path: <bclaise@cisco.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3DDAF21F8F32 for <pm-dir@ietfa.amsl.com>; Sun, 18 Aug 2013 14:06:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.599
X-Spam-Level: 
X-Spam-Status: No, score=-10.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OmSn6+Vjwxf8 for <pm-dir@ietfa.amsl.com>; Sun, 18 Aug 2013 14:06:33 -0700 (PDT)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 9818E11E8156 for <pm-dir@ietf.org>; Sun, 18 Aug 2013 14:06:31 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r7IL6UV1011972 for <pm-dir@ietf.org>; Sun, 18 Aug 2013 23:06:30 +0200 (CEST)
Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r7IL6EBo005107 for <pm-dir@ietf.org>; Sun, 18 Aug 2013 23:06:24 +0200 (CEST)
Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r7IL6CFA000983 for pm-dir@ietf.org; Sun, 18 Aug 2013 23:06:12 +0200 (CEST)
Date: Sun, 18 Aug 2013 23:06:12 +0200
From: Benoit Claise <bclaise@cisco.com>
To: pm-dir@ietf.org
Message-ID: <20130818210612.GA981@sweet-brew-5.cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.3i
Subject: [pm-dir] Performance metrics doctors generated email
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 18 Aug 2013 21:06:49 -0000

Dear all,

This is an automatically generated email.  
It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference.
It also lists all the IETF internet-drafts that contain "performance metric".

Regards, Benoit

===========================================================

Normative References
--------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
    
Informative References
----------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-decodability-12        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

drafts containing performance metric
------------------------------------
draft-ietf-alto-deployments-07                    Active	
draft-ietf-cdni-footprint-capabilities-semantics-00Active	
draft-ietf-ippm-2330-update-00                    Active	
draft-ietf-ippm-lmap-path-00                      Active	
draft-ietf-ippm-model-based-metrics-00            Active	
draft-ietf-ippm-rate-problem-03                   Active	
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-manet-smf-mib-07                       In IESG processing - ID Tracker state <AD Evaluation::Revised I-D Needed>	
draft-ietf-nvo3-framework-03                      Active	
draft-ietf-opsawg-oam-overview-09                 In IESG processing - ID Tracker state <AD Evaluation>	
draft-ietf-ppsp-base-tracker-protocol-01          Active	
draft-ietf-ppsp-peer-protocol-07                  Active	
draft-ietf-rtcweb-rtp-usage-07                    Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-decodability-12        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

From bclaise@cisco.com  Sun Aug 25 14:17:45 2013
Return-Path: <bclaise@cisco.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 34B9411E80AD for <pm-dir@ietfa.amsl.com>; Sun, 25 Aug 2013 14:17:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.531
X-Spam-Level: 
X-Spam-Status: No, score=-10.531 tagged_above=-999 required=5 tests=[AWL=0.068, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TlAL4JvzVnCA for <pm-dir@ietfa.amsl.com>; Sun, 25 Aug 2013 14:17:39 -0700 (PDT)
Received: from av-tac-bru.cisco.com (weird-brew.cisco.com [144.254.15.118]) by ietfa.amsl.com (Postfix) with ESMTP id 3DDF321F9FCF for <pm-dir@ietf.org>; Sun, 25 Aug 2013 14:17:34 -0700 (PDT)
X-TACSUNS: Virus Scanned
Received: from strange-brew.cisco.com (localhost.cisco.com [127.0.0.1]) by av-tac-bru.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r7PLHUFe025280 for <pm-dir@ietf.org>; Sun, 25 Aug 2013 23:17:32 +0200 (CEST)
Received: from sweet-brew-5.cisco.com (sweet-brew-5.cisco.com [144.254.10.206]) by strange-brew.cisco.com (8.13.8+Sun/8.13.8) with ESMTP id r7PL6DVu023313 for <pm-dir@ietf.org>; Sun, 25 Aug 2013 23:06:23 +0200 (CEST)
Received: (from bclaise@localhost) by sweet-brew-5.cisco.com (8.13.8+Sun/8.13.6/Submit) id r7PL6Cm1011818 for pm-dir@ietf.org; Sun, 25 Aug 2013 23:06:12 +0200 (CEST)
Date: Sun, 25 Aug 2013 23:06:12 +0200
From: Benoit Claise <bclaise@cisco.com>
To: pm-dir@ietf.org
Message-ID: <20130825210612.GA11815@sweet-brew-5.cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.2.3i
Subject: [pm-dir] Performance metrics doctors generated email
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 25 Aug 2013 21:17:45 -0000

Dear all,

This is an automatically generated email.  
It lists the IETF internet-drafts that reference the PMOL RFC 6390, as a normative or informative reference.
It also lists all the IETF internet-drafts that contain "performance metric".

Regards, Benoit

===========================================================

Normative References
--------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
    
Informative References
----------------------
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

drafts containing performance metric
------------------------------------
draft-ietf-alto-deployments-07                    Active	
draft-ietf-cdni-footprint-capabilities-semantics-00Active	
draft-ietf-ippm-2330-update-00                    Active	
draft-ietf-ippm-lmap-path-00                      Active	
draft-ietf-ippm-model-based-metrics-00            Active	
draft-ietf-ippm-rate-problem-03                   Active	
draft-ietf-ippm-testplan-rfc2680-03               Active	
draft-ietf-manet-smf-mib-07                       In IESG processing - ID Tracker state <AD Evaluation::Revised I-D Needed>	
draft-ietf-nvo3-framework-03                      Active	
draft-ietf-opsawg-oam-overview-09                 In IESG processing - ID Tracker state <AD Evaluation>	
draft-ietf-ppsp-base-tracker-protocol-01          Active	
draft-ietf-ppsp-peer-protocol-07                  Active	
draft-ietf-rtcweb-rtp-usage-07                    Active	
draft-ietf-xrblock-rtcp-xr-burst-gap-discard-14   In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-15             In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-discard-rle-metrics-06 In IESG processing - ID Tracker state <IESG Evaluation::Revised I-D Needed>	
draft-ietf-xrblock-rtcp-xr-jb-14                  In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-loss-conceal-08        Active	
draft-ietf-xrblock-rtcp-xr-qoe-10                 Active	
draft-ietf-xrblock-rtcp-xr-summary-stat-11        In IESG processing - ID Tracker state <RFC Ed Queue>	
draft-ietf-xrblock-rtcp-xr-synchronization-06     Active	

From acmorton@att.com  Mon Aug 26 07:12:59 2013
Return-Path: <acmorton@att.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1147311E8197 for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 07:12:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -108.599
X-Spam-Level: 
X-Spam-Status: No, score=-108.599 tagged_above=-999 required=5 tests=[AWL=-2.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UCsFDG16DHm9 for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 07:12:54 -0700 (PDT)
Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id D433E11E81A5 for <pm-dir@ietf.org>; Mon, 26 Aug 2013 07:12:53 -0700 (PDT)
Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-pink.research.att.com (Postfix) with ESMTP id BAF0112067A; Mon, 26 Aug 2013 10:12:50 -0400 (EDT)
Received: from njfpsrvexg8.research.att.com (njfpsrvexg8.research.att.com [135.207.178.36]) by mail-blue.research.att.com (Postfix) with ESMTP id 4EF18F036E; Mon, 26 Aug 2013 10:12:53 -0400 (EDT)
Received: from NJFPSRVEXG8.research.att.com ([fe80::a44a:8177:9a5d:ac00]) by njfpsrvexg8.research.att.com ([fe80::a44a:8177:9a5d:ac00%15]) with mapi; Mon, 26 Aug 2013 10:12:53 -0400
From: "MORTON JR., ALFRED C (AL)" <acmorton@att.com>
To: "Romascanu, Dan (Dan)" <dromasca@avaya.com>, "pm-dir@ietf.org" <pm-dir@ietf.org>
Date: Mon, 26 Aug 2013 10:12:51 -0400
Thread-Topic: request for an early PMDIR review
Thread-Index: Ac6O6XPW5Zq72zT7QLWIIwOJ4bPZYgTb2ZBg
Message-ID: <2845723087023D4CB5114223779FA9C83BEF44ED@njfpsrvexg8.research.att.com>
References: <9904FB1B0159DA42B0B887B7FA8119CA1289486E@AZ-FFEXMB04.global.avaya.com>
In-Reply-To: <9904FB1B0159DA42B0B887B7FA8119CA1289486E@AZ-FFEXMB04.global.avaya.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: Roni Even <ron.even.tlv@gmail.com>, Shida Schubert <shida@ntt-at.com>, Varun Singh <varun@comnet.tkk.fi>, "Huangyihong \(Rachel\)" <rachel.huang@huawei.com>
Subject: Re: [pm-dir] request for an early PMDIR review
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Aug 2013 14:12:59 -0000

Hi Dan,

As promised, here's an early pm-dir review of the draft:
https://datatracker.ietf.org/doc/draft-huang-xrblock-rtcweb-rtcp-xr-metrics=
/

Since the purpose of this draft is to make recommendations=20
for measurements to satisfy the WebRTC requirement cited in the draft:

   "F38    The browser MUST be able to collect statistics, related to
   the transport of audio and video between peers, needed to estimate
   quality of service."

"Statistics" seemed a bit vague to me, especially in a requirement.
One supporting reference is:

   [RTCWEB-STAT] Alvestrand, H., "A Registry for WebRTC statistics
                 identifiers", September 24, 2012.

I took a look at Harald's draft. "RTCWeb Media Statistics" are indeed the=20
result of some performance measurement, and the registry he proposes=20
appears to assume that all needed metric definition details can be
found in a specification or supplied by the expert who prepares the
registry entry. There would likely be overlap between the
"RTCWeb Media Statistics" registry and other registries containing
performance metrics.

There's a *missing* aspect of the WebRTC requirement F38:
Specification of the *minimum* set of metrics that MUST be provided
to accomplish the stated goal: estimate quality of service
(and not quality of experience!).

It seems to me that draft-huang-xrblock-rtcweb-rtcp-xr-metrics
*could* be revised to fulfill that role. Otherwise, the intent and
goal of F38 will never be realized because implementers can make
different choices of statistics, and possibly include a set which
is insufficient. In addition, the draft currently takes the position=20
that some metrics are needed for network performance diagnosis.
This is a valuable set of metrics too, but they need to be distinguished
from the minimum set of metrics needed to estimate QoS and=20
justified through explanation of their diagnostic power=20
(as has been done in some cases already in the draft).
Again, emphasis should be on a *minimum* set of diagnostic metrics,
and which applications and common circumstances.

Some more specific comments follow:

3.2.1 Retransmitted Packet Count Metric
I'm surprised to see this suggested, especially as the first in the memo.
In true Real-time conversational applications, there's little opportunity
for retransmission to be effective. The draft says:
   In low delay networks with low
   loss rates, retransmissions have great value without incurring
   additional complexity.
If the loss ratio is low, it seems retransmission is not very valuable,
and low delay networks are a very restricted set. I don't see what=20
additional diagnostic value this metric has, beyond the other loss metrics.
How widely is the NACK capability deployed and used?
Note: there could be link-layer retransmissions below RTP-layer,
but these are not visible above the link layer, except as=20
reordered packets.


3.2.3 Loss, Discard and Duplicate Packet Count Metric
I agree that Loss and Discard are valuable for QoS and diagnostic
purposes, but in my experience Duplicate packets are rare and have
little effect on QoS (they are usually discarded silently, but they
should not appear in the discard counts).


3.2.5 Run Length Encoded Metrics for Loss, Discard and Post-repair

   Run-length encoding uses a bit vector to encode information about the
   packet. Each bit in the vector represents a packet and depending on
   the signaled metric it defines if the packet was lost, duplicated,
   discarded, or repaired. An endpoint typically uses the run length
   encoding to accurately communicate the status of each packet in the
   interval to the other endpoint.
This seems like an enormous amount of data to convey between
the receiver and sender, a fairly complete record of transmission.
It might be a diagnostic capability turned-on in some specific=20
troubleshooting cases, but I don't see RLE as part of the=20
QoS estimation metrics set.

It's fine to include Jitter and De-jitter buffer metrics in the QoS=20
estimation set. Others could be discussed, but it's probably good to
stop here for now.

hope this helps,
Al
PS I'm switching to other topics, and will probably delay any responses
on this thread for a couple of weeks.


> -----Original Message-----
> From: Romascanu, Dan (Dan) [mailto:dromasca@avaya.com]
> Sent: Thursday, August 01, 2013 3:01 PM
> To: MORTON JR., ALFRED C (AL)
> Cc: Huangyihong (Rachel); Varun Singh; Roni Even; Shida Schubert
> Subject: request for an early PMDIR review
>=20
> Hi Al,
>=20
> Following our discussion earlier today, I would kindly ask for an early
> PM-DIR review of https://datatracker.ietf.org/doc/draft-huang-xrblock-
> rtcweb-rtcp-xr-metrics/.
>=20
> Thanks and Regards,
>=20
> Dan
>=20
>=20
>=20


From varun@comnet.tkk.fi  Mon Aug 26 07:35:16 2013
Return-Path: <varun@comnet.tkk.fi>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 424A621F9B8C for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 07:35:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level: 
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k9L+ZAQD2T8R for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 07:35:09 -0700 (PDT)
Received: from smtp-out-01.aalto.fi (smtp-out-01.aalto.fi [130.233.228.120]) by ietfa.amsl.com (Postfix) with ESMTP id 8D79D21F9B92 for <pm-dir@ietf.org>; Mon, 26 Aug 2013 07:35:04 -0700 (PDT)
Received: from smtp-out-01.aalto.fi (localhost.localdomain [127.0.0.1]) by localhost (Email Security Appliance) with SMTP id 3FC4C11539F_21B6796B; Mon, 26 Aug 2013 14:35:02 +0000 (GMT)
Received: from smtp.netlab.hut.fi (luuri.netlab.hut.fi [130.233.154.177]) by smtp-out-01.aalto.fi (Sophos Email Appliance) with ESMTP id 1D27E1152FE_21B6795F; Mon, 26 Aug 2013 14:35:01 +0000 (GMT)
Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp.netlab.hut.fi (Postfix) with ESMTP id 04D7B1E125; Mon, 26 Aug 2013 17:35:01 +0300 (EEST)
X-Virus-Scanned: by amavisd-new at luuri.netlab.hut.fi
Received: from smtp.netlab.hut.fi ([127.0.0.1]) by localhost (luuri.netlab.hut.fi [127.0.0.1]) (amavisd-new, port 10024) with LMTP id D9UN8w-eZ25o; Mon, 26 Aug 2013 17:34:55 +0300 (EEST)
Received: from wireless-86-50-141-236.open.aalto.fi (wireless-86-50-141-236.open.aalto.fi [86.50.141.236]) by smtp.netlab.hut.fi (Postfix) with ESMTPSA id D0D5C1E01D; Mon, 26 Aug 2013 17:34:54 +0300 (EEST)
Mime-Version: 1.0 (Apple Message framework v1283)
Content-Type: text/plain; charset=us-ascii
From: Varun Singh <varun@comnet.tkk.fi>
In-Reply-To: <2845723087023D4CB5114223779FA9C83BEF44ED@njfpsrvexg8.research.att.com>
Date: Mon, 26 Aug 2013 17:34:56 +0300
Content-Transfer-Encoding: quoted-printable
Message-Id: <01F950D6-F3CA-4C6A-826A-DD4F86AB7EFD@comnet.tkk.fi>
References: <9904FB1B0159DA42B0B887B7FA8119CA1289486E@AZ-FFEXMB04.global.avaya.com> <2845723087023D4CB5114223779FA9C83BEF44ED@njfpsrvexg8.research.att.com>
To: "MORTON JR., ALFRED C (AL)" <acmorton@att.com>
X-Mailer: Apple Mail (2.1283)
X-Mailman-Approved-At: Mon, 26 Aug 2013 07:40:48 -0700
Cc: Roni Even <ron.even.tlv@gmail.com>, "Romascanu, Dan \(Dan\)" <dromasca@avaya.com>, "pm-dir@ietf.org" <pm-dir@ietf.org>, "Huangyihong \(Rachel\)" <rachel.huang@huawei.com>, Shida Schubert <shida@ntt-at.com>
Subject: Re: [pm-dir] request for an early PMDIR review
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Aug 2013 14:35:16 -0000

Hi Al,

Thanks for the review. Comments inline.

Cheers,
Varun

On Aug 26, 2013, at 5:12 PM, MORTON JR., ALFRED C (AL) wrote:

> Hi Dan,
>=20
> As promised, here's an early pm-dir review of the draft:
> =
https://datatracker.ietf.org/doc/draft-huang-xrblock-rtcweb-rtcp-xr-metric=
s/
>=20
> Since the purpose of this draft is to make recommendations=20
> for measurements to satisfy the WebRTC requirement cited in the draft:
>=20
>   "F38    The browser MUST be able to collect statistics, related to
>   the transport of audio and video between peers, needed to estimate
>   quality of service."
>=20
> "Statistics" seemed a bit vague to me, especially in a requirement.
> One supporting reference is:
>=20
>   [RTCWEB-STAT] Alvestrand, H., "A Registry for WebRTC statistics
>                 identifiers", September 24, 2012.
>=20
> I took a look at Harald's draft. "RTCWeb Media Statistics" are indeed =
the=20
> result of some performance measurement, and the registry he proposes=20=

> appears to assume that all needed metric definition details can be
> found in a specification or supplied by the expert who prepares the
> registry entry. There would likely be overlap between the
> "RTCWeb Media Statistics" registry and other registries containing
> performance metrics.
>=20

Our intention with the upcoming versions of the draft was to add to=20
the registry proposed by Harald.

> There's a *missing* aspect of the WebRTC requirement F38:
> Specification of the *minimum* set of metrics that MUST be provided
> to accomplish the stated goal: estimate quality of service
> (and not quality of experience!).
>=20
> It seems to me that draft-huang-xrblock-rtcweb-rtcp-xr-metrics
> *could* be revised to fulfill that role. Otherwise, the intent and
> goal of F38 will never be realized because implementers can make
> different choices of statistics, and possibly include a set which
> is insufficient. In addition, the draft currently takes the position=20=

> that some metrics are needed for network performance diagnosis.
> This is a valuable set of metrics too, but they need to be =
distinguished
> from the minimum set of metrics needed to estimate QoS and=20
> justified through explanation of their diagnostic power=20
> (as has been done in some cases already in the draft).
> Again, emphasis should be on a *minimum* set of diagnostic metrics,
> and which applications and common circumstances.
>=20

I agree we should define a minimum set of metrics for the next version,=20=

the current version of the draft was mainly to organize all the =
available=20
metrics/measures and to facilitate discussion. Would your suggestion=20
be to: just define the minimum set or keep the current discussion and =
then=20
define the minimum set?

> Some more specific comments follow:
>=20
> 3.2.1 Retransmitted Packet Count Metric
> I'm surprised to see this suggested, especially as the first in the =
memo.
> In true Real-time conversational applications, there's little =
opportunity
> for retransmission to be effective. The draft says:
>   In low delay networks with low
>   loss rates, retransmissions have great value without incurring
>   additional complexity.
> If the loss ratio is low, it seems retransmission is not very =
valuable,
> and low delay networks are a very restricted set. I don't see what=20
> additional diagnostic value this metric has, beyond the other loss =
metrics.
> How widely is the NACK capability deployed and used?
> Note: there could be link-layer retransmissions below RTP-layer,
> but these are not visible above the link layer, except as=20
> reordered packets.
>=20

Short answer, Chrome uses NACK or PLI (packet loss indication) =
extensively.=20
My measurements show that there are cases in low delay where the actual =
packet loss=20
is 10-20% and by using retx they are able to bring it down to <1%, these =
measurements=20
are basically within northern europe (end-to-end delay up to 50ms, =
AFAIK, the chrome
implementation tries to meet the 150ms one-way delay guarantees).

They also use FEC, which can sometimes be 10-20% of the overall bit =
rate. We try=20
to capture this in the post-repair metric in 3.2.5. Perhaps, we may have =
to define
a new Post-repair metric which is not RLE, but a measure of number of =
packets=20
recovered by using FEC.


>=20
> 3.2.3 Loss, Discard and Duplicate Packet Count Metric
> I agree that Loss and Discard are valuable for QoS and diagnostic
> purposes, but in my experience Duplicate packets are rare and have
> little effect on QoS (they are usually discarded silently, but they
> should not appear in the discard counts).
>=20
Ok, remove duplicate and keep the discard and loss metric?


> 3.2.5 Run Length Encoded Metrics for Loss, Discard and Post-repair
>=20
>   Run-length encoding uses a bit vector to encode information about =
the
>   packet. Each bit in the vector represents a packet and depending on
>   the signaled metric it defines if the packet was lost, duplicated,
>   discarded, or repaired. An endpoint typically uses the run length
>   encoding to accurately communicate the status of each packet in the
>   interval to the other endpoint.
> This seems like an enormous amount of data to convey between
> the receiver and sender, a fairly complete record of transmission.
> It might be a diagnostic capability turned-on in some specific=20
> troubleshooting cases, but I don't see RLE as part of the=20
> QoS estimation metrics set.
>=20

I agree, RLE may be an overkill for the general monitoring.

> It's fine to include Jitter and De-jitter buffer metrics in the QoS=20
> estimation set. Others could be discussed, but it's probably good to
> stop here for now.
>=20

Thanks again for the comments.

> hope this helps,
> Al
> PS I'm switching to other topics, and will probably delay any =
responses
> on this thread for a couple of weeks.
>=20
>=20
>> -----Original Message-----
>> From: Romascanu, Dan (Dan) [mailto:dromasca@avaya.com]
>> Sent: Thursday, August 01, 2013 3:01 PM
>> To: MORTON JR., ALFRED C (AL)
>> Cc: Huangyihong (Rachel); Varun Singh; Roni Even; Shida Schubert
>> Subject: request for an early PMDIR review
>>=20
>> Hi Al,
>>=20
>> Following our discussion earlier today, I would kindly ask for an =
early
>> PM-DIR review of =
https://datatracker.ietf.org/doc/draft-huang-xrblock-
>> rtcweb-rtcp-xr-metrics/.
>>=20
>> Thanks and Regards,
>>=20
>> Dan
>>=20
>>=20
>>=20


From acmorton@att.com  Mon Aug 26 07:53:39 2013
Return-Path: <acmorton@att.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1507911E81B3 for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 07:53:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -107.599
X-Spam-Level: 
X-Spam-Status: No, score=-107.599 tagged_above=-999 required=5 tests=[AWL=-1.000, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Iq25z-GKjyP6 for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 07:53:28 -0700 (PDT)
Received: from mail-pink.research.att.com (mail-pink.research.att.com [192.20.225.111]) by ietfa.amsl.com (Postfix) with ESMTP id 6808011E81AF for <pm-dir@ietf.org>; Mon, 26 Aug 2013 07:53:28 -0700 (PDT)
Received: from mail-green.research.att.com (unknown [135.207.178.10]) by mail-pink.research.att.com (Postfix) with ESMTP id 5DECE12070B; Mon, 26 Aug 2013 10:53:25 -0400 (EDT)
Received: from njfpsrvexg8.research.att.com (njfpsrvexg8.research.att.com [135.207.178.36]) by mail-green.research.att.com (Postfix) with ESMTP id 14F6CE2536; Mon, 26 Aug 2013 10:52:43 -0400 (EDT)
Received: from NJFPSRVEXG8.research.att.com ([fe80::a44a:8177:9a5d:ac00]) by njfpsrvexg8.research.att.com ([fe80::a44a:8177:9a5d:ac00%15]) with mapi; Mon, 26 Aug 2013 10:53:27 -0400
From: "MORTON JR., ALFRED C (AL)" <acmorton@att.com>
To: Varun Singh <varun@comnet.tkk.fi>
Date: Mon, 26 Aug 2013 10:53:26 -0400
Thread-Topic: request for an early PMDIR review
Thread-Index: Ac6iaX+Id5Xq71v2QRKtrGeq3IMkRgAANjkw
Message-ID: <2845723087023D4CB5114223779FA9C83BEF4523@njfpsrvexg8.research.att.com>
References: <9904FB1B0159DA42B0B887B7FA8119CA1289486E@AZ-FFEXMB04.global.avaya.com> <2845723087023D4CB5114223779FA9C83BEF44ED@njfpsrvexg8.research.att.com> <01F950D6-F3CA-4C6A-826A-DD4F86AB7EFD@comnet.tkk.fi>
In-Reply-To: <01F950D6-F3CA-4C6A-826A-DD4F86AB7EFD@comnet.tkk.fi>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Cc: Roni Even <ron.even.tlv@gmail.com>, "Romascanu, Dan \(Dan\)" <dromasca@avaya.com>, "pm-dir@ietf.org" <pm-dir@ietf.org>, "Huangyihong \(Rachel\)" <rachel.huang@huawei.com>, Shida Schubert <shida@ntt-at.com>
Subject: Re: [pm-dir] request for an early PMDIR review
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Aug 2013 14:53:39 -0000

Hi Varun,
my quick replies to your questions:

> -----Original Message-----
> From: Varun Singh [mailto:varun@comnet.tkk.fi]
...
> > Again, emphasis should be on a *minimum* set of diagnostic metrics,
> > and which applications and common circumstances.
> >
>=20
> I agree we should define a minimum set of metrics for the next version,
> the current version of the draft was mainly to organize all the available
> metrics/measures and to facilitate discussion. Would your suggestion
> be to: just define the minimum set or keep the current discussion and the=
n
> define the minimum set?

I suggest to just define the minimum set for QoS estimation and=20
another, overlapping set of metrics with diagnostic properties
in this draft.  The survey of possible metrics is less valuable, IMO.

>=20
> > Some more specific comments follow:
> >
> > 3.2.1 Retransmitted Packet Count Metric
...
> > How widely is the NACK capability deployed and used?
> > Note: there could be link-layer retransmissions below RTP-layer,
> > but these are not visible above the link layer, except as
> > reordered packets.
> >
>=20
> Short answer, Chrome uses NACK or PLI (packet loss indication)
> extensively.
> My measurements show that there are cases in low delay where the actual
> packet loss
> is 10-20% and by using retx they are able to bring it down to <1%, these
> measurements
> are basically within northern europe (end-to-end delay up to 50ms, AFAIK,
> the chrome
> implementation tries to meet the 150ms one-way delay guarantees).

Ok, I wouldn't have called 10-20% loss "low", and that's significant improv=
ement.
Incidentally, the 150ms one-way delay objective from G.114 includes
the hosts and codecs, basically user to user delay.

>=20
> They also use FEC, which can sometimes be 10-20% of the overall bit rate.
> We try
> to capture this in the post-repair metric in 3.2.5. Perhaps, we may have
> to define
> a new Post-repair metric which is not RLE, but a measure of number of
> packets
> recovered by using FEC.

Yes, that's very valuable to know: there is loss, but within the FEC capabi=
lities
to repair so there's little user impact beyond the FEC overhead and delay
due to block processing, so the trade-off is a good one.

>=20
>=20
> >
> > 3.2.3 Loss, Discard and Duplicate Packet Count Metric
> > I agree that Loss and Discard are valuable for QoS and diagnostic
> > purposes, but in my experience Duplicate packets are rare and have
> > little effect on QoS (they are usually discarded silently, but they
> > should not appear in the discard counts).
> >
> Ok, remove duplicate and keep the discard and loss metric?

I think so.

>=20
>=20
> > 3.2.5 Run Length Encoded Metrics for Loss, Discard and Post-repair
> >
...
> > It might be a diagnostic capability turned-on in some specific
> > troubleshooting cases, but I don't see RLE as part of the
> > QoS estimation metrics set.
> >
>=20
> I agree, RLE may be an overkill for the general monitoring.
>=20
Good, the really valuable info is about the repaired packets.

glad this was useful,
Al


From rachel.huang@huawei.com  Mon Aug 26 20:24:52 2013
Return-Path: <rachel.huang@huawei.com>
X-Original-To: pm-dir@ietfa.amsl.com
Delivered-To: pm-dir@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B47BE11E8139 for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 20:24:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.599
X-Spam-Level: 
X-Spam-Status: No, score=-6.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 79Owl9EXxf+3 for <pm-dir@ietfa.amsl.com>; Mon, 26 Aug 2013 20:24:48 -0700 (PDT)
Received: from lhrrgout.huawei.com (lhrrgout.huawei.com [194.213.3.17]) by ietfa.amsl.com (Postfix) with ESMTP id 309BD21F9BF0 for <pm-dir@ietf.org>; Mon, 26 Aug 2013 20:24:45 -0700 (PDT)
Received: from 172.18.7.190 (EHLO lhreml203-edg.china.huawei.com) ([172.18.7.190]) by lhrrg02-dlp.huawei.com (MOS 4.3.5-GA FastPath queued) with ESMTP id AUT36169; Tue, 27 Aug 2013 03:24:35 +0000 (GMT)
Received: from LHREML404-HUB.china.huawei.com (10.201.5.218) by lhreml203-edg.huawei.com (172.18.7.221) with Microsoft SMTP Server (TLS) id 14.1.323.7; Tue, 27 Aug 2013 04:24:27 +0100
Received: from NKGEML402-HUB.china.huawei.com (10.98.56.33) by lhreml404-hub.china.huawei.com (10.201.5.218) with Microsoft SMTP Server (TLS) id 14.1.323.7; Tue, 27 Aug 2013 04:24:34 +0100
Received: from NKGEML501-MBS.china.huawei.com ([169.254.2.96]) by nkgeml402-hub.china.huawei.com ([10.98.56.33]) with mapi id 14.01.0323.007; Tue, 27 Aug 2013 11:24:27 +0800
From: "Huangyihong (Rachel)" <rachel.huang@huawei.com>
To: "MORTON JR., ALFRED C (AL)" <acmorton@att.com>, "Romascanu, Dan (Dan)" <dromasca@avaya.com>, "pm-dir@ietf.org" <pm-dir@ietf.org>
Thread-Topic: request for an early PMDIR review
Thread-Index: Ac6O6XPW5Zq72zT7QLWIIwOJ4bPZYgTb2ZBgABzS5qA=
Date: Tue, 27 Aug 2013 03:24:26 +0000
Message-ID: <51E6A56BD6A85142B9D172C87FC3ABBB4589D8D6@nkgeml501-mbs.china.huawei.com>
References: <9904FB1B0159DA42B0B887B7FA8119CA1289486E@AZ-FFEXMB04.global.avaya.com> <2845723087023D4CB5114223779FA9C83BEF44ED@njfpsrvexg8.research.att.com>
In-Reply-To: <2845723087023D4CB5114223779FA9C83BEF44ED@njfpsrvexg8.research.att.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.138.41.104]
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-CFilter-Loop: Reflected
X-Mailman-Approved-At: Tue, 27 Aug 2013 07:20:23 -0700
Cc: Roni Even <ron.even.tlv@gmail.com>, Shida Schubert <shida@ntt-at.com>, Varun Singh <varun@comnet.tkk.fi>
Subject: Re: [pm-dir] request for an early PMDIR review
X-BeenThere: pm-dir@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Performance Metrics Directorate Discussion list <pm-dir.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/pm-dir>
List-Post: <mailto:pm-dir@ietf.org>
List-Help: <mailto:pm-dir-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pm-dir>, <mailto:pm-dir-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 27 Aug 2013 03:24:52 -0000

Hi Al,

Thank you for these valuable comments. Please see some additional replies i=
nline.

Best regards,
Rachel

-----Original Message-----
From: MORTON JR., ALFRED C (AL) [mailto:acmorton@att.com]=20
Sent: Monday, August 26, 2013 10:13 PM
To: Romascanu, Dan (Dan); pm-dir@ietf.org
Cc: Huangyihong (Rachel); Varun Singh; Roni Even; Shida Schubert
Subject: RE: request for an early PMDIR review

Hi Dan,

As promised, here's an early pm-dir review of the draft:
https://datatracker.ietf.org/doc/draft-huang-xrblock-rtcweb-rtcp-xr-metrics=
/

Since the purpose of this draft is to make recommendations=20
for measurements to satisfy the WebRTC requirement cited in the draft:

   "F38    The browser MUST be able to collect statistics, related to
   the transport of audio and video between peers, needed to estimate
   quality of service."

"Statistics" seemed a bit vague to me, especially in a requirement.
One supporting reference is:

   [RTCWEB-STAT] Alvestrand, H., "A Registry for WebRTC statistics
                 identifiers", September 24, 2012.

I took a look at Harald's draft. "RTCWeb Media Statistics" are indeed the=20
result of some performance measurement, and the registry he proposes=20
appears to assume that all needed metric definition details can be
found in a specification or supplied by the expert who prepares the
registry entry. There would likely be overlap between the
"RTCWeb Media Statistics" registry and other registries containing
performance metrics.

There's a *missing* aspect of the WebRTC requirement F38:
Specification of the *minimum* set of metrics that MUST be provided
to accomplish the stated goal: estimate quality of service
(and not quality of experience!).

It seems to me that draft-huang-xrblock-rtcweb-rtcp-xr-metrics
*could* be revised to fulfill that role. Otherwise, the intent and
goal of F38 will never be realized because implementers can make
different choices of statistics, and possibly include a set which
is insufficient. In addition, the draft currently takes the position=20
that some metrics are needed for network performance diagnosis.
This is a valuable set of metrics too, but they need to be distinguished
from the minimum set of metrics needed to estimate QoS and=20
justified through explanation of their diagnostic power=20
(as has been done in some cases already in the draft).
Again, emphasis should be on a *minimum* set of diagnostic metrics,
and which applications and common circumstances.

[Rachel]: Are you suggestion that we should focus the metrics evaluating Qo=
S while remove the QoE related ones? Actually, the reason why we consider Q=
oE metrics (such as frame impairment metrics) is because they are useful to=
 diagnostic why media quality degrades. We think it's important because peo=
ple care about the quality. But I agree with you that they're not common en=
ough to all applications. It's fine to take them out.

Some more specific comments follow:

3.2.1 Retransmitted Packet Count Metric
I'm surprised to see this suggested, especially as the first in the memo.
In true Real-time conversational applications, there's little opportunity
for retransmission to be effective. The draft says:
   In low delay networks with low
   loss rates, retransmissions have great value without incurring
   additional complexity.
If the loss ratio is low, it seems retransmission is not very valuable,
and low delay networks are a very restricted set. I don't see what=20
additional diagnostic value this metric has, beyond the other loss metrics.
How widely is the NACK capability deployed and used?
Note: there could be link-layer retransmissions below RTP-layer,
but these are not visible above the link layer, except as=20
reordered packets.

[Rachel]: The metrics are not ordered by their importance, instead, they ar=
e listed from transport level to application level. I'm sorry it confused y=
ou. We can adjust the order in the next version. I have no additional comme=
nts other than Varun.=20


3.2.3 Loss, Discard and Duplicate Packet Count Metric
I agree that Loss and Discard are valuable for QoS and diagnostic
purposes, but in my experience Duplicate packets are rare and have
little effect on QoS (they are usually discarded silently, but they
should not appear in the discard counts).

[Rachel]: Okay. Agree.


3.2.5 Run Length Encoded Metrics for Loss, Discard and Post-repair

   Run-length encoding uses a bit vector to encode information about the
   packet. Each bit in the vector represents a packet and depending on
   the signaled metric it defines if the packet was lost, duplicated,
   discarded, or repaired. An endpoint typically uses the run length
   encoding to accurately communicate the status of each packet in the
   interval to the other endpoint.
This seems like an enormous amount of data to convey between
the receiver and sender, a fairly complete record of transmission.
It might be a diagnostic capability turned-on in some specific=20
troubleshooting cases, but I don't see RLE as part of the=20
QoS estimation metrics set.

[Rachel]: Okay.

It's fine to include Jitter and De-jitter buffer metrics in the QoS=20
estimation set. Others could be discussed, but it's probably good to
stop here for now.


hope this helps,
Al
PS I'm switching to other topics, and will probably delay any responses
on this thread for a couple of weeks.


> -----Original Message-----
> From: Romascanu, Dan (Dan) [mailto:dromasca@avaya.com]
> Sent: Thursday, August 01, 2013 3:01 PM
> To: MORTON JR., ALFRED C (AL)
> Cc: Huangyihong (Rachel); Varun Singh; Roni Even; Shida Schubert
> Subject: request for an early PMDIR review
>=20
> Hi Al,
>=20
> Following our discussion earlier today, I would kindly ask for an early
> PM-DIR review of https://datatracker.ietf.org/doc/draft-huang-xrblock-
> rtcweb-rtcp-xr-metrics/.
>=20
> Thanks and Regards,
>=20
> Dan
>=20
>=20
>=20