Subject: [PMOL] draft-ietf-pce-monitoring
From: "Romascanu, Dan (Dan)" <[email-masked]>
To: <[email-masked]>
Date: Tue, 28 Apr 2009 10:59:59 +0200
List-Id: Performance Metrics at Other Layers <pmol.ietf.org>

The I-D
http://www.ietf.org/internet-drafts/draft-ietf-pce-monitoring-04.txt
completed recently IETF LL and will soon be brought to the IESG for
approval. It would be good if folks could have a look and send their
comments to the authors. 

Dan

Content-class: urn:content-classes:message
Date: Thu, 9 Apr 2009 14:13:57 +0200
From: "Marian Delkinov" <[email-masked]>
To: "Al Morton" <[email-masked]>, <[email-masked]>
Cc: [email-masked]
Subject: Re: [PMOL] WG Last Call: draft-ietf-pmol-sip-perf-metrics-03

All my comments have been addressed in the document. 
Yes, my opinion is this draft should be forwarded to the Area Directors
for publication.

Best regards!
Mario.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Al Morton
Sent: Sunday, 05 April, 2009 21:14
To: [email-masked]
Cc: [email-masked]
Subject: [PMOL] WG Last Call: draft-ietf-pmol-sip-perf-metrics-03

PMOL WG,
cc SIPPING WG,

This message begins a second WG Last Call on the draft:

        Title		: SIP End-to-End Performance Metrics
	Filename	: draft-ietf-pmol-sip-perf-metrics-03

A URL for this Internet-Draft is:
http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-03

There has been discussion of this draft for 3 years, first in the
SIPPING WG, and then in the PMOL WG.
The first WGLC ended on July 25, 2008. All comments to date have been
addressed to the satisfaction of the submitters.

Please weigh-in on whether or not this Draft should be forwarded to the
Area Directors for publication as a Standards Track RFC.
Send your comments to the PMOL list or the co-chairs.

The Last Call will be open till April 20, 2009.

thanks for your review and comments,
Alan and Al
co-chairs, PMOL WG

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

Content-class: urn:content-classes:message
Date: Mon, 6 Apr 2009 09:36:01 -0700
From: "Loki Jorgenson" <[email-masked]>
To: <[email-masked]>
Subject: [PMOL] Comments on G.1010 for framework metrics draft

This is a multi-part message in MIME format.

------_=_NextPart_001_01C9B6D5.A7827EAA
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

In San Francisco at 74, Al Morton responded to my concerns about the
ambiguity of terms in the framework draft.   Specifically, they included
"application performance" as they pertained to QoS (better defined) and
QoE (somewhat defined).  He suggested that a reference ITU-T G.1010
(2001) might suffice.  I said I would review and comment.
 
Having reviewed G.1010, I suggest that a reference to G.1010 should
appear in the framework draft.  It provides a useful list of categories
that at least suggests how application performance relates to network
QoS.  
 
That said, I still see a clear need for something like a framework
definition for application performance (quality of application) and how
it relates to network QoS.  I won't address that here.
 
In particular, I suggest that section 3.2 be revised somewhat to refer
directly to "application performance" as it related to QoS.  Suggested
changes below (in CAPS) in -02 version of text (with additional
references suggested to G.107 and P.800 for illustration):
 
3.2.  Definitions of a Metric
   A metric is a measure of an observable behavior of an application,
   protocol or other system.  The definition of a metric often assumes
   some implicit or explicit underlying statistical process, and a
   metric is an estimate of a parameter of this process.  If the assumed
   statistical process closely models the behavior of the system then
   the metric is "better" in the sense that it more accurately
   characterizes the state or behavior of the system.

   A metric should serve some defined purpose.  This may include the
   measurement of capacity, quantifying how bad some problem is,
   measurement of service level, problem diagnosis or location and other
   such uses.  A metric may also be an input to some other process, for
   example the computation of a composite metric or a model or
   simulation of a system AS MIGHT BE EMPLOYED TO ASSESS THE 
   PERFORMANCE OF A SPECIFIC APPLICATION - FOR EXAMPLE THE EMODEL FOR
VOIP (G107).
   Tests of the "usefulness" of a metric include:

      (i) the degree to which its absence would cause significant loss
      of information on the behavior or state OR PERFORMANCE of the
application or
      system being measured

      (ii) the correlation between the metric and NETWORK QoS [G1000],
APPLICATION-SPECIFIC
      PERFORMANCE (G1010), OR experience delivered to the user (person
or other application) (P800)

      (iii) the degree to which the metric is able to support the
      identification and location of problems affecting service quality.

   For example, consider a distributed application operating over a
   network connection that is subject to packet loss.  A Packet Loss
   Rate (PLR) metric is defined as the mean packet loss rate over some
   time period.  If the application performs poorly over network
   connections with high packet loss rate and always performs well when
   the packet loss rate is zero then the PLR metric is useful to some
   degree.  Some applications are sensitive to short periods of high
   loss (bursty loss) and are relatively insensitive to isolated packet
   loss events; for this type of application there would be very weak
   correlation between PLR and application performance.  A "better"
   metric would consider both the packet loss rate and the distribution
   of loss events.  If application performance is degraded when the PLR
   exceeds some rate then a useful metric may be a measure of the
   duration and frequency of periods during which the PLR exceeds that
   rate.

Loki Jorgenson
Chief Scientist
Apparent Networks
The Hudson House
Suite 400 - 321 Water Street
Vancouver, BC, Canada, V6B 1B8

e   [email-masked]
t   604 433 2333 ext 105
f   604 433 2311
m   604 250-4642
w   www.ApparentNetworks.com

 

------_=_NextPart_001_01C9B6D5.A7827EAA
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=us-ascii" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.18702"></HEAD>
<BODY>
<DIV><SPAN class=148411116-06042009><FONT size=2 face=Arial>In San Francisco at 
74, Al Morton&nbsp;responded to my concerns about the ambiguity of terms in 
the&nbsp;framework draft.&nbsp;&nbsp; Specifically, they included "application 
performance" as they pertained to QoS (better defined) and QoE (somewhat 
defined).&nbsp; He suggested that a reference ITU-T G.1010 (2001) might 
suffice.&nbsp; I said I would review and comment.</FONT></SPAN></DIV>
<DIV><FONT size=2 face=Arial></FONT>&nbsp;</DIV>
<DIV><SPAN class=148411116-06042009><FONT size=2 face=Arial>Having reviewed 
G.1010, I suggest that a reference&nbsp;to G.1010 should appear in the framework 
draft.&nbsp; It provides a useful list of categories that at least suggests how 
application performance relates to network QoS.&nbsp; </FONT></SPAN></DIV>
<DIV><SPAN class=148411116-06042009><FONT size=2 
face=Arial></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=148411116-06042009><FONT size=2 face=Arial>That said, I still 
see a clear need for something like a framework definition for&nbsp;application 
performance&nbsp;(quality of application)&nbsp;and how it relates to network 
QoS.&nbsp; I won't address that here.</FONT></SPAN></DIV>
<DIV><SPAN class=148411116-06042009><FONT size=2 face=Arial></FONT></SPAN><SPAN 
class=148411116-06042009><FONT size=2 face=Arial></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=148411116-06042009><FONT size=2 face=Arial>In particular, 
I&nbsp;suggest&nbsp;that section 3.2 be revised somewhat to refer directly to 
"application performance" as it related to QoS.&nbsp; Suggested changes below 
(in CAPS) in -02 version of text (with additional references suggested to G.107 
and P.800 for illustration):</FONT></SPAN></DIV>
<DIV><SPAN class=148411116-06042009><FONT size=2 
face=Arial></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=148411116-06042009><PRE>3.2.  Definitions of a Metric
   A metric is a measure of an observable behavior of an application,
   protocol or other system.  The definition of a metric often assumes
   some implicit or explicit underlying statistical process, and a
   metric is an estimate of a parameter of this process.  If the assumed
   statistical process closely models the behavior of the system then
   the metric is "better" in the sense that it more accurately
   characterizes the state or behavior of the system.

   A metric should serve some defined purpose.  This may include the
   measurement of capacity, quantifying how bad some problem is,
   measurement of service level, problem diagnosis or location and other
   such uses.  A metric may also be an input to some other process, for
   example the computation of a composite metric or a model or
   simulation of a system AS MIGHT BE EMPLOYED TO ASSESS THE <BR>   PERFORMANCE OF A SPECIFIC APPLICATION - FOR EXAMPLE THE EMODEL FOR VOIP (G107).</PRE><PRE>   Tests of the "usefulness" of a metric include:

      (i) the degree to which its absence would cause significant loss
      of information on the behavior or state<FONT size=2 face=Arial> OR PERFORMANCE</FONT> of the application or
      system being measured

      (ii) the correlation between the metric and NETWORK QoS [G1000], APPLICATION-SPECIFIC<BR>      PERFORMANCE (G1010), OR experience delivered to the user (person or other application) (P800)

      (iii) the degree to which the metric is able to support the
      identification and location of problems affecting service quality.

   For example, consider a distributed application operating over a
   network connection that is subject to packet loss.  A Packet Loss
   Rate (PLR) metric is defined as the mean packet loss rate over some
   time period.  If the application performs poorly over network
   connections with high packet loss rate and always performs well when
   the packet loss rate is zero then the PLR metric is useful to some
   degree.  Some applications are sensitive to short periods of high
   loss (bursty loss) and are relatively insensitive to isolated packet
   loss events; for this type of application there would be very weak
   correlation between PLR and application performance.  A "better"
   metric would consider both the packet loss rate and the distribution
   of loss events.  If application performance is degraded when the PLR
   exceeds some rate then a useful metric may be a measure of the
   duration and frequency of periods during which the PLR exceeds that
   rate.
</PRE></SPAN></DIV><!-- Converted from text/plain format -->
<P><FONT size=2>Loki Jorgenson<BR>Chief Scientist<BR>Apparent Networks<BR>The 
Hudson House<BR>Suite 400 - 321 Water Street<BR>Vancouver, BC, Canada, V6B 
1B8<BR><BR>e&nbsp;&nbsp; [email-masked]<BR>t&nbsp;&nbsp; 604 
433 2333 ext 105<BR>f&nbsp;&nbsp; 604 433 2311<BR>m&nbsp;&nbsp; 604 
250-4642<BR>w&nbsp;&nbsp; www.ApparentNetworks.com<BR><BR></FONT></P>
<DIV>&nbsp;</DIV></BODY></HTML>

------_=_NextPart_001_01C9B6D5.A7827EAA--

Sensitivity: Normal
Importance: Normal
To: "Al Morton" <[email-masked]>,[email-masked],[email-masked]
From: "Scott Poretsky" <[email-masked]>
Date: Sun, 5 Apr 2009 20:05:33 +0000
Cc: [email-masked]
Subject: Re: [PMOL] WG Last Call: draft-ietf-pmol-sip-perf-metrics-03
Reply-To: [email-masked]

Date: Sun, 05 Apr 2009 15:14:07 -0400
To: [email-masked]
From: Al Morton <[email-masked]>
Cc: [email-masked]
Subject: [PMOL] WG Last Call: draft-ietf-pmol-sip-perf-metrics-03

PMOL WG,
cc SIPPING WG,

This message begins a second WG Last Call on the draft:

        Title		: SIP End-to-End Performance Metrics
	Filename	: draft-ietf-pmol-sip-perf-metrics-03

A URL for this Internet-Draft is:
http://tools.ietf.org/html/draft-ietf-pmol-sip-perf-metrics-03

There has been discussion of this draft for 3 years,
first in the SIPPING WG, and then in the PMOL WG.
The first WGLC ended on July 25, 2008. All comments to date
have been addressed to the satisfaction of the submitters.

Please weigh-in on whether or not this Draft should be forwarded
to the Area Directors for publication as a Standards Track RFC.
Send your comments to the PMOL list or the co-chairs.

The Last Call will be open till April 20, 2009.

thanks for your review and comments,
Alan and Al
co-chairs, PMOL WG

Date: Sun, 05 Apr 2009 15:20:22 -0400
To: [email-masked]
From: Al Morton <[email-masked]>
Subject: [PMOL] Draft Minutes of PMOL Session at IETF-74

for your review and comments (by April 20, please):
http://www.ietf.org/proceedings/09mar/minutes/pmol.html

thanks to Mike Hamilton for taking notes!

Al
co-chair, pmol wg