From nobody Wed Dec 2 08:06:16 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6E733A1471 for ; Wed, 2 Dec 2020 08:06:13 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.101 X-Spam-Level: X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OZIbjrtR4hbc for ; Wed, 2 Dec 2020 08:06:10 -0800 (PST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40078.outbound.protection.outlook.com [40.107.4.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8EE6C3A1699 for ; Wed, 2 Dec 2020 08:05:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N2ztVX8ymigjmFXacboXiiMMYXm+7L0oiMaUX7Z9/aX4f/nbTgCi75OQnodRcGwE9Z9HD92Qmq51IsMqSZc02MRZd6dEmoNRtGe+9eycJwzPx893fP1676wgYXhzTXInSOlGd83tXLRQ2ip+S3NTPituzrcyde0x5KWypbi9Nk7EKt20ju3jbjPoA2CcBuQvIRH9UT+penE7zVnYr3hOWlq+Et/KEdNQJf0VjWhF6YM7TriSz+xn2rLOCCAmkmndm4NI0F8fi4ZqHlWnw9ODEBF5oRzpVSVADOvGtXOuAsobO+eVOntm9Cg7MvB2JJ8tmoNBs04iaz+6TqI51W+jRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ir139OciYsK3bFCCA98S77h3eRlE+00ntZ/JRIEURCg=; b=D5a5xewFpsC4krEsANYyLEOaUE7b/yshpRjmEtSMIM24mEnSp1dQ4RUbEUeAysP3m4Rr89VDePmf9uJTJ0Acro9osS56CzV3G3JX2CZN4b0EEPcVNoTaHcDFRep4BHLxovGAfCWzxFUGk12XvMn+6NyFct5P9moQ6K/6mVLvtaMmWulAu3rCwjD70MAn4MBxmXR//92psGHTvhZOXSvHKQNdgbRIE7fQLOYr+NKXl1YCWagnITMiX/rQSRsoYNmkqiiM9AQWuYKRfFDRoyWVmZsPSV8krP2rBKUJ6mkwN5CsdNWba9L5ZXSKWpphY4qQxZAky8koW/IYxX1GPeE1Fw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ir139OciYsK3bFCCA98S77h3eRlE+00ntZ/JRIEURCg=; b=Ivvciqr+gsQ2rYRFBGVOnNEPkpxodeMt/U8S1CbMEkijAeNRvZytqjukin6BTtfOZcSZ31et+a2qzeIguRVxX6qAXpYpHuKc5YixP/iAbS+NLay18Bpq7VC7o6Z1WSs0xmtXq9bPOmiXzaAWWXjPa1nSsFvyutRi7QpJih7f/FA= Received: from AM0PR07MB3953.eurprd07.prod.outlook.com (2603:10a6:208:42::26) by AM0PR07MB6353.eurprd07.prod.outlook.com (2603:10a6:20b:153::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3632.6; Wed, 2 Dec 2020 16:05:17 +0000 Received: from AM0PR07MB3953.eurprd07.prod.outlook.com ([fe80::c445:ace0:2cb2:c48c]) by AM0PR07MB3953.eurprd07.prod.outlook.com ([fe80::c445:ace0:2cb2:c48c%3]) with mapi id 15.20.3632.008; Wed, 2 Dec 2020 16:05:17 +0000 From: Szilveszter Nadas To: "tcpprague@ietf.org" CC: Ferenc Fejes , Gombos Gergo , LAKI Sandor , "De Schepper, Koen (Nokia - BE/Antwerp)" , "Tilmans, Olivier (Nokia - BE/Antwerp)" Thread-Topic: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results Thread-Index: AQHWuNcd4GJeRrqtOUWvcu6WwpWhZqnkFQ3A Date: Wed, 2 Dec 2020 16:05:17 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [178.164.169.140] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ca9c4059-f996-4c88-e4ec-08d896dc0fbd x-ms-traffictypediagnostic: AM0PR07MB6353: x-ld-processed: 92e84ceb-fbfd-47ab-be52-080c6b87953f,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Yvnr+8g5Qtm4OYIR9UoJ8VhcWx4Dxgqyq5dmy6cTQKJyFC/pygnxSgXW0mvCp7udXcc/8Tu56XBF6KtOwtCwYy7bnxCvPziCTLDUU9wuPIpNW3mChOqvhIQwiUkLUlulY4AhWXqP3amlA9f8Z/pR/mjJPrPAmVavobLvk9LEM9ssJJNx4MP0Q59yvpn1IXwtUSCEvfItwN5AFItXFqqMyj0Nd7i54KhdD86sbrbd08szmHDqUBz3v1SbU00Dg00uSoQfWZO0ePgXPsDSkLnaCb7TPX403R0FxxaVn/dferhKZjf5guoVukgzRVZn6TOtMbZgh1QcUXqJETTaZZOZc/mi/zBhU6oRQVxSSR8Xxiq8eoHISXeUYvhCFWhO6X2v63ocz8dc4DekgZGOQ19onAVPfg+ItbdwEUs4sNrCkT0QlyD/dG9F8sezKk2JAt6FSo3CgVp9PQrIf0nG9zXfAg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR07MB3953.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(396003)(376002)(346002)(366004)(136003)(9686003)(966005)(53546011)(71200400001)(6506007)(316002)(26005)(6916009)(52536014)(7696005)(55016002)(54906003)(33656002)(66946007)(4326008)(66446008)(66556008)(166002)(186003)(2906002)(76116006)(83080400002)(5660300002)(8936002)(478600001)(66476007)(86362001)(83380400001)(64756008)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?GYC38G/vI0VMX0jvjhvciOHT1pJqWYp7Wg6Hi3xUacKYdjEZ6SD+mE2ASo?= =?iso-8859-1?Q?GmwSf1aL3QmA7iH5Y/3bhO66lNfhP/MOy1jbc1hfr3uzwNxm20f1S50uLZ?= =?iso-8859-1?Q?yG7y5uINIjIXciPgaw49OR/2AFTlrWVAdl8/VIYZCS3WQdfoPeyzU2Fk3C?= =?iso-8859-1?Q?lgrsrhv/u8JqpqtX5/Xfwf64KdQzBtTEqPzesXR8HZW/2NJj3/xyHeIHTS?= =?iso-8859-1?Q?LsLvsbv43AMWwobtHm7nRNnATKt8RJJh6nRF1sNf303579ByKt3QDtrKmW?= =?iso-8859-1?Q?3uQPL9sK/Rxu4tApG601xOeSaoJzOJ6qzfoRgXQpU9xF/75XsK8yQSKw0P?= =?iso-8859-1?Q?u/PX/1FIb+UXAw7B6aZzZN+ELhOASe08mBpGw/EddZRmh/OFEVq4IZmU1L?= =?iso-8859-1?Q?Q7UEbGL1Dmh6+OKUlDLw+BnYgHMRsYPtYvzVRFrSOn/lnCWcjCl5xqna0W?= =?iso-8859-1?Q?V7xEt/ByBFeZxjS/oASR2tAIHXFs9zu1ZgWudCNKn/uzgev4gNOKaG6WUb?= =?iso-8859-1?Q?7c6YhfWEVY+RRG9ywKh/65bJ10bDT9MDc5r952d+1ZloVEhwgPVCX9OQhz?= =?iso-8859-1?Q?7vk6/gcpyvLOy7Cbr3JnNjX7oYbiEle1Zauefttw8+6Xr3rsLtKeX58Xvm?= =?iso-8859-1?Q?qJF6zAlxkPepw7NtPUsjfabCKyQrViwAyp5vvLZayAsT3Q6FrMW41t/X6C?= =?iso-8859-1?Q?515ScxbLJ598BkropO3fediHyxajAUj7slplfesyukHDajEzLQObRbIL53?= =?iso-8859-1?Q?zESf+07JzMAOR1o2ZrMLEcTyK0G+MtTqdgTbcP+fKwK9IGxdECnXYlfB8T?= =?iso-8859-1?Q?WnZmMyHzyiMUK8JKimAmETwIkPXC/gELwu35ZU6fwmalgVAvXrbQDveHdR?= =?iso-8859-1?Q?SLYL2MbBtjx6HXgLYt0lWHv+oUMJf/oJz747XCbBqVToqHp4ND2duu/eqI?= =?iso-8859-1?Q?n0s9yiFp2O5x80R7EuV23rA4Xnd244ttRAgFeEtOCyUnar7Vlu2xfZM6v+?= =?iso-8859-1?Q?XOlf8LFnicbN58v3k=3D?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR07MB39538F2B78AD479F413691878BF30AM0PR07MB3953eurp_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR07MB3953.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: ca9c4059-f996-4c88-e4ec-08d896dc0fbd X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2020 16:05:17.3430 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Wakn8VUo8DKy5Di7pi7/gK2ixnxVukP3WY18olxejPwjJjv6q3LmeSLt8FRMKeHkt98BghJf81ibOG4i6wGRRtJhJaz0sYWRKiEem9u49fc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR07MB6353 Archived-At: Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 16:06:14 -0000 --_000_AM0PR07MB39538F2B78AD479F413691878BF30AM0PR07MB3953eurp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Koen and Olivier, Thanks for the answers. Some follow-up questions: =3D=3DRTT unfairness as 1:5 ratio.=3D=3D Is there a way to compensate it in the DualPI2 scheduler? My understanding = was that with the right parameters it is compensated "automatically", there= was good fairness among DC and Cubic flows in the original (Dual)PI2 paper= s. I guess that for single PI2 that was because of the shared queue. But wh= at about DualPI2? Can it be configured in a way that fairness remains also= considering queueing for Cubic? (I vaguely remember a description of this = in one of your papers, but I cannot find it now). I guess that the good fai= rness with DCTCP vs. Cubic was an actual anomaly, not the worse fairness wi= th Prague vs. Cubic? =3D RTT independent Prague version=3D Can you provide a pointer how to configure that? Are the TCP Prague CC para= meter defaults in the version we used not meaningful? Shall we change other= parameters? Cheers, Szilveszter -----Original Message----- From: Tilmans, Olivier (Nokia - BE/Antwerp) Sent: November 5, 2020 12:10 To: Szilveszter Nadas ; tcpprague@ietf.org Cc: Ferenc Fejes ; Gombos Gergo ; L= AKI Sandor Subject: RE: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results Hi Szilveszter, > The results are at the below link. We are somewhat surprised by much > = increased aggressiveness of TCP Prague. Can you comment on it? The settings= > and setup we used are described in the pdf (and in the original article= ). Thanks for sharing these. * Short answer: DCTCP actively hurts itself, tripping the step AQM unnecessarily. This is mostly benign in DC environments due to the almost non existent RTT= , and small cwnd, but causes observable performance hits elsewhere. Prague implements fixes for these issues. * Long answer: Please find below the main changes from dctcp to prague. The intuition behi= nd these is to avoid triggering the step AQM part of the dual queue when th= ere is classic traffic, or limit the extent of the received marks. The first goal ensure that, when there is classic traffic, prague only rece= ives marks from the coupled PI2 component, which are strong enough on their= own to keep the L4S queue empty most of the time. Additionally getting mar= ks from the step AQM means that prague is hurting itself, as it will reduce= more than necessary. The second goal ensures that prague does not unnecess= arily drive the AQM (either PI2 or the step) to high-marking rate; again be= cause it will hurts its throughput and also because it will create a standi= ng queue. Note that depending on your experimental setups, not all the below factors = will contribute. 1. Prague is always paced, regardless of the egress qdisc on the data sende= r, and paces itself at 100% of the estimated BDP. DCTCP is only paced when used in combination with fq, and p= aces at 120% of the BDP when that is the case. 2. Prague actively limits its gso bursts (defaults to 250us) DCTCP uses Linux defaults, 1ms 3. Prague implements a more accurate internal marking estimate (alpha) DCTCP, especially when operating with low marking probabili= ties, will tend to push down alpha to 0, delaying its response to mark= s (and thus increasing queue pressure/causing larger reduction down the= line) 4. Prague carry over sub-cwnd reduction, such that multiple marks in= a row occurring at low overall probability will eventually cause a cwnd reduct= ion DCTCP's cwnd reduction code has no memory, i.e., as long as= alpha is low enough compared to cwnd, no cwnd reduction will ever oc= cur. This again drives the aqm to larger mark probabilities. All of these points contribute to prague being more gentle towards the queu= e, and more reactive to the received marks (i.e., achieving a lower overall= marking level)--the direct effect being smaller cwnd reduction (thus sawto= oth) than DCTP, which is critical when operating at higher e2e RTTs with a = shallow queue since it lowers the time to recovery. Additionally, when operating over a long RTT path and controlling by PI2 (i= .e., random marking, such that it receives marks almost every RTT--as oppos= ed to a step where it receives nothing for several RTTs and then a RTT with= most of the packets being marked), DCTCP suffers from the inaccurate/memor= yless computations in 3./4. and its interactions with Linux's PRR code (whi= ch prague does not use), which can prevent it from ever reaching its expect= ed rate. I hope this helps to understand a bit better why prague appears to be more = aggressive--it is actually the opposite: dctcp systematically hurts itself = at longer base RTT, hence progressively give way as the RTT increases. You can validate this by observing that the expected rate ratios between cl= assic flows and prague over the dualQ match the theoretical equations (comp= ounding the queue delay difference) even that higher base RTTs, which is no= t the case for DCTCP. Best, Olivier From: De Schepper, Koen (Nokia - BE/Antwerp) Sent: November 12, 2020 10:35 To: Szilveszter Nadas ; tcpprague@ietf.org Cc: Ferenc Fejes ; Gombos Gergo ; L= AKI Sandor Subject: RE: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results Hi Szilveszter, Interesting work, thanks for sharing. There is indeed a very big difference= between how DCTCP behaves today in recent kernels, than what we used in th= e original L4S work testing (Linux kernel 3.19). That original version was = a "clean" DCTCP version that matched very well the theoretical equations: r= =3D2/(p.RTT) for uniform stable random and r=3D2/(p=B2.RTT) on/off marking.= Due to many interactions, integer range constraints, pacing/GSO interactio= ns and bugs, the performance really degraded in the recent Linux versions. = These issues were fixed within the Linux Prague version, so that is why Pra= gue should perform "better" in respect to matching the equations. It would = be interesting to get an idea of the deviation from the r=3D2/(p.RTT) equat= ion in your tests (relevant in DualPI2 as the coupling gives a smooth proba= bility). Collecting the average RTT (base + queuing latency), marking proba= bility, and rate would make it possible to evaluate this. Main reasons why we saw deviations is when the probability varies a lot in = time. As you might know DCTCP's (and Prague's) equation becomes r=3D2/(p=B2= .RTT) when it receives on/off marking episodes (on episodes in the order of= 1 RTT, off in the order of multiple RTTs). Any not so stable marking proba= bility results in a rate between those 2 boundaries. This is the theory, lo= oking forward on how your practice matches this. >From a first quick look at the results it might not deviate that much, I gu= ess. The more aggressive part is probably due to the RTT unfairness: 1 to 5= rate ratio (with a 5ms base RTT, Classic gets a queue of 5ms+20ms target, = so about 25ms and 5 times less throughput). Did you try to use the RTT inde= pendent version? If you set it to the f=3Dmax(15, RTT) mode, the minimum ef= fective RTT becomes 15ms, so the rate ratio is 3 to 5 in that case. Thanks and Regards, Koen. From: tcpPrague > On Behalf Of Szilveszter Nadas Sent: Tuesday, November 3, 2020 6:30 PM To: tcpprague@ietf.org Cc: Ferenc Fejes >; Gombos Gerg= o >; LAKI Sandor > Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP P= rague preliminary results Hi all, During the review of our article "A Congestion Control Independent L4S Sche= duler" we received comments from one of the reviewers, on why we did not al= so evaluate TCP Prague. So now we rerun the test cases with replacing DCTCP= to TCP Prague. The results are at the below link. We are somewhat surprised by much increa= sed aggressiveness of TCP Prague. Can you comment on it? The settings and s= etup we used are described in the pdf (and in the original article). Results: http://ppv.elte.hu/tcp-prague/ Original article (including YouTube presentation): http://ppv.elte.hu/cc-in= dependent-l4s/ Can you comment on the results and the settings we used? Cheers, Szilveszter --_000_AM0PR07MB39538F2B78AD479F413691878BF30AM0PR07MB3953eurp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Koen and  Oliv= ier,

 

Thanks for the answers= .

 

Some follow-up questio= ns:

 

=3D=3DRTT unfairness a= s 1:5 ratio.=3D=3D

Is there a way to comp= ensate it in the DualPI2 scheduler? My understanding was that with the righ= t parameters it is compensated “automatically”, there was good = fairness among DC and Cubic flows in the original (Dual)PI2 papers. I guess that for single PI2 that was because of the shar= ed queue. But what about DualPI2? Can it be configured in a way that  = fairness remains also considering queueing for Cubic? (I vaguely remember a= description of this in one of your papers, but I cannot find it now). I guess that the good fairness with DCTCP vs. C= ubic was an actual anomaly, not the worse fairness with Prague vs. Cubic?

 

=3D RTT indepen= dent Prague version=3D

Can you provide a poin= ter how to configure that? Are the TCP Prague CC parameter defaults in the = version we used not meaningful? Shall we change other parameters?

 

Cheers,

Szilveszter=

 

-----Original Message-----
From: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-= labs.com>
Sent: November 5, 2020 12:10
To: Szilveszter Nadas <Szilveszter.Nadas@ericsson.com>; tcpprague@iet= f.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; Gombos Gergo <ggombos@inf.el= te.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: RE: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results

 

Hi Szilveszter,

 

 > The results are at the below link. We = are somewhat surprised by much  > increased aggressiveness of TCP P= rague. Can you comment on it? The settings  > and setup we used are= described in the pdf (and in the original article).

 

Thanks for sharing these.

 

* Short answer:

DCTCP actively hurts itself, tripping the step AQ= M unnecessarily.

This is mostly benign in DC environments due to t= he almost non existent RTT, and small cwnd, but causes observable performan= ce hits elsewhere.

Prague implements fixes for these issues.

 

* Long answer:

Please find below the main changes from dctcp to = prague. The intuition behind these is to avoid triggering the step AQM part= of the dual queue when there is classic traffic, or limit the extent of th= e received marks.

The first goal ensure that, when there is classic= traffic, prague only receives marks from the coupled PI2 component, which = are strong enough on their own to keep the L4S queue empty most of the time= . Additionally getting marks from the step AQM means that prague is hurting itself, as it will reduce more t= han necessary. The second goal ensures that prague does not unnecessarily d= rive the AQM (either PI2 or the step) to high-marking rate; again because i= t will hurts its throughput and also because it will create a standing queue.

 

Note that depending on your experimental setups, = not all the below factors will contribute.

 

1. Prague is always paced, regardless of the egre= ss qdisc on the data sender,

   and paces itself at 100% of the esti= mated BDP.

        &= nbsp;       DCTCP is only paced when used in = combination with fq, and paces at 120%

        &= nbsp;       of the BDP when that is the case.=

2. Prague actively limits its gso bursts (default= s to 250us)

        &= nbsp;       DCTCP uses Linux defaults, 1ms

3. Prague implements a more accurate internal mar= king estimate (alpha)

        &= nbsp;       DCTCP, especially when operating = with low marking probabilities, will

        &= nbsp;       tend to push down alpha to 0, del= aying its response to marks (and thus

        &= nbsp;       increasing queue pressure/causing= larger reduction down the line) 4. Prague carry over sub-cwnd reduction, s= uch that multiple marks in a row

   occurring at low overall probability= will eventually cause a cwnd reduction

        &= nbsp;       DCTCP's cwnd reduction code has n= o memory, i.e., as long as alpha is

        &= nbsp;       low enough compared to cwnd, no c= wnd reduction will ever occur. This

        &= nbsp;       again drives the aqm to larger ma= rk probabilities.

 

All of these points contribute to prague being mo= re gentle towards the queue, and more reactive to the received marks (i.e.,= achieving a lower overall marking level)--the direct effect being smaller = cwnd reduction (thus sawtooth) than DCTP, which is critical when operating at higher e2e RTTs with a shallow q= ueue since it lowers the time to recovery.

 

Additionally, when operating over a long RTT path= and controlling by PI2 (i.e., random marking, such that it receives marks = almost every RTT--as opposed to a step where it receives nothing for severa= l RTTs and then a RTT with most of the packets being marked), DCTCP suffers from the inaccurate/memoryless co= mputations in 3./4. and its interactions with Linux's PRR code (which pragu= e does not use), which can prevent it from ever reaching its expected rate.=

 

I hope this helps to understand a bit better why = prague appears to be more aggressive--it is actually the opposite: dctcp sy= stematically hurts itself at longer base RTT, hence progressively give way = as the RTT increases.

You can validate this by observing that the expec= ted rate ratios between classic flows and prague over the dualQ match the t= heoretical equations (compounding the queue delay difference) even that hig= her base RTTs, which is not the case for DCTCP.

 

 

Best,

Olivier

 

 

From: De Schepper, Koen (Nokia - BE/Antwerp) = <koen.de_schepper@nokia-bell-labs.com>
Sent: November 12, 2020 10:35
To: Szilveszter Nadas <Szilveszter.Nadas@ericsson.com>; tcppra= gue@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; Gombos Gergo <ggombos= @inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: RE: [tcpPrague] A Congestion Control Independent L4S Schedu= ler - TCP Prague preliminary results

 

Hi Szilveszter,

 

Interesting work, thanks for sharing. There is indee= d a very big difference between how DCTCP behaves today in recent kernels, = than what we used in the original L4S work testing (Linux kernel 3.19). Tha= t original version was a “clean” DCTCP version that matched very well the theoretical equations: r=3D2/(p.RTT) fo= r uniform stable random and r=3D2/(p=B2.RTT) on/off marking. Due to many in= teractions, integer range constraints, pacing/GSO interactions and bugs, th= e performance really degraded in the recent Linux versions. These issues were fixed within the Linux Prague version, s= o that is why Prague should perform “better” in respect to matc= hing the equations. It would be interesting to get an idea of the deviation= from the r=3D2/(p.RTT) equation in your tests (relevant in DualPI2 as the coupling gives a smooth probability). Collecti= ng the average RTT (base + queuing latency), marking probability, and rate = would make it possible to evaluate this.

 

Main reasons why we saw deviations is when the proba= bility varies a lot in time. As you might know DCTCP’s (and Prague= 217;s) equation becomes r=3D2/(p=B2.RTT) when it receives on/off marking ep= isodes (on episodes in the order of 1 RTT, off in the order of multiple RTTs). Any not so stable marking probability results= in a rate between those 2 boundaries. This is the theory, looking forward = on how your practice matches this.

 

From a first quick look at the results it might not = deviate that much, I guess. The more aggressive part is probably due to the= RTT unfairness: 1 to 5 rate ratio (with a 5ms base RTT, Classic gets a que= ue of 5ms+20ms target, so about 25ms and 5 times less throughput). Did you try to use the RTT independent versi= on? If you set it to the f=3Dmax(15, RTT) mode, the minimum effective RTT b= ecomes 15ms, so the rate ratio is 3 to 5 in that case.

 

Thanks and Regards,

Koen.

 

From: tcpPrague <tcpprague-bounces@ietf.org> On Behalf Of Szilveszter Nadas
Sent: Tuesday, November 3, 2020 6:30 PM
To: tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.= elte.hu>; Gombos Gergo <gg= ombos@inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler = - TCP Prague preliminary results

 

Hi all,

 

During the review of our article “A Congestion= Control Independent L4S Scheduler” we received comments from one of = the reviewers, on why we did not also evaluate TCP Prague. So now we rerun = the test cases with replacing DCTCP to TCP Prague.

 

The results are at the below link. We are somewhat s= urprised by much increased aggressiveness of TCP Prague. Can you comment on= it? The settings and setup we used are described in the pdf (and in the or= iginal article).

Results: = http://ppv.elte.hu/tcp-prague/

Original article (including YouTube presentation): <= a href=3D"http://ppv.elte.hu/cc-independent-l4s/"> http://ppv.elte.hu/cc-independent-l4s/

 

Can you comment on the results and the settings we u= sed?

 

Cheers,

Szilveszter

--_000_AM0PR07MB39538F2B78AD479F413691878BF30AM0PR07MB3953eurp_-- From nobody Wed Dec 2 13:24:00 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0B5E33A1A32 for ; Wed, 2 Dec 2020 13:23:51 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qo4eb6IEPVeb for ; Wed, 2 Dec 2020 13:23:47 -0800 (PST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80093.outbound.protection.outlook.com [40.107.8.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DB2043A1706 for ; Wed, 2 Dec 2020 13:22:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bCzkdQ/hFN50UsOVmALjFCUUZIOLDWi91kYqN3wmp0TLRwasx+XaOQAkSPBcUAyHt6XM91CRO1irGliuq06JIFCuY+B/G7OFLezirMqw9iyIzOP/2uC6KpfUSXuFy8ZQO5Y/V0rn4dKYU1uUR51XlZIX2Dhplhp9vwSWHOPInr7qUYfHwTEe7A5qZm+eFeXITFlfZwijeoDy0GrPF6ZamJFu8OMIA1WVQCN2tsrHOoYBzy5BxRlNDB8Ma0ndbfmiy3O0oiz1FzHPzYnD7w78+3I+ssK50n957flplgo4mEI+0bWXTm0AAwA5BQETkmphzFu4yFTPo0iDZJ5soQ3ShQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uQUaNmIgfYFjtyr/cShu2o8AXwz5i6jEAJVz3gAtAHM=; b=NxR8lSz2spYUyws1+NWdNpT4OIal5y0w3anfsXyR/Wjpx3tQkDU5OOYNtgToCk28I5zJXai1rn4FA1RpCJQQAIkx2O+/3M5SNY0WOzoRLOfccnLCYioEtdcaypmq5x3YkbzZOToEpJ9f0WTobk7brr78qosDMwR1vgD3th1bAls7HFu0lg4F9DPCgdUWyKaWW5m4camv20cf5GzzcdeaBmNapvTg7xj28xvejHyqyNHVSYLbNMGHUj8iOnywvxSpGTOUI4B2MHn31Wv80LiK7RlK7A/ZPddfjnX1lwfRj9ME7K1hC+T/279oG+vx1HsNyKc85cCC9aqukoq9JSdyZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-bell-labs.com; dmarc=pass action=none header.from=nokia-bell-labs.com; dkim=pass header.d=nokia-bell-labs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uQUaNmIgfYFjtyr/cShu2o8AXwz5i6jEAJVz3gAtAHM=; b=SIeyrFh7KURC0d6cK0HrsudaUUdQkNOSoxjyaI4sLm5dDF80/Yp+/Xj3YdD9ceW7XJl2OEdou8q9JNDVBMMaEQ3SfaomZmQoDZSNFOhbbqs/aM8lEu8oZaMaPcAzJQyiJbCaomwV3+nUiyN18kFvdbtPDC33HfbiTkOPpL+4hAI= Received: from AM8PR07MB7476.eurprd07.prod.outlook.com (2603:10a6:20b:24e::12) by AM0PR07MB4004.eurprd07.prod.outlook.com (2603:10a6:208:47::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3632.6; Wed, 2 Dec 2020 21:22:09 +0000 Received: from AM8PR07MB7476.eurprd07.prod.outlook.com ([fe80::4002:1615:680b:fe63]) by AM8PR07MB7476.eurprd07.prod.outlook.com ([fe80::4002:1615:680b:fe63%4]) with mapi id 15.20.3654.005; Wed, 2 Dec 2020 21:22:09 +0000 From: "De Schepper, Koen (Nokia - BE/Antwerp)" To: Szilveszter Nadas , "tcpprague@ietf.org" CC: Ferenc Fejes , Gombos Gergo , LAKI Sandor , "Tilmans, Olivier (Nokia - BE/Antwerp)" Thread-Topic: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results Thread-Index: AdayBknBddpo1c1ATr20xSsm+1eksgGzRlkAA/xieYAACVrTsA== Date: Wed, 2 Dec 2020 21:22:09 +0000 Message-ID: References: In-Reply-To: Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ericsson.com; dkim=none (message not signed) header.d=none;ericsson.com; dmarc=none action=none header.from=nokia-bell-labs.com; x-originating-ip: [2a02:1810:1e00:cb00:c4f5:4915:631:4406] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 9c83f5bb-08ad-41c3-9bf4-08d8970853ad x-ms-traffictypediagnostic: AM0PR07MB4004: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: AViPdOdY147ZU33AFpZQaadwF5BNqa+VFyPdl7QMV1TchS5wurhlkr7rvzMroPHgdKLAgXxAgNV8KH2fP+DBUkk9Sj/NHL/6notIa9fqBU4P4+QX8J+eB9rkyGTqcT2EU7cGndFZTPP5o0MEtcXwaMPTQGCKz9bE4R8DT8AMdCWWvq6ySaH1NOdiAvmF22UEd0Q/YiJcrUKXF51ODTeajFFnK3KS7FfRdLZqyS+a0c/UfLo6QLTRZ7WH5qxFJ3jPQxhDM8A5/YP7YWQKPx0+WNQHWkyKbpK2O2tJHqSm+gzkbavoNzQ8yt8XOgq4KT6Y53lC1ZHecQLWNSppuHHXiKysEP6EX7B2aXcmt7PdWpKTWAt7S3PwB1trA1294eaXIKBXZhx1qjCWjYPrWmho3DVVfH/ZnU+IIIhqRvHmRAaeRtpFEE5VMl//dLXApW8cED/G44LBwhQcp4MpG16hFg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR07MB7476.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(136003)(366004)(396003)(376002)(346002)(316002)(107886003)(71200400001)(76116006)(66946007)(83080400002)(4326008)(9326002)(6506007)(53546011)(9686003)(186003)(83380400001)(2906002)(110136005)(54906003)(86362001)(7696005)(8936002)(5660300002)(8676002)(966005)(166002)(33656002)(64756008)(66476007)(52536014)(66556008)(66446008)(30864003)(55016002)(478600001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?92JehOcOsUYHiyV6zRp80ctVYDEL1em/RYSZHslHdodD0So3kTIOE/JOcE?= =?iso-8859-1?Q?SYPsa8YtknMGqVKUb33m/e/hjgreHZSEpPjLiA7yaCS6I1ybfcL40C7bd/?= =?iso-8859-1?Q?6muB7jsq6ncU14nS+u7drX+SYO6Bqt7i71C/hNpTcO8fnsMhs+rXIcdtza?= =?iso-8859-1?Q?8eUeDaTqs23XHdgyxsDZ1UIYfoOgu3oQwlNnqlvZCF12VvbrseD8TcXwJu?= =?iso-8859-1?Q?GZjRH1BMLuuAU94mJL22el6B/sHay1igXCpd8eYuF7SH8E5wg+Ay8mLcdp?= =?iso-8859-1?Q?vwAbJF1O028OywOSifXg9siovuy19gGkIjTjZzcIqhdD6DJknDcSBwtyHb?= =?iso-8859-1?Q?rOFmya7qmGMqpOG0CN4bL4TCgRWgm4Pl3Ecjm99Adyg8OP+vxzpHohRuEk?= =?iso-8859-1?Q?NEgCPV6giE/4V5o+kxFDVcfZp6HAync5fy2oYASz38PVjA8GE0l2Ozs6Bv?= =?iso-8859-1?Q?dObrvBHcmkrOEwEB/E1eEmFMAIOuaJZ13BAf+usKwRFfZ2T3JqpSHg+L0+?= =?iso-8859-1?Q?PnXbtUStGmYx6emcWUvzJ7mKb1gpMtKW7S6rmCqurVoN7Gm2EoEZPokRTU?= =?iso-8859-1?Q?QcGjlGSIEGtSnV5kSDmqJh7N9dvMVoYx78wb7NaPPwjEAW5G1VvZV+lrQj?= =?iso-8859-1?Q?BG89HelrfTKkgN70WYaGoQED0vLS8DWjEZ53/5mOrcYaFLzS/FHgM7djyg?= =?iso-8859-1?Q?IX5XIKrc3BL7qzWt8aHLCMWiIMF6TBJqmpHwIizYO19B/rDouq+7h3mSzp?= =?iso-8859-1?Q?wUF8QsU7PrV4rs6Ojud0Num66aIzK+qU7kg/qIt9f2BT4+u1uoJWMU+D+L?= =?iso-8859-1?Q?tbYIMvWlgWpInnRSrO5MGZnVY2Zp7NplPYuedMnr7Itpj24Dlv57jbyKMQ?= =?iso-8859-1?Q?MEGoXE0JRt7vQYb1zOhvBcPyA9KEeEtHsfJ6+ND6eTRAhMRZBHYLtbXVN5?= =?iso-8859-1?Q?WS2JFE7RivW7s3XRkf+oj+DAirndR6LcMm26e7GGbDEgr9f0mQ7R8xEpfL?= =?iso-8859-1?Q?a6LO67TLbjd8JfrRCViZf9fuuB2Bw3OJ9lF3Vxgyb3g2UpWndaZ72SNqWz?= =?iso-8859-1?Q?x/i9U9WxuxYhGnUHqkCQP/c=3D?= Content-Type: multipart/alternative; boundary="_000_AM8PR07MB7476958FF16BA93CCE92433FB9F30AM8PR07MB7476eurp_" MIME-Version: 1.0 X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM8PR07MB7476.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9c83f5bb-08ad-41c3-9bf4-08d8970853ad X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2020 21:22:09.1547 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: tAnKxJNeNzrNOg2qxYufcvxB9QlsPdPBuYshNS24OSxNFMd6+volxf+YSv02Fb7A55CR//5OS2kCLbqhs2qvsnta2YOufGRUIRLAXJeYYvf9oGJ0lRgFMVoENRZpJpFd X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR07MB4004 Archived-At: Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Dec 2020 21:23:58 -0000 --_000_AM8PR07MB7476958FF16BA93CCE92433FB9F30AM8PR07MB7476eurp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Szilveszter, First thing is to check that you can reproduce our results. With the previo= us (non-RTT-independent) Prague, can you confirm you get the following resu= lts? Normally with the right coupling (p_C =3D p'^2 and P_L =3D 2*p') and if the= ir total RTT is the same, Cubic (in Reno mode) and Prague get about the sam= e throughput (Prague might get up to twice the throughput max). Total RTT m= eans base RTT + all Queuing delay for each flow's round trip path. Can you = confirm this, or do you see a bigger difference? One way to check this on an equal base RTT setup is to use a singleQ PI2 wh= ich is doing the correct marking or dropping according to the above equati= ons, because the single Q gives both classes the same latency (single Q = =3D same Q delay). Another way is to use a DualQ and compensate the 15ms Cl= assic latency with a 15ms bigger base RTT for Prague. In both cases you wou= ld see them getting around the same throughput. If the BDP (rate and/or RTT= ) becomes bigger, Cubic will start to get a higher throughput than Prague. Note that if you install the latest Prague version from our L4STeam git, th= e default is now to be RTT independent below 25ms. So any Prague with a low= er than 25 ms throughput will behave as a 25ms RTT flow (effective after 50= 0 RTTs), and will get the same throughput as a 25ms RTT flow. You could tes= t this with Prague on a DualPi2, when competing with a Cubic flow with a ba= se RTT of 10ms. All RTTs (you can try even a mix of different ones in paral= lel) below 25ms should now get the same throughput as the Cubic one (becaus= e it has also a total RTT of 25ms). Any other effective RTT combinations different than the equal ones should f= ollow the "throughput ratio equals the inverse RTT ratio" rule. One excepti= on is when non-Rtt-independent Prague flows with different RTTs ares compet= ing on a STEP AQM. In that case the difference is bigger than that ratio, b= ecause the lowest RTTs are getting on-off marking, and behave as 1/p=B2, in= stead of 1/p as they will get a more stable marking probability over their = (much bigger) RTTs. Let me know if you can reproduce the above behavior. If not, maybe try usin= g the Linux DualPI2, and if still not as expected, then we need to dive dee= per to find the reason. Thanks, Koen. From: Szilveszter Nadas Sent: Wednesday, December 2, 2020 5:05 PM To: tcpprague@ietf.org Cc: Ferenc Fejes ; Gombos Gergo ; L= AKI Sandor ; De Schepper, Koen (Nokia - BE/Antwerp) ; Tilmans, Olivier (Nokia - BE/Antwerp) <= olivier.tilmans@nokia-bell-labs.com> Subject: RE: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results Hi Koen and Olivier, Thanks for the answers. Some follow-up questions: =3D=3DRTT unfairness as 1:5 ratio.=3D=3D Is there a way to compensate it in the DualPI2 scheduler? My understanding = was that with the right parameters it is compensated "automatically", there= was good fairness among DC and Cubic flows in the original (Dual)PI2 paper= s. I guess that for single PI2 that was because of the shared queue. But wh= at about DualPI2? Can it be configured in a way that fairness remains also= considering queueing for Cubic? (I vaguely remember a description of this = in one of your papers, but I cannot find it now). I guess that the good fai= rness with DCTCP vs. Cubic was an actual anomaly, not the worse fairness wi= th Prague vs. Cubic? =3D RTT independent Prague version=3D Can you provide a pointer how to configure that? Are the TCP Prague CC para= meter defaults in the version we used not meaningful? Shall we change other= parameters? Cheers, Szilveszter -----Original Message----- From: Tilmans, Olivier (Nokia - BE/Antwerp) > Sent: November 5, 2020 12:10 To: Szilveszter Nadas >; tcpprague@ietf.org Cc: Ferenc Fejes >; Gombos Gerg= o >; LAKI Sandor > Subject: RE: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results Hi Szilveszter, > The results are at the below link. We are somewhat surprised by much > = increased aggressiveness of TCP Prague. Can you comment on it? The settings= > and setup we used are described in the pdf (and in the original article= ). Thanks for sharing these. * Short answer: DCTCP actively hurts itself, tripping the step AQM unnecessarily. This is mostly benign in DC environments due to the almost non existent RTT= , and small cwnd, but causes observable performance hits elsewhere. Prague implements fixes for these issues. * Long answer: Please find below the main changes from dctcp to prague. The intuition behi= nd these is to avoid triggering the step AQM part of the dual queue when th= ere is classic traffic, or limit the extent of the received marks. The first goal ensure that, when there is classic traffic, prague only rece= ives marks from the coupled PI2 component, which are strong enough on their= own to keep the L4S queue empty most of the time. Additionally getting mar= ks from the step AQM means that prague is hurting itself, as it will reduce= more than necessary. The second goal ensures that prague does not unnecess= arily drive the AQM (either PI2 or the step) to high-marking rate; again be= cause it will hurts its throughput and also because it will create a standi= ng queue. Note that depending on your experimental setups, not all the below factors = will contribute. 1. Prague is always paced, regardless of the egress qdisc on the data sende= r, and paces itself at 100% of the estimated BDP. DCTCP is only paced when used in combination with fq, and p= aces at 120% of the BDP when that is the case. 2. Prague actively limits its gso bursts (defaults to 250us) DCTCP uses Linux defaults, 1ms 3. Prague implements a more accurate internal marking estimate (alpha) DCTCP, especially when operating with low marking probabili= ties, will tend to push down alpha to 0, delaying its response to mark= s (and thus increasing queue pressure/causing larger reduction down the= line) 4. Prague carry over sub-cwnd reduction, such that multiple marks in= a row occurring at low overall probability will eventually cause a cwnd reduct= ion DCTCP's cwnd reduction code has no memory, i.e., as long as= alpha is low enough compared to cwnd, no cwnd reduction will ever oc= cur. This again drives the aqm to larger mark probabilities. All of these points contribute to prague being more gentle towards the queu= e, and more reactive to the received marks (i.e., achieving a lower overall= marking level)--the direct effect being smaller cwnd reduction (thus sawto= oth) than DCTP, which is critical when operating at higher e2e RTTs with a = shallow queue since it lowers the time to recovery. Additionally, when operating over a long RTT path and controlling by PI2 (i= .e., random marking, such that it receives marks almost every RTT--as oppos= ed to a step where it receives nothing for several RTTs and then a RTT with= most of the packets being marked), DCTCP suffers from the inaccurate/memor= yless computations in 3./4. and its interactions with Linux's PRR code (whi= ch prague does not use), which can prevent it from ever reaching its expect= ed rate. I hope this helps to understand a bit better why prague appears to be more = aggressive--it is actually the opposite: dctcp systematically hurts itself = at longer base RTT, hence progressively give way as the RTT increases. You can validate this by observing that the expected rate ratios between cl= assic flows and prague over the dualQ match the theoretical equations (comp= ounding the queue delay difference) even that higher base RTTs, which is no= t the case for DCTCP. Best, Olivier From: De Schepper, Koen (Nokia - BE/Antwerp) > Sent: November 12, 2020 10:35 To: Szilveszter Nadas >; tcpprague@ietf.org Cc: Ferenc Fejes >; Gombos Gerg= o >; LAKI Sandor > Subject: RE: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results Hi Szilveszter, Interesting work, thanks for sharing. There is indeed a very big difference= between how DCTCP behaves today in recent kernels, than what we used in th= e original L4S work testing (Linux kernel 3.19). That original version was = a "clean" DCTCP version that matched very well the theoretical equations: r= =3D2/(p.RTT) for uniform stable random and r=3D2/(p=B2.RTT) on/off marking.= Due to many interactions, integer range constraints, pacing/GSO interactio= ns and bugs, the performance really degraded in the recent Linux versions. = These issues were fixed within the Linux Prague version, so that is why Pra= gue should perform "better" in respect to matching the equations. It would = be interesting to get an idea of the deviation from the r=3D2/(p.RTT) equat= ion in your tests (relevant in DualPI2 as the coupling gives a smooth proba= bility). Collecting the average RTT (base + queuing latency), marking proba= bility, and rate would make it possible to evaluate this. Main reasons why we saw deviations is when the probability varies a lot in = time. As you might know DCTCP's (and Prague's) equation becomes r=3D2/(p=B2= .RTT) when it receives on/off marking episodes (on episodes in the order of= 1 RTT, off in the order of multiple RTTs). Any not so stable marking proba= bility results in a rate between those 2 boundaries. This is the theory, lo= oking forward on how your practice matches this. >From a first quick look at the results it might not deviate that much, I gu= ess. The more aggressive part is probably due to the RTT unfairness: 1 to 5= rate ratio (with a 5ms base RTT, Classic gets a queue of 5ms+20ms target, = so about 25ms and 5 times less throughput). Did you try to use the RTT inde= pendent version? If you set it to the f=3Dmax(15, RTT) mode, the minimum ef= fective RTT becomes 15ms, so the rate ratio is 3 to 5 in that case. Thanks and Regards, Koen. From: tcpPrague > On Behalf Of Szilveszter Nadas Sent: Tuesday, November 3, 2020 6:30 PM To: tcpprague@ietf.org Cc: Ferenc Fejes >; Gombos Gerg= o >; LAKI Sandor > Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP P= rague preliminary results Hi all, During the review of our article "A Congestion Control Independent L4S Sche= duler" we received comments from one of the reviewers, on why we did not al= so evaluate TCP Prague. So now we rerun the test cases with replacing DCTCP= to TCP Prague. The results are at the below link. We are somewhat surprised by much increa= sed aggressiveness of TCP Prague. Can you comment on it? The settings and s= etup we used are described in the pdf (and in the original article). Results: http://ppv.elte.hu/tcp-prague/ Original article (including YouTube presentation): http://ppv.elte.hu/cc-in= dependent-l4s/ Can you comment on the results and the settings we used? Cheers, Szilveszter --_000_AM8PR07MB7476958FF16BA93CCE92433FB9F30AM8PR07MB7476eurp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Szilveszter,

 

First thing is to check that you can reproduce our r= esults. With the previous (non-RTT-independent) Prague, can you confirm you= get the following results?

 

Normally with the right coupling (p_C =3D p’^2= and P_L =3D 2*p’) and if their total RTT is the same, Cubic (in Reno= mode) and Prague get about the same throughput (Prague might get up to twi= ce the throughput max). Total RTT means base RTT + all Queuing delay for each flow’s round trip path. Can you confirm t= his, or do you see a bigger difference?

 

One way to check this on an equal base RTT setup is = to use a singleQ PI2 which is doing the  correct marking or dropping a= ccording to the above equations, because the single Q  gives both clas= ses the same latency (single Q =3D same Q delay). Another way is to use a DualQ and compensate the 15ms Classic latency with= a 15ms bigger base RTT for Prague. In both cases you would see them gettin= g around the same throughput. If the BDP (rate and/or RTT) becomes bigger, = Cubic will start to get a higher throughput than Prague.

 

Note that if you install the latest Prague version f= rom our L4STeam git, the default is now to be RTT independent below 25ms. S= o any Prague with a lower than 25 ms throughput will behave as a 25ms RTT f= low (effective after 500 RTTs), and will get the same throughput as a 25ms RTT flow. You could test this with = Prague on a DualPi2, when competing with a Cubic flow with a base RTT of 10= ms. All RTTs (you can try even a mix of different ones in parallel) below 2= 5ms should now get the same throughput as the Cubic one (because it has also a total RTT of 25ms).

 

Any other effective RTT combinations different than = the equal ones should follow the “throughput ratio equals the inverse= RTT ratio” rule. One exception is when non-Rtt-independent Prague fl= ows with different RTTs ares competing on a STEP AQM. In that case the difference is bigger than that ratio, because the lo= west RTTs are getting on-off marking, and behave as 1/p=B2, instead of 1/p = as they will get a more stable marking probability over their (much bigger)= RTTs.

 

Let me know if you can reproduce the above behavior.= If not, maybe try using the Linux DualPI2, and if still not as expected, t= hen we need to dive deeper to find the reason.

 

Thanks,

Koen.

 

 

From: Szilveszter Nadas <Szilveszter.Nadas= @ericsson.com>
Sent: Wednesday, December 2, 2020 5:05 PM
To: tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; Gombos Gergo <ggombos= @inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>; De Schepper, Koen = (Nokia - BE/Antwerp) <koen.de_schepper@nokia-bell-labs.com>; Tilmans,= Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-labs.com> Subject: RE: [tcpPrague] A Congestion Control Independent L4S Schedu= ler - TCP Prague preliminary results

 

Hi Koen and  Oliv= ier,

 

Thanks for the answers= .

 

Some follow-up questio= ns:

 

=3D=3DRTT unfairness a= s 1:5 ratio.=3D=3D

Is there a way to comp= ensate it in the DualPI2 scheduler? My understanding was that with the righ= t parameters it is compensated “automatically”, there was good = fairness among DC and Cubic flows in the original (Dual)PI2 papers. I guess that for single PI2 that was because of the shar= ed queue. But what about DualPI2? Can it be configured in a way that  = fairness remains also considering queueing for Cubic? (I vaguely remember a= description of this in one of your papers, but I cannot find it now). I guess that the good fairness with DCTCP vs. C= ubic was an actual anomaly, not the worse fairness with Prague vs. Cubic?

 

=3D RTT indepen= dent Prague version=3D

Can you provide a poin= ter how to configure that? Are the TCP Prague CC parameter defaults in the = version we used not meaningful? Shall we change other parameters?

 

Cheers,

Szilveszter=

 

-----Original Message-----
From: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-labs.com>
Sent: November 5, 2020 12:10
To: Szilveszter Nadas <Szilveszter.Nadas@ericsson.com>; tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu= >; Gombos Gergo <ggombos@i= nf.elte.hu>; LAKI Sandor <la= kis@inf.elte.hu>
Subject: RE: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results

 

Hi Szilveszter,

 

 

 > The results are at the below link. We = are somewhat surprised by much  > increased aggressiveness of TCP P= rague. Can you comment on it? The settings  > and setup we used are= described in the pdf (and in the original article).

 

Thanks for sharing these.

 

* Short answer:

DCTCP actively hurts itself, tripping the step AQ= M unnecessarily.

This is mostly benign in DC environments due to t= he almost non existent RTT, and small cwnd, but causes observable performan= ce hits elsewhere.

Prague implements fixes for these issues.

 

* Long answer:

Please find below the main changes from dctcp to = prague. The intuition behind these is to avoid triggering the step AQM part= of the dual queue when there is classic traffic, or limit the extent of th= e received marks.

The first goal ensure that, when there is classic= traffic, prague only receives marks from the coupled PI2 component, which = are strong enough on their own to keep the L4S queue empty most of the time= . Additionally getting marks from the step AQM means that prague is hurting itself, as it will reduce more t= han necessary. The second goal ensures that prague does not unnecessarily d= rive the AQM (either PI2 or the step) to high-marking rate; again because i= t will hurts its throughput and also because it will create a standing queue.

 

Note that depending on your experimental setups, = not all the below factors will contribute.

 

1. Prague is always paced, regardless of the egre= ss qdisc on the data sender,

   and paces itself at 100% of the esti= mated BDP.

        &= nbsp;       DCTCP is only paced when used in = combination with fq, and paces at 120%

        &= nbsp;       of the BDP when that is the case.=

2. Prague actively limits its gso bursts (default= s to 250us)

        &= nbsp;       DCTCP uses Linux defaults, 1ms

3. Prague implements a more accurate internal mar= king estimate (alpha)

        &= nbsp;       DCTCP, especially when operating = with low marking probabilities, will

        &= nbsp;       tend to push down alpha to 0, del= aying its response to marks (and thus

        &= nbsp;       increasing queue pressure/causing= larger reduction down the line) 4. Prague carry over sub-cwnd reduction, s= uch that multiple marks in a row

   occurring at low overall probability= will eventually cause a cwnd reduction

        &= nbsp;       DCTCP's cwnd reduction code has n= o memory, i.e., as long as alpha is

        &= nbsp;       low enough compared to cwnd, no c= wnd reduction will ever occur. This

        &= nbsp;       again drives the aqm to larger ma= rk probabilities.

 

All of these points contribute to prague being mo= re gentle towards the queue, and more reactive to the received marks (i.e.,= achieving a lower overall marking level)--the direct effect being smaller = cwnd reduction (thus sawtooth) than DCTP, which is critical when operating at higher e2e RTTs with a shallow q= ueue since it lowers the time to recovery.

 

Additionally, when operating over a long RTT path= and controlling by PI2 (i.e., random marking, such that it receives marks = almost every RTT--as opposed to a step where it receives nothing for severa= l RTTs and then a RTT with most of the packets being marked), DCTCP suffers from the inaccurate/memoryless co= mputations in 3./4. and its interactions with Linux's PRR code (which pragu= e does not use), which can prevent it from ever reaching its expected rate.=

 

I hope this helps to understand a bit better why = prague appears to be more aggressive--it is actually the opposite: dctcp sy= stematically hurts itself at longer base RTT, hence progressively give way = as the RTT increases.

You can validate this by observing that the expec= ted rate ratios between classic flows and prague over the dualQ match the t= heoretical equations (compounding the queue delay difference) even that hig= her base RTTs, which is not the case for DCTCP.

 

 

Best,

Olivier

 

 

From: De Schepper, Koen (Nokia - BE/Antwerp) = <koen.de_scheppe= r@nokia-bell-labs.com>
Sent: November 12, 2020 10:35
To: Szilveszter Nadas <Szilveszter.Nadas@ericsson.com>; tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.= elte.hu>; Gombos Gergo <gg= ombos@inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: RE: [tcpPrague] A Congestion Control Independent L4S Schedu= ler - TCP Prague preliminary results

 

Hi Szilveszter,

 

Interesting work, thanks for sharing. There is indee= d a very big difference between how DCTCP behaves today in recent kernels, = than what we used in the original L4S work testing (Linux kernel 3.19). Tha= t original version was a “clean” DCTCP version that matched very well the theoretical equations: r=3D2/(p.RTT) fo= r uniform stable random and r=3D2/(p=B2.RTT) on/off marking. Due to many in= teractions, integer range constraints, pacing/GSO interactions and bugs, th= e performance really degraded in the recent Linux versions. These issues were fixed within the Linux Prague version, s= o that is why Prague should perform “better” in respect to matc= hing the equations. It would be interesting to get an idea of the deviation= from the r=3D2/(p.RTT) equation in your tests (relevant in DualPI2 as the coupling gives a smooth probability). Collecti= ng the average RTT (base + queuing latency), marking probability, and rate = would make it possible to evaluate this.

 

Main reasons why we saw deviations is when the proba= bility varies a lot in time. As you might know DCTCP’s (and Prague= 217;s) equation becomes r=3D2/(p=B2.RTT) when it receives on/off marking ep= isodes (on episodes in the order of 1 RTT, off in the order of multiple RTTs). Any not so stable marking probability results= in a rate between those 2 boundaries. This is the theory, looking forward = on how your practice matches this.

 

From a first quick look at the results it might not = deviate that much, I guess. The more aggressive part is probably due to the= RTT unfairness: 1 to 5 rate ratio (with a 5ms base RTT, Classic gets a que= ue of 5ms+20ms target, so about 25ms and 5 times less throughput). Did you try to use the RTT independent versi= on? If you set it to the f=3Dmax(15, RTT) mode, the minimum effective RTT b= ecomes 15ms, so the rate ratio is 3 to 5 in that case.

 

Thanks and Regards,

Koen.

 

From: tcpPrague <tcpprague-bounces@ietf.org> On Behalf Of Szilveszter Nadas
Sent: Tuesday, November 3, 2020 6:30 PM
To: tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.= elte.hu>; Gombos Gergo <gg= ombos@inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler = - TCP Prague preliminary results

 

Hi all,

 

During the review of our article “A Congestion= Control Independent L4S Scheduler” we received comments from one of = the reviewers, on why we did not also evaluate TCP Prague. So now we rerun = the test cases with replacing DCTCP to TCP Prague.

 

The results are at the below link. We are somewhat s= urprised by much increased aggressiveness of TCP Prague. Can you comment on= it? The settings and setup we used are described in the pdf (and in the or= iginal article).

Results: = http://ppv.elte.hu/tcp-prague/

Original article (including YouTube presentation): <= a href=3D"http://ppv.elte.hu/cc-independent-l4s/"> http://ppv.elte.hu/cc-independent-l4s/

 

Can you comment on the results and the settings we u= sed?

 

Cheers,

Szilveszter

--_000_AM8PR07MB7476958FF16BA93CCE92433FB9F30AM8PR07MB7476eurp_--