From nobody Sun Nov 1 01:22:50 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 06F5D3A0BFF; Sun, 1 Nov 2020 01:22:47 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qG_RAp4egz_b; Sun, 1 Nov 2020 01:22:44 -0700 (PDT) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 15B6C3A0F44; Sun, 1 Nov 2020 01:22:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1604218917; bh=vAQ9G9RT5tFNZ7nBLVIw7h0k39pfBJYel7VkMZjQoPw=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=CSlHSeqR2pUBtxuf1HgatJOqhi+SVKspK/s8fYEPQ59e4liCeprTypDHwPgb0DzeY ib6EOvwWlJlbADhBLivGmiGUwtdOptQQbx3FjX7Erwer5lUa4+B+1OWSAxSvyasshY WOnR5z0pqs1f+lsYTnu6BDh3XWHloWx6mZa4sqhY= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [134.76.241.253] ([134.76.241.253]) by web-mail.gmx.net (3c-app-gmx-bs25.server.lan [172.19.170.77]) (via HTTP); Sun, 1 Nov 2020 09:21:56 +0100 MIME-Version: 1.0 Message-ID: From: Sebastian Moeller To: "De Schepper, Koen (Nokia - BE/Antwerp)" Cc: Bob Briscoe , tsvwg IETF list , iccrg IRTF list , TCP Prague List Content-Type: text/plain; charset=UTF-8 Date: Sun, 1 Nov 2020 09:21:56 +0100 Importance: normal Sensitivity: Normal In-Reply-To: References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:UnDdnhzY92GHS54M6p+LEWXEvwEgARG1+OA+qMqnfrO1kj9g2+T0C6kL8DlKgXqtB52BM IwL7FMe1QgpRBCU8q7e+tM83jQioKzIaDtmkxNsDN77bu3wwjYOIqo3u2kuooPcli15kjbZiSezt 0wJ1+B3StJvJQsLpDnGkiav4uGeTwmzsmfLTOtXY/UTfGs1RGo7WWiXwL3kBJYqFiA0pGpZACXVG 0fMZIWxQx9jl2xvd4n0HA8mK3uN+Ee/Vbwf1DxFmEQF0Ig+BUsAq4cUGG+2Mr8NEj1sU7nErVUC0 Ig= X-UI-Out-Filterresults: notjunk:1;V03:K0:kZB4Yzj59W8=:pYtmQJlERaCOllVTBLEP8W AScrmsAF5mQBytXtRluHITEyFtP2NwKB2JevsSDS+w1vlikOibfl2bD4bXhujFm/piqbeB9oW ZkISbZb+pfJN++kqlRnOKg6GciCtCEF0n86kiOmBlzkVTXtAjWvIOL8WQRqnAi8THGLQ726B9 f81SU/8nwAv/kMH7Gg4jnTUku6zebXrd4imf4ophUml7sSJLLlMXFs5XB/7Y5eQ3pTzSBOEFQ 77hMFwsMrOfHwBHEcaHVNWSmCp2fQnLWN/r4NMhmUaHXdm5JWqI5bTSvfe1aUpFwQdvBapBiA avpLtTT6kh5j7dAzD7MFMufsyQRj3NVPq4yXvbbtPb7ZlEoXZcrfHFFIjECMAbLissfR1nPat 2RkdXs1RWkBtvFOeqTbQshACWdQwnUp/DsbKI8v0NitbIjzSPpD7/9g7MWYsV5/gF3B8w62cG +uY3LN6gKzdEwH/2WHO8KEO1du9GcSYkmkfZsuaZKrGEouIYebzAGxT/1XryekN3e4D4wzNXj aATwWnaX/vYxxG5gjX+cNvjTUuHOvq1of4jO3OC0W9K6Mwfs6MpTYrtUrUkM77ILp9B62CEvQ YONv2JKn5xfBQ= Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 08:22:47 -0000 Hi Koen, more below in-line, prefixed [SM2] =C2=A0 =C2=A0 =C2=A0 Gesendet:=C2=A0Sonntag, 01=2E November 2020 um 00:42 Uhr Von:=C2=A0"De Schepper, Koen (Nokia - BE/Antwerp)" An:=C2=A0"Sebastian Moeller" , "Bob Briscoe" Cc:=C2=A0"tsvwg IETF list" , "iccrg IRTF list" , "TCP Prague List" Betreff:=C2=A0RE: [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classi= c ECN detection Text Hi Sebastian, =C2=A0 >> I appreciate that team L4S is finally admitting that it over-promised a= nd under-delivered =C2=A0 I think the problem is that you interpret =E2=80=9Crequirements=E2=80=9D a= s =E2=80=9Cpromises=E2=80=9D=2E=20 [SM2] I disagree, with promises, I typically address the claims ab= out what L4S will achieve=2E The stated goal was that L4S will do no harm t= o non-L4S traffic (by e=2Eg=2E adapting to rfc3168 bottlenecked paths by sw= itching away from its CE interpretation) but the proposed text, basically c= hanges that into a "the operator is free to do as she/he pleases"=2E That i= s a considerable reduction in the promise to do no harm=2E Or are you saying that "do no harm" was never a design goal for L4S = but only adopted as it was dragged in by external requirements? If true, it= would explain a lot=2E Most people understand the potential of L4S=20 [SM2] Most people have not even heard about L4S, and even in the g= roup of people that have, the majority probably from reading your internet = drafts without actually looking into the few papers presenting data tht all= ows to evaluation whether the L4S drafts are realistic=2E IMHO that means, = that quite a lot of people that know about L4S understand its promises=2E N= ow, if by potential you admit uncertainty whether this potential can actual= ly be realised in the real-world internet then I agree=2E I am just convinc= ed that a more productive path forward would have been to work harder on fi= guring out whether L4S potential can be realised, before trying to accelera= te the ratification proces=2E and accept that special care needs to be taken to protect Single Q Classic= ENC bottlenecks=2E [SM2] But the proposed change actually deemphasises this, from a m= andate to protect, it now is a mandate to detect and if one pleases to prot= ect=2E This seems like an odd way to protection=2E The draft needs to put the attention to (potential) problems that need to= be solved or avoided by interesting parties that want to develop their L4S= Prague-compliant CCs=2E So it is in the best interest for CC developers to= push for feasible/realistic requirements=2E=20 [SM2] Well, the realistic part would have been something like "A s= calable congestion control MUST do its best to react to ECN marking from a = non-L4S but ECN-capable bottleneck in a way that will coexist with a TCP Re= no congestion control" The change to react from "MUST react" to "SHOULD be = capable (dependent on configuration) of automatically adapting", does not s= trike me as an improvement in either feasibility nor realism, it is simply = a relaxation of a requirement, that proved harder to achieve than expected,= no? I heard quite some people (including you) being concerned about the feasib= ility of detecting a Single Queue Classic ECN AQM,=20 [SM2] Well, testing data indicated, that the proposed detector was= /is riddled with false detections and missing detection, but the quality of= the detector is orthogonal to what to do with the result of said detector,= so please do not conflate these issues=2E and said they would rather apply other mechanisms to avoid issues with the= se types of bottlenecks (ref operational guidelines that tries to collect a= ll useful ideas that were shared during list discussions)=2E=20 [SM2] To which, I have sent comments, against my better judgement = of that being busy work=2E So the changes announced by Bob should reflect this=2E I understood you ar= e quite OK with the wording? If not, then your suggestions are very welcome= =2E [SM2] No, I am not not okay with Bob's proposal, and I wonder how = you could come to that conclusion=2E As long as that issue exists (because = the meaning of CE is redefined in a rfc3168 incompatible fashion) L4S proto= cols, IMHO, "MUST adapt" which also includes a requirement that they MUST b= e capable=2E I hope that this is explicit enough to ne being mis understood= as an endorsement for the proposed change=2E I explicitly stated "make bot= h of these MUSTs" which clearly indicates that I disagree with the relaxati= on of the requirements, even though I admit, it does not cover my full crit= icism=2E Again, how you can interpret that as being " OK with the wording" = puzzles me to no end=2E =C2=A0 >> "In the unlikely case of the L4S experiment being declared a failure, t= his replacement will need to be permanent, and the ECT(1) responsive elemen= ts in AQM nodes also needs to be disabled"=2E =C2=A0 L4S will succeed if 1) L4S capable network nodes get deployed (including C= lassic ECN bottlenecks being upgraded) and 2) CCs are deployed that can mee= t the TCP Prague requirements (is not disrupting Classic flows and other L4= S flows) AND can show great benefits=2E If one of those or both will not ha= ppen (like with Classic ECN deployment) it fails!! So I guess no need to wo= rry about disabling permanently if it never gets deployed =F0=9F=98=8A=2E [SM2] Well, let' try logic, for a change, let's ignore the wishful= thinking of "including Classic ECN bottlenecks being upgraded" as L4S need= s to prove its merits in the existing internet, not a hypothetical one, if = L4S nodes get deployed 1) and L4S compatible CCs also get deployed, that st= ill leaves the point that they need to "show great benefits"=2E If that imp= ortant result will not be achieved than disabling the AQM and CC nodes alre= ady deployed seems an important measure to keep a failed experiment from a)= wasting a precious IP header code-point ad infinitum and to keep complexit= y of TCP stacks low=2E Again I fail to understand how your argument can be = considered logically convincing=2E Best Regards Sebastian =C2=A0 Thanks and regards, Koen=2E =C2=A0 From: Sebastian Moeller Sent: Saturday, October 31, 2020 3:54 PM To: Bob Briscoe Cc: tsvwg IETF list ; iccrg IRTF list = ; TCP Prague List ; De Schepper, Koen (Nokia - BE/Ant= werp) Subject: Aw: [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN= detection Text =C2=A0 Dear Koen, =C2=A0 since Bob claims that he intends to correspond with my input, I am address= ing this response to you, See my comments in-line prefixed [SM]=2E =C2=A0 Tl;dr: I appreciate that team L4S is finally admitting that it over-promis= ed and under-delivered, I am less happy about the solution to this issue by= simply reducing the requirments ot match the unsatisfactory state of the L= 4S implementation=2E After years of basically adveritizing on these require= ments, watering those down at the last minute does not qualify as a good fa= ith effort in my book=2E It does confirm my negative view on team L4S' acum= en and confirms my judgement that L4S "offers too little too late"=2E Unfor= tunately I do not kid myself that this obvious fact is going to stop the cu= rrent intenet drafts gaining RFC status without any meanigful changes=2E=C2= =A0 =C2=A0 =C2=A0 Gesendet:=C2=A0Samstag, 31=2E Oktober 2020 um 10:54 Uhr Von:=C2=A0"Bob Briscoe" An:=C2=A0"tsvwg IETF list" Cc:=C2=A0"iccrg IRTF list" , "T= CP Prague List" , "De Sc= hepper, Koen (Koen)" Betreff:=C2=A0[tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic EC= N detection Text Folks, The co-authors of ECN L4S ID have been reviewing the correctness of the no= rmative 'Prague' requirements=2E =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0[SM] This seems less about=C2=A0correctne= ss and more about whether you managed ot hit those targets with your implem= entation=2E Changing those requirements post-hoc might be justified, but pl= eas do not frame that as a correctness isssue=2E =C2=A0 =C2=A0=C2=A0=C2=A0 See https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-ec= n-l4s-id-10#section-4=2E3[https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-= ecn-l4s-id-10#section-4=2E3] This is the second of 2 emails, about 2 of the requirements that we think = ought to be reworded a little=2E If you agree with the rationale, but think the new wording still doesn't f= ully capture the requirement, pls suggest sthg better=2E If you disagree with the rationale, pls discuss=2E 4=2E3=2E=C2=A0 Prerequisite Congestion Response =2E=2E=2E CURRENT: =C2=A0 =C2=A0=C2=A0 o=C2=A0 A scalable congestion control MUST react to ECN marki= ng from a =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 non-L4S but ECN-capable bottleneck in a way= that will coexist with =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 a TCP Reno congestion control [RFC5681[http= s://tools=2Eietf=2Eorg/html/rfc5681]] (see Appendix A=2E1=2E4[https://tools= =2Eietf=2Eorg/html/draft-ietf-tsvwg-ecn-l4s-id-10#appendix-A=2E1=2E4] for =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rationale)=2E =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0Note that a scalable congestion control is = not expected to change =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to setting ECT(0) while it falls back to co= exist with Reno=2E =C2=A0 PROPOSED: =C2=A0=C2=A0 o=C2=A0 A scalable congestion control MUST implement monitori= ng in order =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to detect a likely non-L4S but ECN-capable = AQM at the bottleneck=2E =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 On detection of a likely ECN-capable bottle= neck it SHOULD be =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 capable (dependent on configuration) of aut= omatically adapting its =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 congestion response to coexist with TCP Ren= o congestion controls =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [RFC5681] (see Appendix A=2E1=2E4 for ratio= nale and a referenced =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 algorithm)=2E =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [SM] This is a significantly watering down, an= d it seems rather non-logical, IFF an implementation does not want/need to = actually react to the result of a detection then performing that detection = seems=C2=A0pure busy work=2E So make both of these MUSTs=2E Otherwise your = requirments pretend to care about existing rfc3168 AQMs but in name only=2E= =C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Note that a scalable congestion control is = not expected to change =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to setting ECT(0) while it falls back to co= exist with Reno=2E RATIONALE: 1/ The requirement as currently written says what an omniscient sender MUS= T do=2E So there's an implied requirement that a sender MUST be omniscient,= which is of course impossible=2E 2/ The requirement needs to be recast to require a sender to aim to be as = knowledgeable as possible=2E Then, what it does as a result needs to take i= nto account the a priori likelihood of there being a non-L4S bottleneck pre= sent=2E 3/ This includes the possibility that the operator of the host knows that = the network it serves has not deployed any single queue classic ECN AQM (e= =2Eg=2E in a CDN case they're doing out of band testing, or they've asked t= he ISP)=2E So we've included the possibility of fall-back being disabled by= configuration=2E 4/ Nonetheless, as has been pointed out on the list, there is still a poss= ibility that there is a Classic ECN AQM somewhere else on the path (to cont= inue the CDN example, perhaps beyond the ISP in a home network)=2E The 'MUS= T monitor' requirement still stands to ensure the operator doesn't miss the= se cases=2E 5/ Then, if the server operators have disabled fall-back for their deploym= ent, they can reconsider their policy or at least do more focused testing i= f they are frequently detecting a single-queue Classic ECN AQM=2E =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [SM] And we are back to engineering/network ad= ministration by wishful thinking=2E=2E=2E Again with a syszem (L4S) that ha= s not yet been proven to actually work over the existing internet with the = sole exception of short RTT/low hop count paths (CDN to end-user fast-lanes= copme to mind)=2E Items 3-5 are the "react via management" model that I've talked about on t= he list, given the unfairness doesn't amount to starvation, and it is possi= ble that the prevalence of the problem is very low=2E =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [SM] Given that you never offered your own spe= cific defintion of what starvation entails, I fail to see how that claim ca= n b tre in a verifiable fashion? Finally, after the bullet list of requirements in section 4=2E3, (which ar= e prerequisites for setting the ECT1 codepoint), we propose to add the foll= owing requirement, as suggested on the tsvwg list: =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 To participate in the L4S experiment, a sca= lable congestion control MUST =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 be capable of being replaced by a Classic c= ongestion control (by =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 application and by administrative control)= =2E A Classic congestion control =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 will not tag its packets with the ECT(1) co= depoint=2E =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [SM] +1; I would appreciate if we could add a = sentence like: =C2=A0 "In the unlikely case of the L4S experiment being declared a failure, this= replacement will need to be permanent, and the ECT(1) responsive elements = in AQM nodes also needs to be disabled"=2E =C2=A0 I added the unlikely in spite of my own predictions, to account for the fa= ct that the drafts are written on the hypothesis that L4S works as advertiz= ed=2E=2E=2E=2E=C2=A0 =C2=A0 Best =C2=A0 =C2=A0 =C2=A0 =C2=A0 Sebastian =C2=A0 Cheers Bob =C2=A0 -- ________________________________________________________________ Bob Briscoe=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 http://bobbriscoe=2Enet/[http://= bobbriscoe=2Enet/] From nobody Sun Nov 1 01:55:57 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 420AC3A100A; Sun, 1 Nov 2020 01:55:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8XRCBAeA_TGM; Sun, 1 Nov 2020 01:55:52 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 553F53A0976; Sun, 1 Nov 2020 01:55:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1604224507; bh=unF9f2Fnl31oD7szxkCT7ywkabzLt7gLW/aCaTUxwRM=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=gKRU+tWWsV84ih1dRrz3sbzQ9S0G481rJuFX1E8tN9mPTxNBo3sMh+QK44PKwdRMd 7LBeSpkgdGmWMJBoCaxvbAVvkfttt9uCh4Fp7LeHN81tri0A2Ir3ltecy8LGggeNmi Ix/GnpA9Aa09q+KNaqpNvr+npk5FlvsvfvZpq7gk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [134.76.241.253] ([134.76.241.253]) by web-mail.gmx.net (3c-app-gmx-bs25.server.lan [172.19.170.77]) (via HTTP); Sun, 1 Nov 2020 10:55:07 +0100 MIME-Version: 1.0 Message-ID: From: Sebastian Moeller To: "De Schepper, Koen (Nokia - BE/Antwerp)" Cc: Bob Briscoe , tsvwg IETF list , iccrg IRTF list , TCP Prague List Content-Type: text/plain; charset=UTF-8 Date: Sun, 1 Nov 2020 10:55:07 +0100 Importance: normal Sensitivity: Normal In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:FU59iJhQm2QvieiB1I6mJOyL8zyJt1PfVs4WoFDTS3RiPZp+kH9fSNEmTLrQThSGgV8iQ OygEykakpWS5IjoPTSPNFQoDtdt82GUSMYr/1k/DZ/D7reawUC8bDGOot/oFS7USKTV/UyEPwVi4 5HomGVYjTFSDz8JpcRJikxa0G92yiZffawb9L+6jkakd1+BlAl+z61MrkfCIIF9GCTeeQV5TyLfT jA5InUJvxI40Z1HZS9HyTtYbZ+jTuFYfXcNk2SoKo6EE0LUp6zPQs4tmIbMFhT23mGBDVBhtXRa1 WQ= X-UI-Out-Filterresults: notjunk:1;V03:K0:nnCDNn3S06U=:lfTmh4f/cXcYum6wBwB67E 6FudlZkBE4Fcd75+xoL7QUxR4Y7AFCs6mAmXoA4sw4U1kFwR+8XuLYBCQXZ9wJo2G9NqRe3Sv NftD/95XpZXG18OcfRsTQvS3nuMsm6q/tYaDyveM7DraA92eg/qVS42CRMQ27HLTl6EU519Fv xflgrq3wezB35nq4AzfX5+6sQkA7KS52mPwCN/4hhSPcCzbMTqjUL2HSLropERcS9DRATfZeT V86Ru4oMxIAUrd/STY2eaNIKgvFVu1akg+ok2deQmtkhPWCC8rmRj1AwiKxmU1tC17YwqnmxJ +larX55y+NfM4TpKcqVH8h2PwQaaOngJi0dB1gDVBDzWBKudvqzDT7lCa+iUbGLjv4tImcpYI gRMtUedLoabPtFRz2JZkcVggldBVGb5zYnoBiIiw4N/iEQ+A0irCAMEgBkXecEXMB5gTnQ5Z8 c7pXg1F530+tn+lQgRF2sTE/45jW/itrWNzSPVtfDK/i4wWIqaLRR0FV3NZWSIqm4D0JJ+t98 37ZDdhphDMfJqH64eI1HlS/dFIUG179Dtt8qVo1CkKa0ZjFDitLjzTvlGTe62izpJRz93VSq1 VAsq7ByY18FDU= Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative RTT Bias Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 09:55:55 -0000 Hi Koen, more below in-line, prefixed with [SM] =C2=A0 =C2=A0 =C2=A0 Gesendet:=C2=A0Samstag, 31=2E Oktober 2020 um 23:33 Uhr Von:=C2=A0"De Schepper, Koen (Nokia - BE/Antwerp)" An:=C2=A0"Sebastian Moeller" , "Bob Briscoe" Cc:=C2=A0"tsvwg IETF list" , "iccrg IRTF list" , "TCP Prague List" Betreff:=C2=A0RE: [tsvwg] ecn-l4s-id: Proposed Changed to Normative RTT Bi= as Text Hi Sebastian, =C2=A0 It was never our intent to develop the Congestion Control itself, as it wa= s not in our interest (as a network provider)=2E=20 [SM] I understand that, but it turns out that to asses functionali= ty and safety of the proposed changes to ECN signalling you absolutely requ= ired a way to actually test under realistic conditions, and hence your hand= was forced if ever to succeed=2E As a consequence TCP Prague developed int= o your umping ground for features you did not really want but had to accept= =2E=2E=2E like the abysmal 15ms hack (good luck getting that into any OS ne= tworking stack)=2E There are plenty of bright people that can develop a Prague-compliant CC t= hat will benefit their business (and at least comply to the safety measures= )=2E=20 [SM] But it still is incumbent upon you top demonstrate that the r= equirements are actually achievable and sufficient to guarantee functionali= ty and safety (as that is the prove that L4S actually would work)=2E I unde= rstand that most of the little conditions tested used DCTCP as protocol, so= even for the one condition tested so far, short-RTT low HOP-count paths, m= ost tests will need to be repeated with TCP Prague, not that I am holding m= y breath to see those results, given that I outlined a number of completely= missing tests to evaluate functionality and safety outside the narrow cond= itions tested so far=2E >From the start we realized that the small buffer is a potential safety iss= ue for the typical r~=3D1/RTT congestion controls (also confirmed by our ea= rly tests and later by Pete=E2=80=99s tests too)=2E [SM] IIRC, that was a well established fact and not a terribly nov= el insight=2E=2E=2E We hoped by showing that (once one puzzles out how) making a CC RTT indep= endent for the unsafe range (small RTTs) is an easy obvious hack=2E To plea= se you, we even made a =E2=80=9CSebastian-version=E2=80=9D with f(rtt)=3Drt= t+15ms that only removes the RTT benefit from the DualQ (one of your main c= omplaints), and behaves further as a Classic RTT dependent flow=2E The Seba= stian-version solves the safety issue and your fairness issue, but is not o= ur preferred version=2E [SM] I do take offence in that name Sebastian-version, and in pres= enting that hack as anything but a work-around for a DualQ design error=2E = As I explained patiently in the past, normal RTT dependence of TCP throughp= ut is not a function of the endpoints (as as long as a single flow along a = path can saturate the link, the end-points obviously are capable of reachin= g their goal independent of RTT, hence are RTT independent, I know this is = within reason, as there are limits on the aviebale throughput depending on = RTT and maximal window sizes, but these are not the issues at play here) bu= t of the buffering in the network, most importantly the buffering at the bo= ttlenecks=2E And hence that issue should be solved there=2E BUT to add insult to injury, you are only proposing to make up for the 15m= s DualQ might be configured with (might be as last time I looked at the cod= e it looked more like classic's target was configured to 10ms, but I digres= s)=2E We like more the one that solves the safety issue with f(rtt)=3Dmax(rtt, = 15ms), and still allows L4S to benefit from the lower delay in all other ca= ses=2E [SM] This is not a safety issue=2E It is a rate throttling issue= =2E But humour me, how does curvy RED actually perform with all of these gr= oss hacks, under which version does it achieve equitable sharing?=20 =C2=A0 Then thanks to your insisting to realizse literally what we have written,= =20 [SM] So you intended to formulate requirements in your drafts as l= oose guiding principles without actual interpreting these as binding defini= tions? And you needed my input to realise basically what "requirement" mean= s=2E Koen, I have a hard time believing that=2E=20 we realized that the text was indeed too strict if read out of the larger = context=2E In the context of the safety requirements section, we never want= ed to enforce RTT fairness for flows with a bigger than the reference RTT, = as they don=E2=80=99t pose a safety issue (it is a reality for all longer R= TTs today, partially solved by Cubic already), and the quoted text could in= deed be interpreted that way out of the larger context=2E Blindly trying to= implement RTT independence for larger RTTs could even make it unsafe, if n= ot done correctly=2E Of course, if wanted and beneficial, others are free t= o further optimize=2E =C2=A0 [SM] You have been advertising L4S on pie-in-the-sky features like= RTT-independence for years, in spite of having been told about it impossib= ility before, and now you seem to finally understand that=2E Again, I have = a hard time believing that=2E I am also puzzled how this fits with how engi= neers typically are thought to work=2E So I hope you understand now why we want to fix this text (and don=E2=80= =99t see this as a failure), and hope you are ok too, to keep this safety r= equirement to the safety aspect only=2E=20 [SM] No, as I stated in the past, the 15ms issue, is a DialQ issue= , and needs to be fixed in DualQ, then the RTT-independence requirement can= be kept as purely inspirational=2E The current version with its hedge "as = wide a range of RTTs as possible" is IMHO still better than the new "the mi= nimum likely RTT and typical RTTs expected in the intended deployment scena= rio" because that is plainly wrong=2E By artificially coupling the CC respo= nse and the AQM (with the brain dead 15ms) your AQM no MUST not account for= likely or expected RTTs but it MUST account for what the CCs are configure= d for, otherwise the uneqitable sharing issue will crop up again=2E Again this is a logical conclusion based on the unfortunate layer v= iolating design choices you made, so please do not frame this as an issue o= f RTTs=2E IMHO, the fact that DualQ has this issue is a clear indication, th= at the cute coupling idea, simply is not suitable for the messy reality, an= d hacking around that by a requirement for transport protocols is insane, B= UT then not clearly pointing out the dependency in the requirement section = like "the CC's temporal response dynamic lower bound needs to be configured= to match the L4S AQM expected on the path, or 15ms by default" is not goin= g to make protocol implementers understand this subtle issue, as this is NO= T an RTT bias issue to start with that is supposed to be covered by this re= quirement, but a DualQ misdesgn=2E Hiding this relations does not help anyb= ody=2E If you think the text is still not clearly representing the safety require= ment of the RTT independence aspect, then your suggestions are very welcome= =2E =C2=A0 [SM] If you truly believe that, then I fear you have not understoo= d the issue very well=2E Try to read this recommendation from the perspecti= ve of a naive protocol implementer=2E How will she know that she needs to l= ook into the code/default configuration of the DualQ to be able to properly= define the response dynamics lower bound, heck how is she supposed to unde= rstand that she needs to artificially slow down the CC response for some RT= Ts and make the CC control loop actively worse from the text of the require= ment=2E IMHO the way this is phrased tries to hide the actual causalities a= t play here, and that never is a good idea in engineering or science=2E As to your request to let the AQM make flows RTT independent, that is unfo= rtunately impossible without knowing the RTT of each packet or identifying = and treating the flows individually=2E=20 [SM] Well, it is less impossible than for the end-points=2E As lon= g as you keep that requirement at all (and I can agree with getting rid of = of it, as it is impossible to achieve realistically) you IMHO NEED to solve= it at the component/layer responsible for that issue=2E Arguing it is hard= for the AQM does not hold water as it is IMPOSSIBLE for the end-points in = any useful generalized fashion=2E Giving the current proposals, I can almos= t guarantee that none of alternative transport protocols to TCP Prague will= implement that 15ms hack at all (as the requirement is sufficiently vague= to make it impossible to understand what is actually going on unless the i= mplementer followed the L4S saga on this list*), and hence your safety issu= e will not be addressed at all=2E Again, I have a hard time believing that = you assume that the changed text can be interpreted correctly without a lot= of unwritten information, so how about you add a description of how a errr= in DualQ makes a 15ms recommended? So can you explain why you claim that what actually is easy (RTT-indep CC)= is impossible and why you insist on what is impossible (RTT-indep AQM) sho= uld be done instead??? =C2=A0 [SM] Yes, easily=2E RTT dependence is a direct result of processes= of competition at the bottleneck queue, where shorter RTT translates to ti= ghter control loops and hence faster reactions to changing estimates of cap= acity, that way shorter RTT flows pick up the excess capacity the transient= ly appears when (longer RTT) flows go to multiplicative decrease=2E In addi= tion the competition the bottleneck buffer can lead to a higher dropping pr= obability for log RTT flows (precisely because they react slower)=2E There = are two ways to tackle this: a) artificially slow down all flows to the same response dynamics as the l= ongest RTT flow displays=2E That is the way you started on, but you clearly= realised how sub-optimal this is, given that RTT are not really bounded to= anything sane (think paths over geostationary satellites), so you already = cheated by limiting that to the 15ms you foster on non L4S flows in your AQ= M=2E Now, I admit that technically it is not completely IMPOSSIBLE to ge= nerally solve this issue, by making all flows react as if their RTT would b= e >=3D say 500ms, but I do mantain, that nobody sane would accept that as a= viable solution=2E Making it effectively possible but inacceptable=2E b) ameliorate the competition in the bottleneck buffers=2E That is exactly= par for the course for advanced queue management=2E I have made the same arguments in the past, without convincing demonstrati= ons that my theory is flawed, but feel free to poke holes into it=2E But ag= ain, hard to believe that you seem to have forgotten, that the two of us ha= d pretty much that discussion before=2E Best Regards Sebastian *) Comments from members of the BBR and Quicc communities indicate no suff= iciently deep involvement as far as I can tell=2E Thanks and regards, Koen=2E =C2=A0 =C2=A0 From: Sebastian Moeller Sent: Saturday, October 31, 2020 4:50 PM To: Bob Briscoe Cc: tsvwg IETF list ; iccrg IRTF list = ; TCP Prague List ; De Schepper, Koen (Nokia - BE/Ant= werp) Subject: Aw: [tsvwg] ecn-l4s-id: Proposed Changed to Normative RTT Bias Te= xt =C2=A0 Dear list, =C2=A0 please note that both of these "requirements" hedge against the fact that = RTT independence is almost impossible to achieve from the endpoints (as it = is a consequence of buffering at the bottleneck), originally bzy stating "a= s wide a range of RTTs as possible" (which given the phisical constraints i= n reality is equal to the empty set)=2E Replacing that non-requirement with= a differently worded version ("as much as possible") does not change anyth= ing substantially, except for making TCP Prague appear less of a failure=2E= =C2=A0 IMNHO this requirement for an L4S compatible transport, should be replaced= with a requirement for an L4S compliant AQM (as it is limited queueing at = the bottleneck that causes most the issue) MUST at least perform as well as= a dumb FIFO, a mark which DualQ at short RTTs misses badly=2E But I guess,= a no regressions against the status quo requirement, will probably not app= ly to L4S=2E =C2=A0 Best Regards =C2=A0 =C2=A0 =C2=A0 =C2=A0 Sebastian =C2=A0 =C2=A0 Gesendet:=C2=A0Freitag, 30=2E Oktober 2020 um 19:42 Uhr Von:=C2=A0"Bob Briscoe" An:=C2=A0"tsvwg IETF list" Cc:=C2=A0"iccrg IRTF list" , "T= CP Prague List" , "De Sc= hepper, Koen (Koen)" Betreff:=C2=A0[tsvwg] ecn-l4s-id: Proposed Changed to Normative RTT Bias T= ext Folks, The co-authors of ECN L4S ID have been reviewing the correctness of the no= rmative 'Prague' requirements=2E =C2=A0=C2=A0=C2=A0 See https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-ec= n-l4s-id-10#section-4=2E3[https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-= ecn-l4s-id-10#section-4=2E3] This is the first of 2 emails, about 2 of the requirements that we think o= ught to be reworded a little=2E If you agree with the rationale, but think the new wording still doesn't c= apture the requirement well, pls suggest sthg better=2E If you disagree with the rationale, pls discuss=2E =C2=A0 4=2E3=2E=C2=A0 Prerequisite Congestion Response =2E=2E=2E CURRENT: =C2=A0=C2=A0 o=C2=A0 A scalable congestion control MUST reduce or eliminat= e RTT bias =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 over as wide a range of RTTs as possible, o= r at least over the =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 typical range of RTTs that will interact in= the intended =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 deployment scenario (see Appendix A=2E1=2E5= [https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-ecn-l4s-id-10#appendix-A= =2E1=2E5] for rationale)=2E PROPOSED: A scalable congestion control MUST eliminate RTT bias as much as possible = in the range between the minimum likely RTT and typical RTTs expected in the intended deployment scenario=C2=A0 (see Appendix A=2E1=2E5 for rati= onale)=2E RATIONALE: 1/ "eliminate as much as possible" is stronger than "reduce or eliminate"= =2E 2/ This requirement was motivated by 'do no harm to others' relative to ex= isting standard (RFC5681 Reno) congestion control=2E So there is no need to= mandate that an L4S implementer does no harm to themselves, which window-b= ased congestion controls tend to do at higher RTT=2E Of course, this doesn'= t preclude implementers reducing or eliminating RTT bias for larger than ty= pical RTTs, but it removes any requirement to do so=2E Cheers Bob =C2=A0 -- ________________________________________________________________ Bob Briscoe=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 http://bobbriscoe=2Enet/[http://= bobbriscoe=2Enet/] From nobody Sun Nov 1 03:19:22 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 778D13A1070; Sun, 1 Nov 2020 03:19:10 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hr8jEDaNvv5h; Sun, 1 Nov 2020 03:19:08 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 305253A1071; Sun, 1 Nov 2020 03:19:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1604229499; bh=9MTt05YU5yjok8zjVaNPLYN54bPjdZVQ/Ne51rWaYY0=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=NlMCEcj2gexrk+evxtJ0uZjkwXQ8dH3JsVitVD0f7EzM28Et0lyic9LClfFAqr23y OROy8H5OGloZU0jkrnCrViqwf6QnuEwTkzTd7OcndcggsVmDcXE2QtrzPx4u2SLPiy yR2OHx19xmfPudaQKMMDjXokS203xkqtlofMIwB4= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [134.76.241.253] ([134.76.241.253]) by web-mail.gmx.net (3c-app-gmx-bs25.server.lan [172.19.170.77]) (via HTTP); Sun, 1 Nov 2020 12:18:19 +0100 MIME-Version: 1.0 Message-ID: From: Sebastian Moeller To: Bob Briscoe Cc: tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" Content-Type: text/plain; charset=UTF-8 Date: Sun, 1 Nov 2020 12:18:19 +0100 Importance: normal Sensitivity: Normal In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:+7bFwXNx2+AlXPGpFLlL8PQ8BY7+FxmQHZg1VdGoiq2C3R9IPYK+eGjYN/IYYj4mdhfMd cX6sbY/A4XOKhkFWu/9cQeVj5O0ZRtzam0cSdgNGgV8UNjx6WsKAddeAqhjGHfbGsNjYwYEyQsWH FDOZzL+yErzX80U6g3ABsCqF2OqsLmzrL1RcytyHuRi68fVjfiHiqmO77xgf0oiQsJBtSLe1X1tj NRxF4MJ1s+ujCYzZU+NHwv8fQokClErBSqChEwiktD/D+JvAiogDlCcoM5d4bluKtGUiGa4MeF4t 8c= X-UI-Out-Filterresults: notjunk:1;V03:K0:fr84n6WAIqE=:u6IzcQs+NASRwAb1pIKvDH 4rKfPo1zJeQM0RDvDiM8qTW/NpJ5zggHzGt84EUVP1c4U0WvH/6X7rSN9YouhYJ+YK85Hvd9T mVq8xgu9GPLpJDwD8KM5j8nxZ9kuRgP42kN1raKceBNvjQtlE/u1HHWbWJv/uAU/ON2vhrqDZ Zm1uTfSMDRAuRPxivCEn3eCF7Yzxr6BWdi+zz+6EnGuWDcq3YyXB+N0qUtozEEADuybJCOdgT CYfwmJGE18HAjnL48IL2u+0zvAdkxCozOgX1hy6BaCaC8IVKa01wmN5g/aNMm0wSCzbhlYjOB Op8+VcLMWmP+ZdRuzAEeuNEBTlSXRv4xoNKuf6mXsNgeg9zqVAmm/zCDG3HHvQWk9O/jRpYFv m31K+uhUwofN9Txsy5zvbVLNTxvW0z/HgXZF9W8ERggRL+H7wXH2bxRcJ8yBT8pEs3o+IO0pE BpAW4eEbXEbqzytJrR0Ug/geNQGxQlZJZsP8iZQFkQBTIoviNSWX9S/U/yMGtoybcm2OQf9vq XiNkL0XQPmGxDRrO+IwEni9KWdrQ5JpnGZZXRwbZWCUu79D7hIdF432ke0n5Ce3igKzETExk0 KD7Ow/mOsA0L8= Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative RTT Bias Text [Proposal by SM] X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 11:19:11 -0000 Bob, Koen, Wes, please find my proposal for a clarification below, prefixed [SM] =C2=A0 =C2=A0 =C2=A0 Gesendet:=C2=A0Freitag, 30=2E Oktober 2020 um 19:42 Uhr Von:=C2=A0"Bob Briscoe" An:=C2=A0"tsvwg IETF list" Cc:=C2=A0"iccrg IRTF list" , "TCP Prague List" , "De Schepper, Koen (Koen)" Betreff:=C2=A0[tsvwg] ecn-l4s-id: Proposed Changed to Normative RTT Bias T= ext Folks, The co-authors of ECN L4S ID have been reviewing the correctness of the no= rmative 'Prague' requirements=2E =C2=A0=C2=A0=C2=A0 See https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-ec= n-l4s-id-10#section-4=2E3 This is the first of 2 emails, about 2 of the requirements that we think o= ught to be reworded a little=2E If you agree with the rationale, but think the new wording still doesn't c= apture the requirement well, pls suggest sthg better=2E If you disagree with the rationale, pls discuss=2E =C2=A0 4=2E3=2E Prerequisite Congestion Response =2E=2E=2E CURRENT: o A scalable congestion control MUST reduce or eliminate RTT bias over as wide a range of RTTs as possible, or at least over the typical range of RTTs that will interact in the intended deployment scenario (see Appendix A=2E1=2E5[https://tools=2Eietf=2Eo= rg/html/draft-ietf-tsvwg-ecn-l4s-id-10#appendix-A=2E1=2E5] for rationale)= =2E [=2E=2E=2E] [SM] CLARIFIED: Any protocol aiming at L4S compliance will need to account for = the fact that L4S dedicated reference AQM, the dual queue coupled AQM, infl= ates the RTT of flows in the deep-queue by 15ms (if the AQM is configured a= s described in the DualQ internet draft)=2E This will effectively lead to s= hort RTT flows in the shallow queue staving flows in the deep queue=2E Sinc= e DualQ#s coupling heuristic is not designed to undo this damage, it is inc= umbent on L4S-compliant transport protocols to undo this damage, by artific= ially slowing down their congestion response "simulating" an RTT of at leas= t that 15ms introduced by DualQ=2E RATIONALE: This a) clearly names the root cause of the issue and spells ou= t the remedy in a way that protocol implementers do not need to guess what = is implied by "eliminate RTT bias" and why this is not just an generally as= pirational goal, but why the 15ms hack is required for L4S to actually be d= eployable, as the avoidance of congestion collapse for short RTT flows in t= he deep queue really depends on all L4S protocols implementing that hack fa= ithfully=2E The proposed text also removes any residual mentioning of eliminati= on of RTT bias on first principle, since not even the designated reference = protocol TCP Prague actually achieves any meaningful generic RTT de-biasing= that is actually applicable in good faith to arbitrary large RTTs=2E=20 I add, that I still consider that hack unacceptable and am of the opinion,= that L4S AQM's MUST to be required not to create this situation in the fir= st place=2E BUT if the WG disagrees and is fine with rolling out a clearly = sub-optimal AQM with a-priori known catastrophic failure modes, then at lea= st my proposed text should be added, to unambiguously inform protocol desig= ners and implementers of their responsibility=2E Best Regards Sebastian RATIONALE: 1/ "eliminate as much as possible" is stronger than "reduce or eliminate"= =2E 2/ This requirement was motivated by 'do no harm to others' relative to ex= isting standard (RFC5681 Reno) congestion control=2E So there is no need to= mandate that an L4S implementer does no harm to themselves, which window-b= ased congestion controls tend to do at higher RTT=2E=20 [SM] You realise that this is pretty much included in the definiti= on of reducing or eliminating RTT bias, to not do harm to flows of the same= protocol at larger RTTs? That requirement, I am willing to bet, was not or= iginally written to save Reno from L4S flows (if it has, it has been except= ionally badly phrased) but really to imply that the "new" way of things is = going to be better in all relevant dimensions, more RTT-independent, less r= e-ordering sensitive, =2E=2E=2E This whole rationale makes me somehow think= "Oceania had always been at war with Eastasia=2E" Then again, this whole wiggling around and post-hoc changing of dr= aft text, basically just confirms my statement, that generic RTT-independen= ce can not be achieved by transport protocols, not without significant invo= lvement of the bottleneck queue's management entity=2E So again, I am okay = with dropping the RTT-independence requirement, as "long as the fix DualQ's= 15ms hack" is still added as requirement, unless preferably the AQM design= and implementation is fixed and the issue is eliminated at its source=2E BUT some chairs seem to indicate that the time for real changes is over an= d all that is still available are small changes in the text, and so I propo= se this change=2E Of course, this doesn't preclude implementers reducing or eliminating RTT = bias for larger than typical RTTs, but it removes any requirement to do so= =2E Cheers Bob =C2=A0 --=20 ________________________________________________________________ Bob Briscoe http://bobbriscoe=2Enet/[http://= bobbriscoe=2Enet/] From nobody Sun Nov 1 05:03:28 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DC53D3A07E6; Sun, 1 Nov 2020 05:03:20 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RrnNIcFM_Swu; Sun, 1 Nov 2020 05:03:18 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 652D33A0820; Sun, 1 Nov 2020 05:03:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1604235748; bh=EEh7n1KOIdMLzZbkAgnJ8DvJnJJIxHC5ZGJo2t2te6E=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=C3sVLRtX5ww6v5W4nFXJkhj8+RdOaFd+hypnLSoeGTUCBdor+EPNqnRgLTsUH1dt/ At+Sn4VHOcp3oZNX8oztCLD4mIB+qnYY/rNRwnkTby/HRYk7JSWEgP6KwhJ2MMswN2 34rVUL22mjv9l2XaUcLOjTFGNgA2q1g3mihbE0us= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [134.76.241.253] ([134.76.241.253]) by web-mail.gmx.net (3c-app-gmx-bs25.server.lan [172.19.170.77]) (via HTTP); Sun, 1 Nov 2020 14:02:28 +0100 MIME-Version: 1.0 Message-ID: From: Sebastian Moeller To: Christian Huitema Cc: Bob Briscoe , tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" Content-Type: text/plain; charset=UTF-8 Date: Sun, 1 Nov 2020 14:02:28 +0100 Importance: normal Sensitivity: Normal In-Reply-To: <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:hzvJxyoDylKZVHhiXLqno+1Prnh9ezBFoUmnT3xHwwSWsA1WdKrQL3x3sOodJXv1Y+pqx dmhdckHjFCIQdhOBDQnrx2YNFTpHyVVFgSVj4bf9/9MJAZatoWzlLHUKsQ7yRAZcxBK6OXM0/wDn 1Wocn2ZAw5eBfv/Gfk6waGvGWx+AQNrofxMZt3DvCDa/D+KpP/LYMRkD0B0Gvlen+YLCuYKMxQkv 90vl6TjH2hGH5eP0gjwX5TkHD1QROsopL9x9XQQySrY6q2zcV5hUV9zIbMv0z0kJqnPcYng0uoJL W4= X-UI-Out-Filterresults: notjunk:1;V03:K0:Ak71coR95FM=:K1XGHQG6RTVs1Ztn5jONgj i8zYzngoKMn8V5Kd7G6Rk0e2Uxzjv64MfQx4dYn2xaBvgXHSkq55MEaF+3DzdD9F++8DQIWm6 +cEtBl78YCUkdI0HsgRimiqfSqsCjQUCLOBT6C12bD6/l9WWASXhwITBLA6Xe2EDz7DRD7AID 1cmylyya3GoUX5B7O1K0j62k1gv0GUCWs0d2pvUk1lfR33KKxZfB7jx+gyDI8kxd04C/33zwC D/CUfDLxI61ykbYU49OX4ifEoyuNkxtadk+hvn51ERvj+99h6ibUpmVcHPGsmeVYj4ZST8XIr PakVl2IB+yqud2Dt3MLJ6PsgJ9wrAFMqdcBpj6Zla3oUFWQI7GEKZsDtZ7plv+9J3vIbRADkY yX5ldWjYinOnPODkjBRP62iUhnxnxZ1mLNTCLCbojzl3kbuMzw/+dj4RGzXjOKzHaErWqMqd7 3ZfkETsptduTS9qms2W82UIKwxyPJVtS86SB/ypEe3jKsh3XYrx2/ryStSXN7tPC0x87kZJqU QC7hD46BXuTohHDisn1jyJ3VFr9ziwCZrOzZ3nywVccl7tX1BRcJMuLrhtWPmjn5WPRs5BieQ qnIic3mTr5wUs= Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 13:03:21 -0000 Dear Christian, I do not really want to drag you into the L4S protocol requirements issue,= but since you seem to be contemplating making your protocol L4S-aware and = -compliant, what is your interpretation of the "eliminate RTT bias" require= ment in its original and in Bob's proposed modified version=2E Did you unde= rstand that this required you to account for 15ms higher sustained RTT unde= r load in the classic queue of an L4S AQM by slowing down the CC response b= y ECT(1) flows by the same 15ms? If no, could you opinie whether my propose= d change for this requirement is any clearer?=20 I top post this as it does not address your specific issues (but I am quit= e intersted in learning team L4S' response to your questions), but seems su= fficiently related to merit as response (past attempts at soliciting input = from L4S implementers fell on deaf ears, so I will not cold-post any such q= uestions anymore)=2E Thank you very much in advance! =C2=A0 Best Regards Sebastian =C2=A0 Gesendet:=C2=A0Sonntag, 01=2E November 2020 um 02:07 Uhr Von:=C2=A0"Christian Huitema" An:=C2=A0"Bob Briscoe" , "tsvwg IETF list" Cc:=C2=A0"iccrg IRTF list" , "TCP Prague List" , "De Schepper, Koen (Koen)" Betreff:=C2=A0Re: [tsvwg] [tcpPrague] ecn-l4s-id: Proposed Changed to Norm= ative Classic ECN detection Text I am reading the L4S ECN-AQM proposal with an eye on responding to it in a= n implementation of QUIC, and I have a couple of questions regarding use of= ECN marking with QUIC=2E The document does not mention QUIC, yet QUIC is already used in a large fr= action of Internet traffic=2E QUIC does specify support for ECN, and QUIC a= cknowledgements may carry counts of each category of ECN marks received fro= m the peer -- three counters for ECT(0), ECT(1) and CE=2E In theory, QUIC i= mplementations could take advantage of L4S -- in fact, at least one impleme= ntation supports DC-TCP like CC already=2E Is there interest in specifying = L4S for QUIC? My next question regards the interaction of the proposed L4S ECN-AQM with = CC algorithms like BBR that attempt to discover the bottleneck packet rate = for the connection, and use pacing to send packets at that rate=2E I observ= ed that BBR is never mentioned in the draft, yet BBR is used in a sizeable = part of Internet traffic=2E Do we have data on how a non-L4S aware implemen= tation of BBR interacts with the proposed L4S AQM? My last question regards potential use of ECT(1) marking=2E Most current i= mplementations set ECT(0), but setting ECT(1) instead is trivial=2E This sh= ould elicit an L4S compatible response in L4S-AQM, and the BBR implementati= on might be modified to use the signals as part of the bottleneck bandwidth= tracking=2E But there is a small issue there=2E With BBR, QUIC packets are= supposedly paced at just under the bottleneck rate, except during "probe" = periods in which they probe for 1 RTT at a slightly higher rate=2E The L4S = AGM might degenerate in a form of ON-OFF control -- no feedback at all most= of the time, then a bunch of CE marks if the probe rate exceeds the bottle= neck bandwidth=2E As anyone experimented with that? -- Christian Huitema =C2=A0 On 10/31/2020 2:54 AM, Bob Briscoe wrote:Folks, The co-authors of ECN L4S ID have been reviewing the correctness of the no= rmative 'Prague' requirements=2E =C2=A0=C2=A0=C2=A0 See https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-ec= n-l4s-id-10#section-4=2E3 This is the second of 2 emails, about 2 of the requirements that we think = ought to be reworded a little=2E If you agree with the rationale, but think the new wording still doesn't f= ully capture the requirement, pls suggest sthg better=2E If you disagree with the rationale, pls discuss=2E =C2=A0 4=2E3=2E Prerequisite Congestion Response =2E=2E=2E CURRENT: o A scalable congestion control MUST react to ECN marking from a non-L4S but ECN-capable bottleneck in a way that will coexist with a TCP Reno congestion control [RFC5681[https://tools=2Eietf=2Eorg/ht= ml/rfc5681]] (see Appendix A=2E1=2E4[https://tools=2Eietf=2Eorg/html/draft-= ietf-tsvwg-ecn-l4s-id-10#appendix-A=2E1=2E4] for rationale)=2E =C2=A0Note that a scalable congestion control is not expected to chan= ge to setting ECT(0) while it falls back to coexist with Reno=2E =C2=A0 PROPOSED: =C2=A0=C2=A0 o=C2=A0 A scalable congestion control MUST implement monitori= ng in order =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to detect a likely non-L4S but ECN-capable = AQM at the bottleneck=2E =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 On detection of a likely ECN-capable bottle= neck it SHOULD be =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 capable (dependent on configuration) of aut= omatically adapting its =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 congestion response to coexist with TCP Ren= o congestion controls =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 [RFC5681] (see Appendix A=2E1=2E4 for ratio= nale and a referenced =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 algorithm)=2E =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Note that a scalable congestion control is = not expected to change =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to setting ECT(0) while it falls back to co= exist with Reno=2E RATIONALE: 1/ The requirement as currently written says what an omniscient sender MUS= T do=2E So there's an implied requirement that a sender MUST be omniscient,= which is of course impossible=2E 2/ The requirement needs to be recast to require a sender to aim to be as = knowledgeable as possible=2E Then, what it does as a result needs to take i= nto account the a priori likelihood of there being a non-L4S bottleneck pre= sent=2E 3/ This includes the possibility that the operator of the host knows that = the network it serves has not deployed any single queue classic ECN AQM (e= =2Eg=2E in a CDN case they're doing out of band testing, or they've asked t= he ISP)=2E So we've included the possibility of fall-back being disabled by= configuration=2E 4/ Nonetheless, as has been pointed out on the list, there is still a poss= ibility that there is a Classic ECN AQM somewhere else on the path (to cont= inue the CDN example, perhaps beyond the ISP in a home network)=2E The 'MUS= T monitor' requirement still stands to ensure the operator doesn't miss the= se cases=2E 5/ Then, if the server operators have disabled fall-back for their deploym= ent, they can reconsider their policy or at least do more focused testing i= f they are frequently detecting a single-queue Classic ECN AQM=2E Items 3-5 are the "react via management" model that I've talked about on t= he list, given the unfairness doesn't amount to starvation, and it is possi= ble that the prevalence of the problem is very low=2E Finally, after the bullet list of requirements in section 4=2E3, (which ar= e prerequisites for setting the ECT1 codepoint), we propose to add the foll= owing requirement, as suggested on the tsvwg list: =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 To participate in the L4S experiment, a sca= lable congestion control MUST =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 be capable of being replaced by a Classic c= ongestion control (by =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 application and by administrative control)= =2E A Classic congestion control =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 will not tag its packets with the ECT(1) co= depoint=2E Cheers Bob =C2=A0 --=20 ________________________________________________________________ Bob Briscoe http://bobbriscoe=2Enet/[http://= bobbriscoe=2Enet/] =C2=A0 =C2=A0 _______________________________________________ tcpPrague mailing listtcpPrague@ietf=2Eorg[mailto:tcpPrague@ietf=2Eorg]htt= ps://www=2Eietf=2Eorg/mailman/listinfo/tcpprague From nobody Sun Nov 1 05:31:12 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 22FE43A08B2; Sun, 1 Nov 2020 05:31:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_FAIL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ae-8gm0Ra34g; Sun, 1 Nov 2020 05:31:04 -0800 (PST) Received: from cl3.bcs-hosting.net (cl3.bcs-hosting.net [3.11.37.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AD7A03A08C0; Sun, 1 Nov 2020 05:31:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=bKcHTUNNDh6B6UxrQeDIeWsYKI3EeRKcQm4UI42amKE=; b=2Yb1rQswQlpVXVcsfGmbM1hwz kopA+6boWs1Rruxw1u9GAHH2O5/jpcsoJoMz2o+am/KeM2t/tOvvHqIFJGDlYxwqa8rkgERedk9ku CLolrgXnHqBrIOGwna7QSDyjSosHnkmIXdO82E4E6mBY6ydT8VbVzqa86XSHqLLrzelWZOlIGITU1 wuacxYXUg1SqL5XL9chZqwW+J4zoMhI9v9wDoGxkl1KK/4hdUTSzTHeUyun6V9iHkbH9VUOtvJZKm ypoPPzvp1dZuY86ejOGkmwFiwFimBBRHPzmpbSWVGdEngMbGFKqCINU1buGypBTNpEeqWvdHJSO9m dvQSIJ13Q==; Received: from 67.153.238.178.in-addr.arpa ([178.238.153.67]:34964 helo=[192.168.1.3]) by cl3.bcs-hosting.net with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1kZDRl-007gu3-EQ; Sun, 01 Nov 2020 13:31:01 +0000 To: Christian Huitema , tsvwg IETF list Cc: iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> From: Bob Briscoe Message-ID: <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> Date: Sun, 1 Nov 2020 13:30:59 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> Content-Type: multipart/alternative; boundary="------------1513BD8FB92639E16973A561" Content-Language: en-GB X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cl3.bcs-hosting.net X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: cl3.bcs-hosting.net: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: cl3.bcs-hosting.net: in@bobbriscoe.net X-Source: X-Source-Args: X-Source-Dir: Archived-At: Subject: [tcpPrague] L4S and BBR (was: [iccrg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text) X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 13:31:08 -0000 This is a multi-part message in MIME format. --------------1513BD8FB92639E16973A561 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Christian, I've changed the subject line given it's no longer appropriate. See inline tagged [BB]... On 01/11/2020 01:07, Christian Huitema wrote: > > I am reading the L4S ECN-AQM proposal with an eye on responding to it > in an implementation of QUIC, > [BB] That's good news. > and I have a couple of questions regarding use of ECN marking with QUIC. > > The document does not mention QUIC, yet QUIC is already used in a > large fraction of Internet traffic. QUIC does specify support for ECN, > and QUIC acknowledgements may carry counts of each category of ECN > marks received from the peer -- three counters for ECT(0), ECT(1) and > CE. In theory, QUIC implementations could take advantage of L4S -- in > fact, at least one implementation supports DC-TCP like CC already. Is > there interest in specifying L4S for QUIC? > [BB] Certainly. Nonetheless, when I search for the string "QUIC" in L4S ECN-AQM proposal, I find it (assuming you mean draft-ietf-tsvwg-aqm-dualq-coupled ). But you won't find much else about QUIC there, 'cos that doc is about the AQM. There are three main L4S docs. You need to be reading the doc with the requirements for L4S transports: draft-ietf-tsvwg-ecn-l4s-id And the L4S architecture: draft-ietf-tsvwg-l4s-arch Both of which explain that QUIC already provides the necessary feedback. Quentin De Coninck wrote an initial implementation of a "QUIC Prague" congestion control based on your picoQUIC code. It can be accessed via the L4S landing page here:     https://riteproject.eu/dctth/#code He started it at an IETF hackathon. I don't think it's been maintained since it was first written, but it might be a start. > My next question regards the interaction of the proposed L4S ECN-AQM > with CC algorithms like BBR that attempt to discover the bottleneck > packet rate for the connection, and use pacing to send packets at that > rate. I observed that BBR is never mentioned in the draft, yet BBR is > used in a sizeable part of Internet traffic. Do we have data on how a > non-L4S aware implementation of BBR interacts with the proposed L4S AQM? > [BB] Again, if you're looking at the L4S AQM doc, you won't find much about CC. For BBR, take a look at section 5.2 of l4s-arch . When BBRv2 was released it included initial support for L4S ECN. Neal and Yuchung presented BBRv2's L4S ECN queue delay results when they first talked about BBRv2 in iccrg. I'm sure more work is needed on the L4S ECN part, but I guess that will rise in priority once ISPs start deploying L4S AQMs (e.g. Greg White reported that cable implementations were being interop tested some time ago, and Nokia released an L4S WiFi AP a while back). So, if there is an L4S AQM at the bottleneck, the resulting ECN signals would make BBRv2 morph into an L4S CC. However, your question is about how a non-L4S aware BBR interacts with an L4S AQM. * For that, I'll have to take 'BBR' to mean something like BBRv2, but without any support for ECN - i.e. without the L4S ECN part. * And just to be super-precise, I'll take the 'L4S AQM' to mean the DualQ Coupled AQM (not an FQ one). Then the answer is, I think, only a few tests of that combination have been done. Am I right, Olga, Asad, Olivier, Neal, Yuchung, anyone? I suspect this mainly because there doesn't seem to be a reason why the L4S ECN part of BBRv2 would be disabled. I can say what I suspect would happen in what I think is your hypothetical case. The DualQ Coupled AQM design relies on a flow that is classified into the Classic queue (e.g.. the non-L4S aware BBRv2 you ask about) responding to loss and/or delay in a way that would be friendly to Reno. The v2 changes to BBR were intended to be more friendly to other flows, in particular the response to >1% loss. Then, these non-L4S aware BBRv2 flows that you posit should co-exist with L4S flows in some way. But whether they would coexist well isn't something I've looked into, 'cos I think it's rather a hypothetical question (if I've understood you correctly). Before BBRv2 when BBRv1 had no response to loss (and no ECN support), I can say that it usually starved L4S flows over the same DualQ AQM for the same reason it usually starved Cubic and other flows in a FIFO. But hopefully that is in the past now. > My last question regards potential use of ECT(1) marking. Most current > implementations set ECT(0), but setting ECT(1) instead is trivial. > This should elicit an L4S compatible response in L4S-AQM, and the BBR > implementation might be modified to use the signals as part of the > bottleneck bandwidth tracking. But there is a small issue there. With > BBR, QUIC packets are supposedly paced at just under the bottleneck > rate, except during "probe" periods in which they probe for 1 RTT at a > slightly higher rate. The L4S AGM might degenerate in a form of ON-OFF > control -- no feedback at all most of the time, then a bunch of CE > marks if the probe rate exceeds the bottleneck bandwidth. As anyone > experimented with that? > [BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'allowed' to send packets over the Internet as ECT(1). Then, yes, an L4S DualQ Coupled AQM will classify BBRv2 packets into the L4S queue. This should have a very shallow ECN marking threshold (500us - 1ms), so even if the flow (whether QUIC or TCP) is flying just under the available capacity, bunching of packets means it is unlikely to completely avoid ECN marking between probes. If it could avoid ECN marking, you are right that it would get a bump of ECN marks during the probe. I haven't studied the code, but when it experiences ECN marking I believe it switches into an L4S ECN mode for a while, and uses ECN rather than delay probing to track available capacity. I assume it switches back to BBR's delay probing mode if it gets no ECN for a while (e.g. the bottleneck might have moved). But I haven't looked at BBRv2's ECN behaviour in detail. I'm sure someone on one of the lists that this is posted to will be able to answer tho. HTH Bob > -- Christian Huitema > > > On 10/31/2020 2:54 AM, Bob Briscoe wrote: >> Folks, >> >> The co-authors of ECN L4S ID have been reviewing the correctness of >> the normative 'Prague' requirements. >>     See >> https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-10#section-4.3 >> This is the second of 2 emails, about 2 of the requirements that we >> think ought to be reworded a little. >> >> If you agree with the rationale, but think the new wording still >> doesn't fully capture the requirement, pls suggest sthg better. >> If you disagree with the rationale, pls discuss. >> >> 4.3. Prerequisite Congestion Response >> ... >> CURRENT: >> >> o A scalable congestion control MUST react to ECN marking from a >> non-L4S but ECN-capable bottleneck in a way that will coexist with >> a TCP Reno congestion control [RFC5681 ] (seeAppendix A.1.4 for >> rationale). >> >>  Note that a scalable congestion control is not expected to change >> to setting ECT(0) while it falls back to coexist with Reno. >> >> PROPOSED: >>    o  A scalable congestion control MUST implement monitoring in order >>       to detect a likely non-L4S but ECN-capable AQM at the bottleneck. >>       On detection of a likely ECN-capable bottleneck it SHOULD be >>       capable (dependent on configuration) of automatically adapting its >>       congestion response to coexist with TCP Reno congestion controls >>       [RFC5681] (see Appendix A.1.4 for rationale and a referenced >>       algorithm). >> >>       Note that a scalable congestion control is not expected to change >>       to setting ECT(0) while it falls back to coexist with Reno. >> >> RATIONALE: >> 1/ The requirement as currently written says what an omniscient >> sender MUST do. So there's an implied requirement that a sender MUST >> be omniscient, which is of course impossible. >> 2/ The requirement needs to be recast to require a sender to aim to >> be as knowledgeable as possible. Then, what it does as a result needs >> to take into account the a priori likelihood of there being a non-L4S >> bottleneck present. >> 3/ This includes the possibility that the operator of the host knows >> that the network it serves has not deployed any single queue classic >> ECN AQM (e.g. in a CDN case they're doing out of band testing, or >> they've asked the ISP). So we've included the possibility of >> fall-back being disabled by configuration. >> 4/ Nonetheless, as has been pointed out on the list, there is still a >> possibility that there is a Classic ECN AQM somewhere else on the >> path (to continue the CDN example, perhaps beyond the ISP in a home >> network). The 'MUST monitor' requirement still stands to ensure the >> operator doesn't miss these cases. >> 5/ Then, if the server operators have disabled fall-back for their >> deployment, they can reconsider their policy or at least do more >> focused testing if they are frequently detecting a single-queue >> Classic ECN AQM. >> >> Items 3-5 are the "react via management" model that I've talked about >> on the list, given the unfairness doesn't amount to starvation, and >> it is possible that the prevalence of the problem is very low. >> >> >> Finally, after the bullet list of requirements in section 4.3, (which >> are prerequisites for setting the ECT1 codepoint), we propose to add >> the following requirement, as suggested on the tsvwg list: >> >>       To participate in the L4S experiment, a scalable congestion >> control MUST >>       be capable of being replaced by a Classic congestion control (by >>       application and by administrative control). A Classic >> congestion control >>       will not tag its packets with the ECT(1) codepoint. >> >> Cheers >> >> >> Bob >> >> >> -- >> ________________________________________________________________ >> Bob Briscoehttp://bobbriscoe.net/ >> >> _______________________________________________ >> tcpPrague mailing list >> tcpPrague@ietf.org >> https://www.ietf.org/mailman/listinfo/tcpprague > > _______________________________________________ > iccrg mailing list > iccrg@irtf.org > https://www.irtf.org/mailman/listinfo/iccrg -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ --------------1513BD8FB92639E16973A561 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Christian,

I've changed the subject line given it's no longer appropriate.
See inline tagged [BB]...

On 01/11/2020 01:07, Christian Huitema wrote:

I am reading the L4S ECN-AQM proposal with an eye on responding to it in an implementation of QUIC,


[BB] That's good news.

and I have a couple of questions regarding use of ECN marking with QUIC.

The document does not mention QUIC, yet QUIC is already used in a large fraction of Internet traffic. QUIC does specify support for ECN, and QUIC acknowledgements may carry counts of each category of ECN marks received from the peer -- three counters for ECT(0), ECT(1) and CE. In theory, QUIC implementations could take advantage of L4S -- in fact, at least one implementation supports DC-TCP like CC already. Is there interest in specifying L4S for QUIC?


[BB] Certainly.
Nonetheless, when I search for the string "QUIC" in L4S ECN-AQM proposal, I find it (assuming you mean draft-ietf-tsvwg-aqm-dualq-coupled ).
But you won't find much else about QUIC there, 'cos that doc is about the AQM.

There are three main L4S docs.
You need to be reading the doc with the requirements for L4S transports: draft-ietf-tsvwg-ecn-l4s-id
And the L4S architecture: draft-ietf-tsvwg-l4s-arch
Both of which explain that QUIC already provides the necessary feedback.

Quentin De Coninck wrote an initial implementation of a "QUIC Prague" congestion control based on your picoQUIC code. It can be accessed via the L4S landing page here:
    https://riteproject.eu/dctth/#code
He started it at an IETF hackathon. I don't think it's been maintained since it was first written, but it might be a start.


My next question regards the interaction of the proposed L4S ECN-AQM with CC algorithms like BBR that attempt to discover the bottleneck packet rate for the connection, and use pacing to send packets at that rate. I observed that BBR is never mentioned in the draft, yet BBR is used in a sizeable part of Internet traffic. Do we have data on how a non-L4S aware implementation of BBR interacts with the proposed L4S AQM?


[BB] Again, if you're looking at the L4S AQM doc, you won't find much about CC.
For BBR, take a look at section 5.2 of l4s-arch.

When BBRv2 was released it included initial support for L4S ECN. Neal and Yuchung presented BBRv2's L4S ECN queue delay results when they first talked about BBRv2 in iccrg. I'm sure more work is needed on the L4S ECN part, but I guess that will rise in priority once ISPs start deploying L4S AQMs (e.g. Greg White reported that cable implementations were being interop tested some time ago, and Nokia released an L4S WiFi AP a while back).

So, if there is an L4S AQM at the bottleneck, the resulting ECN signals would make BBRv2 morph into an L4S CC.

However, your question is about how a non-L4S aware BBR interacts with an L4S AQM.
* For that, I'll have to take 'BBR' to mean something like BBRv2, but without any support for ECN - i.e. without the L4S ECN part.
* And just to be super-precise, I'll take the 'L4S AQM' to mean the DualQ Coupled AQM (not an FQ one).
Then the answer is, I think, only a few tests of that combination have been done. Am I right, Olga, Asad, Olivier, Neal, Yuchung, anyone? I suspect this mainly because there doesn't seem to be a reason why the L4S ECN part of BBRv2 would be disabled.

I can say what I suspect would happen in what I think is your hypothetical case. The DualQ Coupled AQM design relies on a flow that is classified into the Classic queue (e.g.. the non-L4S aware BBRv2 you ask about) responding to loss and/or delay in a way that would be friendly to Reno. The v2 changes to BBR were intended to be more friendly to other flows, in particular the response to >1% loss. Then, these non-L4S aware BBRv2 flows that you posit should co-exist with L4S flows in some way. But whether they would coexist well isn't something I've looked into, 'cos I think it's rather a hypothetical question (if I've understood you correctly).

Before BBRv2 when BBRv1 had no response to loss (and no ECN support), I can say that it usually starved L4S flows over the same DualQ AQM for the same reason it usually starved Cubic and other flows in a FIFO. But hopefully that is in the past now.


My last question regards potential use of ECT(1) marking. Most current implementations set ECT(0), but setting ECT(1) instead is trivial. This should elicit an L4S compatible response in L4S-AQM, and the BBR implementation might be modified to use the signals as part of the bottleneck bandwidth tracking. But there is a small issue there. With BBR, QUIC packets are supposedly paced at just under the bottleneck rate, except during "probe" periods in which they probe for 1 RTT at a slightly higher rate. The L4S AGM might degenerate in a form of ON-OFF control -- no feedback at all most of the time, then a bunch of CE marks if the probe rate exceeds the bottleneck bandwidth. As anyone experimented with that?


[BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'allowed' to send packets over the Internet as ECT(1). Then, yes, an L4S DualQ Coupled AQM will classify BBRv2 packets into the L4S queue. This should have a very shallow ECN marking threshold (500us - 1ms), so even if the flow (whether QUIC or TCP) is flying just under the available capacity, bunching of packets means it is unlikely to completely avoid ECN marking between probes. If it could avoid ECN marking, you are right that it would get a bump of ECN marks during the probe. I haven't studied the code, but when it experiences ECN marking I believe it switches into an L4S ECN mode for a while, and uses ECN rather than delay probing to track available capacity. I assume it switches back to BBR's delay probing mode if it gets no ECN for a while (e.g. the bottleneck might have moved). But I haven't looked at BBRv2's ECN behaviour in detail.

I'm sure someone on one of the lists that this is posted to will be able to answer tho.

HTH


Bob


-- Christian Huitema


On 10/31/2020 2:54 AM, Bob Briscoe wrote:
Folks,

The co-authors of ECN L4S ID have been reviewing the correctness of the normative 'Prague' requirements.
    See https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-10#section-4.3
This is the second of 2 emails, about 2 of the requirements that we think ought to be reworded a little.

If you agree with the rationale, but think the new wording still doesn't fully capture the requirement, pls suggest sthg better.
If you disagree with the rationale, pls discuss.

4.3.  Prerequisite Congestion Response
...
CURRENT:

   o  A scalable congestion control MUST react to ECN marking from a
      non-L4S but ECN-capable bottleneck in a way that will coexist with
      a TCP Reno congestion control [RFC5681] (see Appendix A.1.4 for
      rationale).

      Note that a scalable congestion control is not expected to change
      to setting ECT(0) while it falls back to coexist with Reno.
 
PROPOSED:
   o  A scalable congestion control MUST implement monitoring in order
      to detect a likely non-L4S but ECN-capable AQM at the bottleneck.
      On detection of a likely ECN-capable bottleneck it SHOULD be
      capable (dependent on configuration) of automatically adapting its
      congestion response to coexist with TCP Reno congestion controls
      [RFC5681] (see Appendix A.1.4 for rationale and a referenced
      algorithm).

      Note that a scalable congestion control is not expected to change
      to setting ECT(0) while it falls back to coexist with Reno.

RATIONALE:
1/ The requirement as currently written says what an omniscient sender MUST do. So there's an implied requirement that a sender MUST be omniscient, which is of course impossible.
2/ The requirement needs to be recast to require a sender to aim to be as knowledgeable as possible. Then, what it does as a result needs to take into account the a priori likelihood of there being a non-L4S bottleneck present.
3/ This includes the possibility that the operator of the host knows that the network it serves has not deployed any single queue classic ECN AQM (e.g. in a CDN case they're doing out of band testing, or they've asked the ISP). So we've included the possibility of fall-back being disabled by configuration.
4/ Nonetheless, as has been pointed out on the list, there is still a possibility that there is a Classic ECN AQM somewhere else on the path (to continue the CDN example, perhaps beyond the ISP in a home network). The 'MUST monitor' requirement still stands to ensure the operator doesn't miss these cases.
5/ Then, if the server operators have disabled fall-back for their deployment, they can reconsider their policy or at least do more focused testing if they are frequently detecting a single-queue Classic ECN AQM.

Items 3-5 are the "react via management" model that I've talked about on the list, given the unfairness doesn't amount to starvation, and it is possible that the prevalence of the problem is very low.


Finally, after the bullet list of requirements in section 4.3, (which are prerequisites for setting the ECT1 codepoint), we propose to add the following requirement, as suggested on the tsvwg list:

      To participate in the L4S experiment, a scalable congestion control MUST
      be capable of being replaced by a Classic congestion control (by
      application and by administrative control). A Classic congestion control
      will not tag its packets with the ECT(1) codepoint.

Cheers


Bob


-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

_______________________________________________
tcpPrague mailing list
tcpPrague@ietf.org
https://www.ietf.org/mailman/listinfo/tcpprague


_______________________________________________
iccrg mailing list
iccrg@irtf.org
https://www.irtf.org/mailman/listinfo/iccrg


-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
--------------1513BD8FB92639E16973A561-- From nobody Sun Nov 1 09:00:18 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 385E63A0C29; Sun, 1 Nov 2020 09:00:15 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.848 X-Spam-Level: X-Spam-Status: No, score=-1.848 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VnNaxIkO7NwD; Sun, 1 Nov 2020 09:00:13 -0800 (PST) Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [IPv6:2a00:1450:4864:20::135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 41B473A0C1F; Sun, 1 Nov 2020 09:00:13 -0800 (PST) Received: by mail-lf1-x135.google.com with SMTP id j30so14335092lfp.4; Sun, 01 Nov 2020 09:00:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=9oa7zDMiQ29mTKXhE5SagZSZXO7Tu3uARg0u4YuXEYA=; b=oXrqZ9T6MTxmKkFDNjb7A4ngZuu/iGYW/LtJKOwcTbUqErhKAy1ieWia9narLV1Jqw FmxEjHJGWjQNfO9rW5byn4JjJRaGPGQGsEVePrQSs4olKTexBCjXj9NTDTzPhxWvriGY OGDRki3dEGwtDXaRkXujJEz+5hOoyCMHNCacaEItYkY8KF9nMWv7p8ASBgRX3K52Ohba y8EN6IKRdwtWwCZF60hzDysLOFz3QzZYvvGWtGNnUnxuyUx9/rtoFwQNEwUItuqjU5AF QzkJNFfKdZF1oKPQxaQR4cMXcfgnmH9dCjHC71S51A/2BVKwOuVYck477hZqvaTObbyk A7mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=9oa7zDMiQ29mTKXhE5SagZSZXO7Tu3uARg0u4YuXEYA=; b=VRTlVCK4RKPLPix1hlYDa5xI2XNd+YAtVL0zH692xdA9aRWDKHn5y550DeIboAgH4U 2lns+Nn4AU1q/iZQTTbdoV1mMVkFSc8sDRsF+D3TOPuJCx4bxT8kvLFu8+oMXwE6v93p TBw4aXqn2Buzpo72eHfEyIslSld2y/l+ec9q8X7+CiNW8VJMyWzZUKCg539vHJltps9u 18c2j1UmrlPKd0p6IPE8XF9UDmpVKTY/N3qEY/2NH1/bxgboCs+12KnKuPNd/J0tn15/ Pxm+D77Qmznd4kCs0KBwH85u1bI2KemfH2Y4yQgUYd4RkJucwgu5CkOr3cWH9geO9hTp W8eQ== X-Gm-Message-State: AOAM530ewx+0y+DUJbjF8cSzaaxZOxumVKqkILBSnWnfzuVZTJrLKS4D RdhmUmtPBT/LmJQIgapOpWR7kahXsKdXmg== X-Google-Smtp-Source: ABdhPJw6R+YBuPuIKmdQEch4Ch+5DHKulOUgfUMLjo6VffQ5JRh8vJBCVMpLsv+5lk+LniLjIwYnOA== X-Received: by 2002:a19:e04e:: with SMTP id g14mr4092196lfj.590.1604250011296; Sun, 01 Nov 2020 09:00:11 -0800 (PST) Received: from jonathartonsmbp.lan (178-55-170-35.bb.dnainternet.fi. [178.55.170.35]) by smtp.gmail.com with ESMTPSA id o206sm1609699lff.204.2020.11.01.09.00.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 01 Nov 2020 09:00:10 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.7\)) From: Jonathan Morton In-Reply-To: <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> Date: Sun, 1 Nov 2020 19:00:09 +0200 Cc: Bob Briscoe , tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" Content-Transfer-Encoding: quoted-printable Message-Id: <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> To: Christian Huitema X-Mailer: Apple Mail (2.3445.9.7) Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 17:00:16 -0000 > On 1 Nov, 2020, at 3:07 am, Christian Huitema = wrote: >=20 > I am reading the L4S ECN-AQM proposal with an eye on responding to it = in an implementation of QUIC, and I have a couple of questions regarding = use of ECN marking with QUIC.=20 >=20 > The document does not mention QUIC, yet QUIC is already used in a = large fraction of Internet traffic. QUIC does specify support for ECN, = and QUIC acknowledgements may carry counts of each category of ECN marks = received from the peer -- three counters for ECT(0), ECT(1) and CE. In = theory, QUIC implementations could take advantage of L4S -- in fact, at = least one implementation supports DC-TCP like CC already. Is there = interest in specifying L4S for QUIC? >=20 > My next question regards the interaction of the proposed L4S ECN-AQM = with CC algorithms like BBR that attempt to discover the bottleneck = packet rate for the connection, and use pacing to send packets at that = rate. I observed that BBR is never mentioned in the draft, yet BBR is = used in a sizeable part of Internet traffic. Do we have data on how a = non-L4S aware implementation of BBR interacts with the proposed L4S AQM? >=20 > My last question regards potential use of ECT(1) marking. Most current = implementations set ECT(0), but setting ECT(1) instead is trivial. This = should elicit an L4S compatible response in L4S-AQM, and the BBR = implementation might be modified to use the signals as part of the = bottleneck bandwidth tracking. But there is a small issue there. With = BBR, QUIC packets are supposedly paced at just under the bottleneck = rate, except during "probe" periods in which they probe for 1 RTT at a = slightly higher rate. The L4S AGM might degenerate in a form of ON-OFF = control -- no feedback at all most of the time, then a bunch of CE marks = if the probe rate exceeds the bottleneck bandwidth. As anyone = experimented with that? In my view, the biggest question you should be asking is how QUIC will = distinguish between CE marks applied by an L4S AQM on the path, and = those applied by an RFC-3168 AQM on the path. The latter will treat = ECT(1) marked packets the same as ECT(0), and expects the same = multiplicative-decrease congestion response, but L4S expects a = qualitatively different linear response. You should probably not do any substantial work on integrating L4S into = QUIC until you have a good answer to that question. Alternative = approaches to high-fidelity congestion control exist which resolve that = particular conundrum easily. In particular, the ECN field feedback = mechanism in QUIC can also support SCE. - Jonathan Morton= From nobody Sun Nov 1 11:48:29 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 182FA3A0E1E for ; Sun, 1 Nov 2020 11:48:21 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.134 X-Spam-Level: X-Spam-Status: No, score=-2.134 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.247, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FG34qVCw6Gdk for ; Sun, 1 Nov 2020 11:48:19 -0800 (PST) Received: from mx36-out10.antispamcloud.com (mx36-out10.antispamcloud.com [209.126.121.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 38CDE3A0E17 for ; Sun, 1 Nov 2020 11:48:19 -0800 (PST) Received: from xse235.mail2web.com ([66.113.196.235] helo=xse.mail2web.com) by mx169.antispamcloud.com with esmtp (Exim 4.92) (envelope-from ) id 1kZJKn-000rc5-KP for tcpPrague@ietf.org; Sun, 01 Nov 2020 20:48:16 +0100 Received: from xsmtp22.mail2web.com (unknown [10.100.68.61]) by xse.mail2web.com (Postfix) with ESMTPS id 4CPRRc2NDHzqpp for ; Sun, 1 Nov 2020 11:48:12 -0800 (PST) Received: from [10.5.2.17] (helo=xmail07.myhosting.com) by xsmtp22.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1kZJKm-0000R3-7a for tcpPrague@ietf.org; Sun, 01 Nov 2020 11:48:12 -0800 Received: (qmail 7772 invoked from network); 1 Nov 2020 19:48:10 -0000 Received: from unknown (HELO [192.168.1.107]) (Authenticated-user:_huitema@huitema.net@[172.58.43.139]) (envelope-sender ) by xmail07.myhosting.com (qmail-ldap-1.03) with ESMTPA for ; 1 Nov 2020 19:48:10 -0000 To: Jonathan Morton Cc: iccrg IRTF list , Bob Briscoe , tsvwg IETF list , "De Schepper, Koen (Koen)" , TCP Prague List References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> From: Christian Huitema Autocrypt: addr=huitema@huitema.net; prefer-encrypt=mutual; keydata= mDMEXtavGxYJKwYBBAHaRw8BAQdA1ou9A5MHTP9N3jfsWzlDZ+jPnQkusmc7sfLmWVz1Rmu0 J0NocmlzdGlhbiBIdWl0ZW1hIDxodWl0ZW1hQGh1aXRlbWEubmV0PoiWBBMWCAA+FiEEw3G4 Nwi4QEpAAXUUELAmqKBYtJQFAl7WrxsCGwMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgEC F4AACgkQELAmqKBYtJQbMwD/ebj/qnSbthC/5kD5DxZ/Ip0CGJw5QBz/+fJp3R8iAlsBAMjK r2tmyWyJz0CUkVG24WaR5EAJDvgwDv8h22U6QVkAuDgEXtavGxIKKwYBBAGXVQEFAQEHQJoM 6MUAIqpoqdCIiACiEynZf7nlJg2Eu0pXIhbUGONdAwEIB4h+BBgWCAAmFiEEw3G4Nwi4QEpA AXUUELAmqKBYtJQFAl7WrxsCGwwFCQlmAYAACgkQELAmqKBYtJRm2wD7BzeK5gEXSmBcBf0j BYdSaJcXNzx4yPLbP4GnUMAyl2cBAJzcsR4RkwO4dCRqM9CHpVJCwHtbUDJaa55//E0kp+gH Message-ID: <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> Date: Sun, 1 Nov 2020 11:48:09 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> Content-Type: multipart/alternative; boundary="------------27D8CA2479553515CD6429CB" Content-Language: en-US X-Originating-IP: 66.113.196.235 X-Spampanel-Domain: xsmtpout.mail2web.com X-Spampanel-Username: 66.113.196.235/32 Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.196.235/32@xsmtpout.mail2web.com X-Spampanel-Outgoing-Class: unsure X-Spampanel-Outgoing-Evidence: Combined (0.15) X-Recommended-Action: accept X-Filter-ID: Mvzo4OR0dZXEDF/gcnlw0XvADx2zSFwG+3csxFBPBHmpSDasLI4SayDByyq9LIhVUZbR67CQ7/vm /hHDJU4RXkTNWdUk1Ol2OGx3IfrIJKywOmJyM1qr8uRnWBrbSAGD42+pgDSJdnVPeXOEK1m6uDbl kDZNUkuHiojIVh7uAUvAB1OJHBkjcz9byGlmlUIRUZq99/v+OWrp+SY7iSuea8EuNfnjQsRqWXYP fdewBkdPR9o014ICOs9pIWX8OtLJkJMlHRIWtRpp4V4PED3qDNbNhTNyx2YOvjKTJ3Ps/x2i3GSv wir0OshyEkOwfCYvxs5/T0oXYyKdvWabEYxQFC5Ano+acIK7aoZBBKN3kq8lTiPCf+PwTb/RKAdw /TswIg1fYmNb1kgzViAoNrSrXN1jhnM/Mbva2XLV/LIEzaJm4kYWBU//ZZ8pZ1xZfmaG1NRsUdzW awx6dX0NJ8Bzt99fxN2oReTDHAyOynaY0ClENaQq/2aUAxcG3yLqjApZmdySlZou9qHIGOZDEEo7 O58ZQzrOqjAERHu4pt/Ia6wELzcGxDgkPe7eR6qspNNQGjLhGMBSrFdf8dBbPvtqJwEiRQv+PVjj wa+Z5RFCOMTQVmTYR5SAzTRFbXVNTDv1gM5nBN3RJV9d5r5Q9lo7jszxgxhYBzM2+4BnTFvsXJXQ H50sZPeDQaETTMuyQC0ZCkbu0MIf5gIxteqwqB3UtXuXCC1A5Cukky0WFo38JXT3Y80OmAux3oN1 3+ztUznewCRQPWF2E4IsCIsjaWgrKL83o7TP54FLJfg9sR3SxHLLGnbyn7Qsr0CcmcAsjT/95XkJ osV7x+qD/ILIgQTnY4W+axx2DzNv8BNlsyaVn+WJZaCGVcqE0K0GEdVNC7oG3UVkhX78ZcdZtWJt ri4lXS4599fEDT1GENo62d+DqD+gavEa4Cf1ILpAKBLSDHQENgjpXL2y/ONOC04/YEfTu1ss+n2f fnQxt6aJ7klZab8CvOT2YjlrAxveXsTwUzCTkiX4qyX2d5a1xbDejUjyqRVeiJQ5XjnH4gzAuCMQ 8aUxL7hrJSk60SF3F6RYOYr2 X-Report-Abuse-To: spam@quarantine11.antispamcloud.com Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 19:48:23 -0000 This is a multi-part message in MIME format. --------------27D8CA2479553515CD6429CB Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/1/2020 9:00 AM, Jonathan Morton wrote: >> On 1 Nov, 2020, at 3:07 am, Christian Huitema wr= ote: >> >> I am reading the L4S ECN-AQM proposal with an eye on responding to it = in an implementation of QUIC, and I have a couple of questions regarding = use of ECN marking with QUIC.=20 >> >> The document does not mention QUIC, yet QUIC is already used in a larg= e fraction of Internet traffic. QUIC does specify support for ECN, and QU= IC acknowledgements may carry counts of each category of ECN marks receiv= ed from the peer -- three counters for ECT(0), ECT(1) and CE. In theory, = QUIC implementations could take advantage of L4S -- in fact, at least one= implementation supports DC-TCP like CC already. Is there interest in spe= cifying L4S for QUIC? >> >> My next question regards the interaction of the proposed L4S ECN-AQM w= ith CC algorithms like BBR that attempt to discover the bottleneck packet= rate for the connection, and use pacing to send packets at that rate. I = observed that BBR is never mentioned in the draft, yet BBR is used in a s= izeable part of Internet traffic. Do we have data on how a non-L4S aware = implementation of BBR interacts with the proposed L4S AQM? >> >> My last question regards potential use of ECT(1) marking. Most current= implementations set ECT(0), but setting ECT(1) instead is trivial. This = should elicit an L4S compatible response in L4S-AQM, and the BBR implemen= tation might be modified to use the signals as part of the bottleneck ban= dwidth tracking. But there is a small issue there. With BBR, QUIC packets= are supposedly paced at just under the bottleneck rate, except during "p= robe" periods in which they probe for 1 RTT at a slightly higher rate. Th= e L4S AGM might degenerate in a form of ON-OFF control -- no feedback at = all most of the time, then a bunch of CE marks if the probe rate exceeds = the bottleneck bandwidth. As anyone experimented with that? > In my view, the biggest question you should be asking is how QUIC will = distinguish between CE marks applied by an L4S AQM on the path, and those= applied by an RFC-3168 AQM on the path. The latter will treat ECT(1) ma= rked packets the same as ECT(0), and expects the same multiplicative-decr= ease congestion response, but L4S expects a qualitatively different linea= r response. Yes, that's a significant issue. As it is, nodes can only work with L4S on specific paths that are somehow guaranteed to not mix the "classic" and "L4S" behavior. For now, the only way I would implement it is by adding an "l4s" option, that would only be turned on by default when experimenting with L4S behavior. If we want any large scale deployment, we have to get an IETF consensus on the evolution of ECN -- something prescriptive, not just the "may experiment" of RFC 8311. Currently, implementations can only treat CE marks as an indication that they are probably sending faster than the network allows. > You should probably not do any substantial work on integrating L4S into= QUIC until you have a good answer to that question. Alternative approac= hes to high-fidelity congestion control exist which resolve that particul= ar conundrum easily. In particular, the ECN field feedback mechanism in = QUIC can also support SCE. Yes, QUIC implementations could indeed support SCE. But they could not support both SCE and L4S, and that's a big bummer. I wonder whether there is a way to unify the L4S and SCE behaviors. Transport protocol developers are unlikely to widely develop either option unless the IETF comes to a consensus. This is a shame, because ECN based congestion control does not have the ambiguity of loss-based CC: ECN marks are not ambiguous, they are clearly signals from the network, the application does not need to build logic to differentiate between congestion induced losses and random losses due to, for example, radio event. Failing that ECN unification CC algorithms end up relying a lot on bandwidth and delay measurements, but that is sub-optimal because some radio links do have variable bandwidth and variable delays. By the way, I do think that L4S makes too many hypotheses about the CC. I wish the AQM was based on intrinsic properties, such as "this flow is using more than its fair share of bandwidth" or "this link is congested and the transport should back off" rather than reasoning as if implementations were still using TCP-RENO, which neither Linux or Windows or Chrome does. -- Christian Huitema --------------27D8CA2479553515CD6429CB Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit


On 11/1/2020 9:00 AM, Jonathan Morton wrote:
On 1 Nov, 2020, at 3:07 am, Christian Huitema <huitema@huitema.net> wrote:

I am reading the L4S ECN-AQM proposal with an eye on responding to it in an implementation of QUIC, and I have a couple of questions regarding use of ECN marking with QUIC. 

The document does not mention QUIC, yet QUIC is already used in a large fraction of Internet traffic. QUIC does specify support for ECN, and QUIC acknowledgements may carry counts of each category of ECN marks received from the peer -- three counters for ECT(0), ECT(1) and CE. In theory, QUIC implementations could take advantage of L4S -- in fact, at least one implementation supports DC-TCP like CC already. Is there interest in specifying L4S for QUIC?

My next question regards the interaction of the proposed L4S ECN-AQM with CC algorithms like BBR that attempt to discover the bottleneck packet rate for the connection, and use pacing to send packets at that rate. I observed that BBR is never mentioned in the draft, yet BBR is used in a sizeable part of Internet traffic. Do we have data on how a non-L4S aware implementation of BBR interacts with the proposed L4S AQM?

My last question regards potential use of ECT(1) marking. Most current implementations set ECT(0), but setting ECT(1) instead is trivial. This should elicit an L4S compatible response in L4S-AQM, and the BBR implementation might be modified to use the signals as part of the bottleneck bandwidth tracking. But there is a small issue there. With BBR, QUIC packets are supposedly paced at just under the bottleneck rate, except during "probe" periods in which they probe for 1 RTT at a slightly higher rate. The L4S AGM might degenerate in a form of ON-OFF control -- no feedback at all most of the time, then a bunch of CE marks if the probe rate exceeds the bottleneck bandwidth. As anyone experimented with that?

In my view, the biggest question you should be asking is how QUIC will distinguish between CE marks applied by an L4S AQM on the path, and those applied by an RFC-3168 AQM on the path.  The latter will treat ECT(1) marked packets the same as ECT(0), and expects the same multiplicative-decrease congestion response, but L4S expects a qualitatively different linear response.

Yes, that's a significant issue. As it is, nodes can only work with L4S on specific paths that are somehow guaranteed to not mix the "classic" and "L4S" behavior. For now, the only way I would implement it is by adding an "l4s" option, that would only be turned on by default when experimenting with L4S behavior. If we want any large scale deployment, we have to get an IETF consensus on the evolution of ECN -- something prescriptive, not just the "may experiment" of RFC 8311. Currently, implementations can only treat CE marks as an indication that they are probably sending faster than the network allows.

You should probably not do any substantial work on integrating L4S into QUIC until you have a good answer to that question.  Alternative approaches to high-fidelity congestion control exist which resolve that particular conundrum easily.  In particular, the ECN field feedback mechanism in QUIC can also support SCE.

Yes, QUIC implementations could indeed support SCE. But they could not support both SCE and L4S, and that's a big bummer.

I wonder whether there is a way to unify the L4S and SCE behaviors. Transport protocol developers are unlikely to widely develop either option unless the IETF comes to a consensus. This is a shame, because ECN based congestion control does not have the ambiguity of loss-based CC: ECN marks are not ambiguous, they are clearly signals from the network, the application does not need to build logic to differentiate between congestion induced losses and random losses due to, for example, radio event. Failing that ECN unification CC algorithms end up relying a lot on bandwidth and delay measurements, but that is sub-optimal because some radio links do have variable bandwidth and variable delays.

By the way, I do think that L4S makes too many hypotheses about the CC. I wish the AQM was based on intrinsic properties, such as "this flow is using more than its fair share of bandwidth" or "this link is congested and the transport should back off" rather than reasoning as if implementations were still using TCP-RENO, which neither Linux or Windows or Chrome does.

-- Christian Huitema


--------------27D8CA2479553515CD6429CB-- From nobody Sun Nov 1 12:02:51 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C88C3A0E8C for ; Sun, 1 Nov 2020 12:02:43 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.135 X-Spam-Level: X-Spam-Status: No, score=-2.135 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-0.247, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dpiJgT9ZdRTx for ; Sun, 1 Nov 2020 12:02:42 -0800 (PST) Received: from mx43-out1.antispamcloud.com (mx43-out1.antispamcloud.com [138.201.61.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C08B93A0E77 for ; Sun, 1 Nov 2020 12:02:41 -0800 (PST) Received: from xse507.mail2web.com ([66.113.197.253] helo=xse.mail2web.com) by mx169.antispamcloud.com with esmtp (Exim 4.92) (envelope-from ) id 1kZJYZ-000xTw-I4 for tcpPrague@ietf.org; Sun, 01 Nov 2020 21:02:36 +0100 Received: from xsmtp22.mail2web.com (unknown [10.100.68.61]) by xse.mail2web.com (Postfix) with ESMTPS id 4CPRm20Q3wz9Vt for ; Sun, 1 Nov 2020 12:02:26 -0800 (PST) Received: from [10.5.2.15] (helo=xmail05.myhosting.com) by xsmtp22.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1kZJYX-0005gC-Uv for tcpPrague@ietf.org; Sun, 01 Nov 2020 12:02:25 -0800 Received: (qmail 24754 invoked from network); 1 Nov 2020 20:02:25 -0000 Received: from unknown (HELO [192.168.1.107]) (Authenticated-user:_huitema@huitema.net@[172.58.43.139]) (envelope-sender ) by xmail05.myhosting.com (qmail-ldap-1.03) with ESMTPA for ; 1 Nov 2020 20:02:25 -0000 To: Bob Briscoe , tsvwg IETF list Cc: iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> From: Christian Huitema Autocrypt: addr=huitema@huitema.net; prefer-encrypt=mutual; keydata= mDMEXtavGxYJKwYBBAHaRw8BAQdA1ou9A5MHTP9N3jfsWzlDZ+jPnQkusmc7sfLmWVz1Rmu0 J0NocmlzdGlhbiBIdWl0ZW1hIDxodWl0ZW1hQGh1aXRlbWEubmV0PoiWBBMWCAA+FiEEw3G4 Nwi4QEpAAXUUELAmqKBYtJQFAl7WrxsCGwMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgEC F4AACgkQELAmqKBYtJQbMwD/ebj/qnSbthC/5kD5DxZ/Ip0CGJw5QBz/+fJp3R8iAlsBAMjK r2tmyWyJz0CUkVG24WaR5EAJDvgwDv8h22U6QVkAuDgEXtavGxIKKwYBBAGXVQEFAQEHQJoM 6MUAIqpoqdCIiACiEynZf7nlJg2Eu0pXIhbUGONdAwEIB4h+BBgWCAAmFiEEw3G4Nwi4QEpA AXUUELAmqKBYtJQFAl7WrxsCGwwFCQlmAYAACgkQELAmqKBYtJRm2wD7BzeK5gEXSmBcBf0j BYdSaJcXNzx4yPLbP4GnUMAyl2cBAJzcsR4RkwO4dCRqM9CHpVJCwHtbUDJaa55//E0kp+gH Message-ID: <12c3cebf-e6e1-7c11-38cb-44c49c4e1d6e@huitema.net> Date: Sun, 1 Nov 2020 12:02:25 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: 66.113.197.253 X-Spampanel-Domain: xsmtpout.mail2web.com X-Spampanel-Username: 66.113.197.0/24 Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.197.0/24@xsmtpout.mail2web.com X-Spampanel-Outgoing-Class: unsure X-Spampanel-Outgoing-Evidence: Combined (0.15) X-Recommended-Action: accept X-Filter-ID: Mvzo4OR0dZXEDF/gcnlw0XvADx2zSFwG+3csxFBPBHmpSDasLI4SayDByyq9LIhVUZbR67CQ7/vm /hHDJU4RXkTNWdUk1Ol2OGx3IfrIJKywOmJyM1qr8uRnWBrbSAGD42+pgDSJdnVPeXOEK1m6uDbl kDZNUkuHiojIVh7uAUusCvX6lHrFbQfqNj8zeGJwUZq99/v+OWrp+SY7iSuea8EuNfnjQsRqWXYP fdewBkdPR9o014ICOs9pIWX8OtLJkJMlHRIWtRpp4V4PED3qDNbNhTNyx2YOvjKTJ3Ps/x2i3GSv wir0OshyEkOwfCYvxs5/T0oXYyKdvWabEYxQFC5Ano+acIK7aoZBBKN3kq8lTiPCf+PwTb/RKAdw /TswIg1fYmNb1kgzViAoNrSrXN1jhnM/Mbva2XLV/LIEzaJm4kYWBU//ZZ8pZ1xZfmaG1NRsUdzW awx6dX0NJ8Bzt99fxN2oReTDHAyOynaY0ClENaQq/2aUAxcG3yLqjApZmdySlZou9qHIGOZDEEo7 O58ZQzrOqjAERHu4pt/Ia6wELzcGxDgkPe7eR6qspNNQGjLhGMBSrFdf8dBbPvtqJwEiRQv+PVjj wa+Z5RFCOMTQVmTYR5SAzTRFbXVNTDv1gM5nBN3RJV9d5r5Q9lo7jrxFto+ioZBtsBgWtp7ZN2jB H1fP3YU+2XWwKzL04bcfwzQZWnqpeh+UbGCVNeqba5Xked+P+aSZU/EB7YnRWs2LBDMrD7q/cJog wbqzsuokqvIg/ybcHpfebz3r4YRfd07JPy9twDyaj6un7qWOkNfFCR9d0iUDYlosrtMLxUCkcdhM sjB58V23b46ATDy62ePXdRjeeYOc4D1auWIFhSHlgbhvINytQR7gmhjo7HZew6nIoDr0sXUZ7YZo Z/GZ+hXPnkLS9Oo2rnoDkPMmYws/jALIEk7e/m7I/2vCMQjvMFTIwLG5tR7EnM5HsSwoavLd14/y 82ebPziYNS9mrGfphl+Vcq8rhM3tJ8iXgDJaTYQ7ppmvpzmHp5jJAeLmE9zghLEjHJbHVHmv5sbq r/Q= X-Report-Abuse-To: spam@quarantine11.antispamcloud.com Archived-At: Subject: Re: [tcpPrague] L4S and BBR (was: [iccrg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text) X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 20:02:51 -0000 On 11/1/2020 5:30 AM, Bob Briscoe wrote: > > Quentin De Coninck wrote an initial implementation of a "QUIC Prague" > congestion control based on your picoQUIC code. It can be accessed via > the L4S landing page here: >     https://riteproject.eu/dctth/#code > He started it at an IETF hackathon. I don't think it's been maintained > since it was first written, but it might be a start. Care doing a PR so this could be added to the Picoquic distribution? -- Christian Huitema From nobody Sun Nov 1 12:17:14 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1A7ED3A0EBF; Sun, 1 Nov 2020 12:17:02 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.897 X-Spam-Level: X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Qcj9nODJnOJz; Sun, 1 Nov 2020 12:17:00 -0800 (PST) Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [137.50.19.135]) by ietfa.amsl.com (Postfix) with ESMTP id 4E50D3A0E9C; Sun, 1 Nov 2020 12:16:59 -0800 (PST) Received: from [192.168.1.70] (fgrpf.plus.com [212.159.18.54]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPSA id 00A7C1B000A3; Sun, 1 Nov 2020 20:16:51 +0000 (GMT) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable From: Gorry Fairhurst Mime-Version: 1.0 (1.0) Date: Sun, 1 Nov 2020 20:16:51 +0000 Message-Id: <694BE020-FA2B-4E34-B8DC-01377AA33B19@erg.abdn.ac.uk> References: <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> Cc: Jonathan Morton , iccrg IRTF list , tsvwg IETF list , "De Schepper, Koen (Koen)" , TCP Prague List In-Reply-To: <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> To: Christian Huitema X-Mailer: iPad Mail (17E262) Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 20:17:10 -0000 On just this one point: > On 1 Nov 2020, at 19:48, Christian Huitema wrote: >=20 > Yes, QUIC implementations could indeed support SCE. But they could not sup= port both SCE and L4S, and that's a big bummer. There is currently no intention to progress SCE within TSVWG, so support for= SCE within a PS for QUIC is really is a non-issue. Gorry= From nobody Sun Nov 1 12:24:38 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 62F653A0E5D; Sun, 1 Nov 2020 12:24:34 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.623 X-Spam-Level: X-Spam-Status: No, score=-1.623 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, KHOP_HELO_FCRDNS=0.275, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vxEiYLDbomag; Sun, 1 Nov 2020 12:24:33 -0800 (PST) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2ACCB3A0C98; Sun, 1 Nov 2020 12:24:32 -0800 (PST) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 0A1KNkWH011765; Sun, 1 Nov 2020 12:23:46 -0800 (PST) (envelope-from ietf@gndrsh.dnsmgr.net) Received: (from ietf@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 0A1KNkue011764; Sun, 1 Nov 2020 12:23:46 -0800 (PST) (envelope-from ietf) From: "Rodney W. Grimes" Message-Id: <202011012023.0A1KNkue011764@gndrsh.dnsmgr.net> In-Reply-To: <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> To: Bob Briscoe Date: Sun, 1 Nov 2020 12:23:46 -0800 (PST) CC: Christian Huitema , tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Archived-At: Subject: [tcpPrague] Realistic Queue delay targets X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 20:24:35 -0000 Bob, > Christian, > > I've changed the subject line given it's no longer appropriate. > See inline tagged [BB]... And I am changing it again... as only addressing a statement that I have great concerns about. See inline ragged [RWG] > On 01/11/2020 01:07, Christian Huitema wrote: ... [ much text deleted ] ... > [BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'allowed' > to send packets over the Internet as ECT(1). Then, yes, an L4S DualQ > Coupled AQM will classify BBRv2 packets into the L4S queue. This should > have a very shallow ECN marking threshold (500us - 1ms), so even if the > flow (whether QUIC or TCP) is flying just under the available capacity, > bunching of packets means it is unlikely to completely avoid ECN marking > between probes. If it could avoid ECN marking, you are right that it > would get a bump of ECN marks during the probe. I haven't studied the > code, but when it experiences ECN marking I believe it switches into an > L4S ECN mode for a while, and uses ECN rather than delay probing to > track available capacity. I assume it switches back to BBR's delay > probing mode if it gets no ECN for a while (e.g. the bottleneck might > have moved). But I haven't looked at BBRv2's ECN behaviour in detail. [RWG] I have great concern about this 500uS to 1mS ECN marking threshold given that I have recently learned the latest WiFi standards actually run with a packet aggregation time of 4mS thus making it probably impossible to have such traffic work in such a targeted low latency queue. [RWG] Has any consideration been given to what is already deployed on the Internet as link layer technologies that would preclude the operation of the L4S low latency queue? Regards, -- Rod Grimes rgrimes@freebsd.org From nobody Sun Nov 1 12:54:51 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7C9FB3A0100; Sun, 1 Nov 2020 12:54:43 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RrndkDh5hhnI; Sun, 1 Nov 2020 12:54:42 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 820E93A00E5; Sun, 1 Nov 2020 12:54:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1604264033; bh=HlsAAoo2ZFuGVL7fU+VlJ0O+9ssMSKMEnFdy2U8stMo=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=N+3SqtB8Aq7+1VHjF2YIXSKDRd8pEvXJtZ8gvsvNA5yr0qdv7cbOgkmZ5Eid2YzHQ MDL3o3F0R2OB+8N7mgVCwgScs7UZdwhu0oPeMMGDYxupjC0QR5S7keLcUM7JxwHI1A mYmHRxH5cRoluOtEKktcl7qmpfmKkF8ssyKH3q0U= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [134.76.241.253] ([134.76.241.253]) by web-mail.gmx.net (3c-app-gmx-bs25.server.lan [172.19.170.77]) (via HTTP); Sun, 1 Nov 2020 21:53:53 +0100 MIME-Version: 1.0 Message-ID: From: Sebastian Moeller To: Christian Huitema Cc: Bob Briscoe , tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" Content-Type: text/plain; charset=UTF-8 Date: Sun, 1 Nov 2020 21:53:53 +0100 Importance: normal Sensitivity: Normal In-Reply-To: <12c3cebf-e6e1-7c11-38cb-44c49c4e1d6e@huitema.net> References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> <12c3cebf-e6e1-7c11-38cb-44c49c4e1d6e@huitema.net> Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:VV60FY5lsFDZooeFb25qf5mD3I0It1mBsZH/qkteBW5hRaOiPmXAvWSDNV1Nk52yh3/Yk lvDTyqP9tzO90gSztZqTY75ieNDm3jagwRN7rsCZ1nRz6eKFBsC8v8u11QGHbggB/D7RSDyhfw4N vkbAWazC0YzRMpyUDsn+vPG/PyI+ErQ1E5nI2kr3QfnGWYi31nZ+37QDJLmNCWngGVyEzpUtzD99 XIw+s6cCAYlJZeXeHz4XtPIbSP/y5h8ddi1O/gk9aBQmplLkmSf5Cl9O4n08MQXoBps3Ndp3PNtc h0= X-UI-Out-Filterresults: notjunk:1;V03:K0:qaeKP8/XLS8=:4GsQGzZHRESSdSgl/1ZG0D TKgqTnayIK9bRfqrB4/L6lpkTrMH5wDHTR9bW0El33tvQHoc4Sn5o1QL+scUSW1WOH0pmfzmn YzKWxD0UYmpiHr0a4TuUyXJBjdPYlMraJMcPOOcRloMgKExnn7/rGcJaT+TciChMfAkvFtnSN Wh6BLJqLuncXYiPWXo35fVl8VoThmnEK1yOhlCY62ohrqWX8kg9WZzbM63mWnUk5mJ+jtvLKs VnoqE+TIFX32NTMIGKWdEDPADybpb/34CxkADYfIUuWabYCNzqPA44C/unWDb70uUOer7iKJ1 qWqFJMboamvFYW7TPTiQptWV9x7f1xy/AKLX6As5ePWDRxquFejxE1xHa5Ct9TyAh3vqDFYqs 9QjRbpP/dKqcfjkB3ehEh1+VCEeNssmj3SEob9ztOI9uZOg0M8uAxLGSJmH7RT5WrfxBvT3e3 t2f3Sz9fodT3UNR+JoJWG0zepdwlbBAufybEtop28v6CJ0leNnuIubYdYql7FjbuUhHUxNArq 7vgRteVEvY6BF5ghbeMyYyeGgtiBTEjaqdG8e1mDwMW03HCJDu3C/Lzf+GN3OVEIKewfvZdsn Tzh73AIjXOCSc= Archived-At: Subject: Re: [tcpPrague] [tsvwg] L4S and BBR (was: [iccrg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text) X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 20:54:44 -0000 Dear Christian, more below in-line, prefixed with [SM] > Gesendet: Sonntag, 01=2E November 2020 um 21:02 Uhr > Von: "Christian Huitema" > An: "Bob Briscoe" , "tsvwg IETF list" > Cc: "iccrg IRTF list" , "TCP Prague List" , "De Schepper, Koen (Koen)" > Betreff: Re: [tsvwg] [tcpPrague] L4S and BBR (was: [iccrg] ecn-l4s-id: P= roposed Changed to Normative Classic ECN detection Text) > > On 11/1/2020 5:30 AM, Bob Briscoe wrote: >=20 > > > > Quentin De Coninck wrote an initial implementation of a "QUIC Prague" > > congestion control based on your picoQUIC code=2E It can be accessed v= ia > > the L4S landing page here: > > =C2=A0=C2=A0=C2=A0 https://riteproject=2Eeu/dctth/#code > > He started it at an IETF hackathon=2E I don't think it's been maintain= ed > > since it was first written, but it might be a start=2E >=20 > Care doing a PR so this could be added to the Picoquic distribution? [SM] A cursory look at prague=2Ec at the repository indicates, that= this code might not fulfil the "eliminate RTT bias requirement" yet (I mig= ht have missed the implementation, hence "might not fulfil")=2E Odd since t= he repository is linked from the L4S site at rite, obviously just an oversi= ght that will be fixed in due time? Or a sign, that requiring individual protocols to fix up deficiencies in c= ore AQMs is simply an unsustainable approach, if even a team L4S endorsed i= mplementation is allowed to fall out of requirements=2E Best Regards Sebastian >=20 > -- Christian Huitema >=20 >=20 > From nobody Sun Nov 1 13:54:02 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D7C2E3A0994; Sun, 1 Nov 2020 13:53:49 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m8MsKXcMwnmD; Sun, 1 Nov 2020 13:53:47 -0800 (PST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2126.outbound.protection.outlook.com [40.107.21.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C486B3A098A; Sun, 1 Nov 2020 13:53:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W7tcnSGOmeSd0BegJta18fAhQl9uwPgFM88JCzylXvplAhiXsrOnwwzbt6KCMe5Gv7oh3H7N9XFQFaQXsWAK14e/2j4vshWEVB16q01Py8ODQr53tK5G7OukbxqlMdFfosx5mu+WdtBdfq7r1rvIepQyIvdGTSvVXkshwEu7CmIb/utkgTVzRPTTKOZrrQy7nx0AMrioNp/cHmuHSS8ByAf44SaHWL2qTlTj1Zd4L8IAIyuN6ejz1+1iYZkuH4MhRT1DjpfGDS/6fKJW7LHlb7P2r8xiZhzi2E2DYTOjpBqx3hPPqb7qewXsD6wQI8hxrUcHLMRtjR/3X02qse1Ezw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HrjYeGqKjuaurniTWFi3KSONoUWIbIzlf+Lzt9/MHJE=; b=bOYSVafyZXBLlgZNvfyI5D+d4kyZ6oJmRCWDpsOV0ZbH2aHZ/yHB7kkSM+zkYCMpI6inCGzdVfqjt2XR74ZNpIZlU1of25b6/8YJ/dgwWwYBAP1y6I10BFKB+6DvnmidGWBunfylh0DxUddFtznqOTZWnOloWZDPKU/HznhICUjBVqFaFYIefYgJ1pOH5U1SzOykKBX5L7Trk/3Hee+/nQxlh6/LWyui/om9pknXNOSJAAbpluUqzKOEshJhfF87WZxxOW+wxqT10RJOub+ljQ9j2T5wxrn2AUFeKN4j9ym2SzLMdUf7LXe+SHV4NZHJqc7x2rXKi0R14qR20AvIKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-bell-labs.com; dmarc=pass action=none header.from=nokia-bell-labs.com; dkim=pass header.d=nokia-bell-labs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HrjYeGqKjuaurniTWFi3KSONoUWIbIzlf+Lzt9/MHJE=; b=jTJG3rzcNA/FBZd7q7HGxoprgMOHExZoJANCJsLQX3Hvyz2hSTEZg/UJLYRueVNQPI8wbfoxxWwXh5yW4/otLQsIYVv3EGJHiYcrZ5ycZGCPxhcAiaI/U6ysOeqPHq8rsTdSx3Q7oXiv96gXV5aMugJ9EUEzvGSgr75lwPyF/p8= Received: from AM0PR07MB6114.eurprd07.prod.outlook.com (2603:10a6:208:113::33) by AM8PR07MB7345.eurprd07.prod.outlook.com (2603:10a6:20b:249::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.10; Sun, 1 Nov 2020 21:53:43 +0000 Received: from AM0PR07MB6114.eurprd07.prod.outlook.com ([fe80::a9f4:9199:8226:d6c6]) by AM0PR07MB6114.eurprd07.prod.outlook.com ([fe80::a9f4:9199:8226:d6c6%6]) with mapi id 15.20.3541.010; Sun, 1 Nov 2020 21:53:43 +0000 From: "De Schepper, Koen (Nokia - BE/Antwerp)" To: Christian Huitema , Jonathan Morton CC: iccrg IRTF list , Bob Briscoe , tsvwg IETF list , TCP Prague List Thread-Topic: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text Thread-Index: AQHWsIfwC/FrblPY50a/ND1AVr6ZVKmzwIVw Date: Sun, 1 Nov 2020 21:53:43 +0000 Message-ID: References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> In-Reply-To: <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: huitema.net; dkim=none (message not signed) header.d=none;huitema.net; dmarc=none action=none header.from=nokia-bell-labs.com; x-originating-ip: [81.82.56.234] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 5e36002d-9c76-49dd-6a99-08d87eb09a04 x-ms-traffictypediagnostic: AM8PR07MB7345: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: dNYAxAN3nWdS1D6ltpXUPqza0D+ThLn5WAIbtODPRwILh/VAlKsPZjAX52Ig329uSL9QJihFXAD3q8mwHrsGB01KjXWhQcZ552o+GIYMLkEIbzglIrnLKWsEpGXYJxeCxZLovXSDz+ifCXa6vZkNL8fOkvzVkyEmoAybeYMPbd50eHEUbmXEYAd0YHc9VWba1qh48YXCJsOMu2N8nALuMq+uUQrZuaVg6w3qx/5pyNXQqJ8AUNtQmFDAMLnbVY3qgBs8UE8dFMCPSxwqfzZGtaz0tgZHIzTXFPPyorBRiDC1kuid75C9HMFlqNfV4+5b9W9YoOLL866TqQ/JZhbEYQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR07MB6114.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(346002)(136003)(366004)(376002)(396003)(26005)(9686003)(110136005)(186003)(316002)(66574015)(83380400001)(4326008)(54906003)(6506007)(71200400001)(5660300002)(76116006)(478600001)(66476007)(86362001)(8936002)(9326002)(2906002)(7696005)(64756008)(33656002)(55016002)(52536014)(66946007)(53546011)(8676002)(66446008)(66556008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: MWzaBL/29f6Ad00raRHg/r0UjeoyHmhMhj+hEOnz+XA715GFR4yM08LvU4bCRsyVoj1zgJvC7xwsZgTeOhRFbmJXgRpp7Kxi3mOgfQDjYZkIies1oRE/uIiY5sAhzH+5Zm+oVE3gkaATx65KG1YocVxJ67ZZ/Rpa2sUGOWyJmTglZ+vznGjb0BVLXnyFRRJ1rHpmrnX7k2n/3HqIcPWYjvMKaF2Fbe42g2tOIm/0BHr5kQN3ASHnrFkbp/UZdmc5mL8/+OD/YhDyripbGXodRZ4LXaeHT7ahn19eO5s0iQa+GGGZTIUB+lGIIESWlQeyDIYyY97+/cZdMSNGMzpJXFtt1waDw6v5IlrOuZPIfqcR0zLl9yM6jCDgJQiZdfrP5GTj5MPXX+c/aD7q198A6mT7ldPSrPBGGjbsWNddMF5QyyZVlVQcVsnGKpFpizbQUjFQMpNUU8KuagxoT1jPcoMj1r6UsE2hE83ABngoR3U+2uNvwQ1W1DpQyuDiIkzPZZbMPczPEIs0/a76VsjLcdLakIPx6p4T4yJhtFBuqnnhHY2wswx1NRZ15mF9eRlvhcHF7VtOvmeEZMwpPlCJ3oV+RTVxQNyqa1f381jJEMohFerkMGP5sgvBkcLHl0kNTzaSLDBinX8zG/EmYCHJGg== x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR07MB6114083DB6F3FBA98AA2281FB9130AM0PR07MB6114eurp_" MIME-Version: 1.0 X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR07MB6114.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5e36002d-9c76-49dd-6a99-08d87eb09a04 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Nov 2020 21:53:43.5200 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: S3WH6BNeOdccIyrfprWcEiSLm41ZwndRWE1UIB6ymJVWv9/OOWkaUt10ACyxXVLiO9ydrXg1jn/rAOUxo6oIiNUoUoX/gVqIj+XbZtn5x9JDQaLL/NyS9qlo9/j2ERMV X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM8PR07MB7345 Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 21:53:50 -0000 --_000_AM0PR07MB6114083DB6F3FBA98AA2281FB9130AM0PR07MB6114eurp_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SGkgQ2hyaXN0aWFuLA0KDQoNCj4+QnkgdGhlIHdheSwgSSBkbyB0aGluayB0aGF0IEw0UyBtYWtl cyB0b28gbWFueSBoeXBvdGhlc2VzIGFib3V0IHRoZSBDQy4gSSB3aXNoIHRoZSBBUU0gd2FzIGJh c2VkIG9uIGludHJpbnNpYyBwcm9wZXJ0aWVzLCBzdWNoIGFzICJ0aGlzIGZsb3cgaXMgdXNpbmcg bW9yZSB0aGFuIGl0cyBmYWlyIHNoYXJlIG9mIGJhbmR3aWR0aCINCg0KVGhpcyBsYXN0IHF1b3Rl IGlzIGV4YWN0bHkgd2hhdCB0aGUgTDRTIG1hcmtpbmcgaXMgaW50ZW5kZWQgdG8gZG86IHRoZSBz bW9vdGhlZCBtYXJraW5nIHByb2JhYmlsaXR5IG92ZXIgYSBsb25nZXIgdGltZSBpbnRlcnZhbCAo YWJvdXQgMTAgUlRUcykgaW5kaWNhdGVzIHRoZSBmYWlyIHRhcmdldCByYXRlLiBBc3N1bWluZyB3 ZSBhZ3JlZSBvbiAxNW1zIGJlaW5nIHRoZSByZWZlcmVuY2UgUlRULCB0aGVuIHRoZSBhdmVyYWdl IHJhdGUgY2FuIGJlIHI9Mi8ocCowLjAxNSk9MTMzL3AgcGFja2V0cyBwZXIgc2Vjb25kLiBTbyB5 b3UgY2FuIGV2YWx1YXRlIHlvdXJzZWxmIGlmIHlvdSB1c2UgbW9yZSBvciBsZXNzIHRoYW4gdGhl IGZhaXIgc2hhcmUuIEhvdyB5b3UgcmVzcG9uZCB0byB0aGlzIGtub3dsZWRnZSBpcyB1cCB0byB5 b3VyIENDLg0KDQo+Pm9yICJ0aGlzIGxpbmsgaXMgY29uZ2VzdGVkIGFuZCB0aGUgdHJhbnNwb3J0 IHNob3VsZCBiYWNrIG9mZiINCg0KTDRTIG1hcmtzIGFsc28gZ2l2ZSBhIHZlcnkgYWNjdXJhdGUg cXVldWUgc2l6ZS9kZWxheSBpbmRpY2F0aW9uICh0aGUgcXVldWUgc2l6ZS9kZWxheSB0aGF0IHRo ZSBuZXR3b3JrIHByZWZlcnMpLiBTbyBrZWVwaW5nIHRyYWNrIG9mIHRoZSBSVFQgb2YgbWFya2Vk IHBhY2tldHMsIGVzcGVjaWFsbHkgd2hlbiB0aGV5IGNoYW5nZSBmcm9tIHVubWFya2VkIHRvIG1h cmtlZCBhbmQgdi92IGlzIGEgZ29vZCBpbmRpY2F0b3Igb2YgdGhlIHRhcmdldCBSVFQsIHdpdGhv dXQgdGhlIG5lZWQgdG8gYXNzdW1lIGEgdW5pdmVyc2FsIGRlbGF5IHRhcmdldC4NCg0KPj4gcmF0 aGVyIHRoYW4gcmVhc29uaW5nIGFzIGlmIGltcGxlbWVudGF0aW9ucyB3ZXJlIHN0aWxsIHVzaW5n IFRDUC1SRU5PLCB3aGljaCBuZWl0aGVyIExpbnV4IG9yIFdpbmRvd3Mgb3IgQ2hyb21lIGRvZXMu DQpXZSBvbmx5IGFzc3VtZSBUQ1AtZnJpZW5kbGluZXNzLCB3aGljaCBpcyB0eXBpY2FsbHkgYmFz ZWQgb24gdGhlIFJlbm8vQ3ViaWMgcmVmZXJlbmNlLiBJIGFncmVlLCB0aGVyZSBzaG91bGRu4oCZ dCBiZSBhbnkgZGVwZW5kZW5jeSBvbiB0aGUgaW1wbGVtZW50YXRpb24gb2YgdGhlIGNvbmdlc3Rp b24gY29udHJvbCBpdHNlbGYuDQoNCkkgd291bGQgYXNzdW1lIHRoYXQgQkJSIGNhbiBtb3JlIHRp Z2h0bHkgZm9sbG93IHRoZSBhdmFpbGFibGUgQlcgaWYgaXQgcmVjZWl2ZXMgQ0UgbWFya3MsIGFu ZCB0aGF0IHRoZSBCVyBwcm9iaW5nIGluY3JlYXNlL2ZyZXF1ZW5jeS9kdXJhdGlvbiBpbiB0aGF0 IGNhc2UgY2FuIGJlIGFkYXB0ZWQgdG8gdHJpZ2dlciB0aGUgcmlnaHQgbWFya2luZyBwcm9iYWJp bGl0eSBtYXRjaGluZyB0aGUgY3VycmVudCBCVy4gSWYgbm8gKG9yIGluc3VmZmljaWVudCkgbWFy a3MgYXJlIHJlY2VpdmVkLCB0aGUgbm9ybWFsIEJCUiBCVyBwcm9iaW5nIGNhbiBiZSB1c2VkLiBJ ZiB0b28gbWFueSBtYXJrcyBhcmUgcmVjZWl2ZWQsIHRoZSBCVyBuZWVkcyB0byBiZSByZWR1Y2Vk IGFuZCBub3JtYWwgQlcgcHJvYmluZyBjYW4gYmUgc3VwcHJlc3NlZC9yZWR1Y2Vk4oCmDQoNClJl Z2FyZHMsDQpLb2VuLg0KDQoNCg0KDQpGcm9tOiBDaHJpc3RpYW4gSHVpdGVtYSA8aHVpdGVtYUBo dWl0ZW1hLm5ldD4NClNlbnQ6IFN1bmRheSwgTm92ZW1iZXIgMSwgMjAyMCA4OjQ4IFBNDQpUbzog Sm9uYXRoYW4gTW9ydG9uIDxjaHJvbWF0aXg5OUBnbWFpbC5jb20+DQpDYzogaWNjcmcgSVJURiBs aXN0IDxpY2NyZ0BpcnRmLm9yZz47IEJvYiBCcmlzY29lIDxpZXRmQGJvYmJyaXNjb2UubmV0Pjsg dHN2d2cgSUVURiBsaXN0IDx0c3Z3Z0BpZXRmLm9yZz47IERlIFNjaGVwcGVyLCBLb2VuIChOb2tp YSAtIEJFL0FudHdlcnApIDxrb2VuLmRlX3NjaGVwcGVyQG5va2lhLWJlbGwtbGFicy5jb20+OyBU Q1AgUHJhZ3VlIExpc3QgPHRjcFByYWd1ZUBpZXRmLm9yZz4NClN1YmplY3Q6IFJlOiBbdGNwUHJh Z3VlXSBbdHN2d2ddIGVjbi1sNHMtaWQ6IFByb3Bvc2VkIENoYW5nZWQgdG8gTm9ybWF0aXZlIENs YXNzaWMgRUNOIGRldGVjdGlvbiBUZXh0DQoNCg0KDQpPbiAxMS8xLzIwMjAgOTowMCBBTSwgSm9u YXRoYW4gTW9ydG9uIHdyb3RlOg0KDQpPbiAxIE5vdiwgMjAyMCwgYXQgMzowNyBhbSwgQ2hyaXN0 aWFuIEh1aXRlbWEgPGh1aXRlbWFAaHVpdGVtYS5uZXQ+PG1haWx0bzpodWl0ZW1hQGh1aXRlbWEu bmV0PiB3cm90ZToNCg0KDQoNCkkgYW0gcmVhZGluZyB0aGUgTDRTIEVDTi1BUU0gcHJvcG9zYWwg d2l0aCBhbiBleWUgb24gcmVzcG9uZGluZyB0byBpdCBpbiBhbiBpbXBsZW1lbnRhdGlvbiBvZiBR VUlDLCBhbmQgSSBoYXZlIGEgY291cGxlIG9mIHF1ZXN0aW9ucyByZWdhcmRpbmcgdXNlIG9mIEVD TiBtYXJraW5nIHdpdGggUVVJQy4NCg0KDQoNClRoZSBkb2N1bWVudCBkb2VzIG5vdCBtZW50aW9u IFFVSUMsIHlldCBRVUlDIGlzIGFscmVhZHkgdXNlZCBpbiBhIGxhcmdlIGZyYWN0aW9uIG9mIElu dGVybmV0IHRyYWZmaWMuIFFVSUMgZG9lcyBzcGVjaWZ5IHN1cHBvcnQgZm9yIEVDTiwgYW5kIFFV SUMgYWNrbm93bGVkZ2VtZW50cyBtYXkgY2FycnkgY291bnRzIG9mIGVhY2ggY2F0ZWdvcnkgb2Yg RUNOIG1hcmtzIHJlY2VpdmVkIGZyb20gdGhlIHBlZXIgLS0gdGhyZWUgY291bnRlcnMgZm9yIEVD VCgwKSwgRUNUKDEpIGFuZCBDRS4gSW4gdGhlb3J5LCBRVUlDIGltcGxlbWVudGF0aW9ucyBjb3Vs ZCB0YWtlIGFkdmFudGFnZSBvZiBMNFMgLS0gaW4gZmFjdCwgYXQgbGVhc3Qgb25lIGltcGxlbWVu dGF0aW9uIHN1cHBvcnRzIERDLVRDUCBsaWtlIENDIGFscmVhZHkuIElzIHRoZXJlIGludGVyZXN0 IGluIHNwZWNpZnlpbmcgTDRTIGZvciBRVUlDPw0KDQoNCg0KTXkgbmV4dCBxdWVzdGlvbiByZWdh cmRzIHRoZSBpbnRlcmFjdGlvbiBvZiB0aGUgcHJvcG9zZWQgTDRTIEVDTi1BUU0gd2l0aCBDQyBh bGdvcml0aG1zIGxpa2UgQkJSIHRoYXQgYXR0ZW1wdCB0byBkaXNjb3ZlciB0aGUgYm90dGxlbmVj ayBwYWNrZXQgcmF0ZSBmb3IgdGhlIGNvbm5lY3Rpb24sIGFuZCB1c2UgcGFjaW5nIHRvIHNlbmQg cGFja2V0cyBhdCB0aGF0IHJhdGUuIEkgb2JzZXJ2ZWQgdGhhdCBCQlIgaXMgbmV2ZXIgbWVudGlv bmVkIGluIHRoZSBkcmFmdCwgeWV0IEJCUiBpcyB1c2VkIGluIGEgc2l6ZWFibGUgcGFydCBvZiBJ bnRlcm5ldCB0cmFmZmljLiBEbyB3ZSBoYXZlIGRhdGEgb24gaG93IGEgbm9uLUw0UyBhd2FyZSBp bXBsZW1lbnRhdGlvbiBvZiBCQlIgaW50ZXJhY3RzIHdpdGggdGhlIHByb3Bvc2VkIEw0UyBBUU0/ DQoNCg0KDQpNeSBsYXN0IHF1ZXN0aW9uIHJlZ2FyZHMgcG90ZW50aWFsIHVzZSBvZiBFQ1QoMSkg bWFya2luZy4gTW9zdCBjdXJyZW50IGltcGxlbWVudGF0aW9ucyBzZXQgRUNUKDApLCBidXQgc2V0 dGluZyBFQ1QoMSkgaW5zdGVhZCBpcyB0cml2aWFsLiBUaGlzIHNob3VsZCBlbGljaXQgYW4gTDRT IGNvbXBhdGlibGUgcmVzcG9uc2UgaW4gTDRTLUFRTSwgYW5kIHRoZSBCQlIgaW1wbGVtZW50YXRp b24gbWlnaHQgYmUgbW9kaWZpZWQgdG8gdXNlIHRoZSBzaWduYWxzIGFzIHBhcnQgb2YgdGhlIGJv dHRsZW5lY2sgYmFuZHdpZHRoIHRyYWNraW5nLiBCdXQgdGhlcmUgaXMgYSBzbWFsbCBpc3N1ZSB0 aGVyZS4gV2l0aCBCQlIsIFFVSUMgcGFja2V0cyBhcmUgc3VwcG9zZWRseSBwYWNlZCBhdCBqdXN0 IHVuZGVyIHRoZSBib3R0bGVuZWNrIHJhdGUsIGV4Y2VwdCBkdXJpbmcgInByb2JlIiBwZXJpb2Rz IGluIHdoaWNoIHRoZXkgcHJvYmUgZm9yIDEgUlRUIGF0IGEgc2xpZ2h0bHkgaGlnaGVyIHJhdGUu IFRoZSBMNFMgQUdNIG1pZ2h0IGRlZ2VuZXJhdGUgaW4gYSBmb3JtIG9mIE9OLU9GRiBjb250cm9s IC0tIG5vIGZlZWRiYWNrIGF0IGFsbCBtb3N0IG9mIHRoZSB0aW1lLCB0aGVuIGEgYnVuY2ggb2Yg Q0UgbWFya3MgaWYgdGhlIHByb2JlIHJhdGUgZXhjZWVkcyB0aGUgYm90dGxlbmVjayBiYW5kd2lk dGguIEFzIGFueW9uZSBleHBlcmltZW50ZWQgd2l0aCB0aGF0Pw0KDQpJbiBteSB2aWV3LCB0aGUg YmlnZ2VzdCBxdWVzdGlvbiB5b3Ugc2hvdWxkIGJlIGFza2luZyBpcyBob3cgUVVJQyB3aWxsIGRp c3Rpbmd1aXNoIGJldHdlZW4gQ0UgbWFya3MgYXBwbGllZCBieSBhbiBMNFMgQVFNIG9uIHRoZSBw YXRoLCBhbmQgdGhvc2UgYXBwbGllZCBieSBhbiBSRkMtMzE2OCBBUU0gb24gdGhlIHBhdGguICBU aGUgbGF0dGVyIHdpbGwgdHJlYXQgRUNUKDEpIG1hcmtlZCBwYWNrZXRzIHRoZSBzYW1lIGFzIEVD VCgwKSwgYW5kIGV4cGVjdHMgdGhlIHNhbWUgbXVsdGlwbGljYXRpdmUtZGVjcmVhc2UgY29uZ2Vz dGlvbiByZXNwb25zZSwgYnV0IEw0UyBleHBlY3RzIGEgcXVhbGl0YXRpdmVseSBkaWZmZXJlbnQg bGluZWFyIHJlc3BvbnNlLg0KDQpZZXMsIHRoYXQncyBhIHNpZ25pZmljYW50IGlzc3VlLiBBcyBp dCBpcywgbm9kZXMgY2FuIG9ubHkgd29yayB3aXRoIEw0UyBvbiBzcGVjaWZpYyBwYXRocyB0aGF0 IGFyZSBzb21laG93IGd1YXJhbnRlZWQgdG8gbm90IG1peCB0aGUgImNsYXNzaWMiIGFuZCAiTDRT IiBiZWhhdmlvci4gRm9yIG5vdywgdGhlIG9ubHkgd2F5IEkgd291bGQgaW1wbGVtZW50IGl0IGlz IGJ5IGFkZGluZyBhbiAibDRzIiBvcHRpb24sIHRoYXQgd291bGQgb25seSBiZSB0dXJuZWQgb24g YnkgZGVmYXVsdCB3aGVuIGV4cGVyaW1lbnRpbmcgd2l0aCBMNFMgYmVoYXZpb3IuIElmIHdlIHdh bnQgYW55IGxhcmdlIHNjYWxlIGRlcGxveW1lbnQsIHdlIGhhdmUgdG8gZ2V0IGFuIElFVEYgY29u c2Vuc3VzIG9uIHRoZSBldm9sdXRpb24gb2YgRUNOIC0tIHNvbWV0aGluZyBwcmVzY3JpcHRpdmUs IG5vdCBqdXN0IHRoZSAibWF5IGV4cGVyaW1lbnQiIG9mIFJGQyA4MzExLiBDdXJyZW50bHksIGlt cGxlbWVudGF0aW9ucyBjYW4gb25seSB0cmVhdCBDRSBtYXJrcyBhcyBhbiBpbmRpY2F0aW9uIHRo YXQgdGhleSBhcmUgcHJvYmFibHkgc2VuZGluZyBmYXN0ZXIgdGhhbiB0aGUgbmV0d29yayBhbGxv d3MuDQoNCg0KDQpZb3Ugc2hvdWxkIHByb2JhYmx5IG5vdCBkbyBhbnkgc3Vic3RhbnRpYWwgd29y ayBvbiBpbnRlZ3JhdGluZyBMNFMgaW50byBRVUlDIHVudGlsIHlvdSBoYXZlIGEgZ29vZCBhbnN3 ZXIgdG8gdGhhdCBxdWVzdGlvbi4gIEFsdGVybmF0aXZlIGFwcHJvYWNoZXMgdG8gaGlnaC1maWRl bGl0eSBjb25nZXN0aW9uIGNvbnRyb2wgZXhpc3Qgd2hpY2ggcmVzb2x2ZSB0aGF0IHBhcnRpY3Vs YXIgY29udW5kcnVtIGVhc2lseS4gIEluIHBhcnRpY3VsYXIsIHRoZSBFQ04gZmllbGQgZmVlZGJh Y2sgbWVjaGFuaXNtIGluIFFVSUMgY2FuIGFsc28gc3VwcG9ydCBTQ0UuDQoNClllcywgUVVJQyBp bXBsZW1lbnRhdGlvbnMgY291bGQgaW5kZWVkIHN1cHBvcnQgU0NFLiBCdXQgdGhleSBjb3VsZCBu b3Qgc3VwcG9ydCBib3RoIFNDRSBhbmQgTDRTLCBhbmQgdGhhdCdzIGEgYmlnIGJ1bW1lci4NCg0K SSB3b25kZXIgd2hldGhlciB0aGVyZSBpcyBhIHdheSB0byB1bmlmeSB0aGUgTDRTIGFuZCBTQ0Ug YmVoYXZpb3JzLiBUcmFuc3BvcnQgcHJvdG9jb2wgZGV2ZWxvcGVycyBhcmUgdW5saWtlbHkgdG8g d2lkZWx5IGRldmVsb3AgZWl0aGVyIG9wdGlvbiB1bmxlc3MgdGhlIElFVEYgY29tZXMgdG8gYSBj b25zZW5zdXMuIFRoaXMgaXMgYSBzaGFtZSwgYmVjYXVzZSBFQ04gYmFzZWQgY29uZ2VzdGlvbiBj b250cm9sIGRvZXMgbm90IGhhdmUgdGhlIGFtYmlndWl0eSBvZiBsb3NzLWJhc2VkIENDOiBFQ04g bWFya3MgYXJlIG5vdCBhbWJpZ3VvdXMsIHRoZXkgYXJlIGNsZWFybHkgc2lnbmFscyBmcm9tIHRo ZSBuZXR3b3JrLCB0aGUgYXBwbGljYXRpb24gZG9lcyBub3QgbmVlZCB0byBidWlsZCBsb2dpYyB0 byBkaWZmZXJlbnRpYXRlIGJldHdlZW4gY29uZ2VzdGlvbiBpbmR1Y2VkIGxvc3NlcyBhbmQgcmFu ZG9tIGxvc3NlcyBkdWUgdG8sIGZvciBleGFtcGxlLCByYWRpbyBldmVudC4gRmFpbGluZyB0aGF0 IEVDTiB1bmlmaWNhdGlvbiBDQyBhbGdvcml0aG1zIGVuZCB1cCByZWx5aW5nIGEgbG90IG9uIGJh bmR3aWR0aCBhbmQgZGVsYXkgbWVhc3VyZW1lbnRzLCBidXQgdGhhdCBpcyBzdWItb3B0aW1hbCBi ZWNhdXNlIHNvbWUgcmFkaW8gbGlua3MgZG8gaGF2ZSB2YXJpYWJsZSBiYW5kd2lkdGggYW5kIHZh cmlhYmxlIGRlbGF5cy4NCg0KQnkgdGhlIHdheSwgSSBkbyB0aGluayB0aGF0IEw0UyBtYWtlcyB0 b28gbWFueSBoeXBvdGhlc2VzIGFib3V0IHRoZSBDQy4gSSB3aXNoIHRoZSBBUU0gd2FzIGJhc2Vk IG9uIGludHJpbnNpYyBwcm9wZXJ0aWVzLCBzdWNoIGFzICJ0aGlzIGZsb3cgaXMgdXNpbmcgbW9y ZSB0aGFuIGl0cyBmYWlyIHNoYXJlIG9mIGJhbmR3aWR0aCIgb3IgInRoaXMgbGluayBpcyBjb25n ZXN0ZWQgYW5kIHRoZSB0cmFuc3BvcnQgc2hvdWxkIGJhY2sgb2ZmIiByYXRoZXIgdGhhbiByZWFz b25pbmcgYXMgaWYgaW1wbGVtZW50YXRpb25zIHdlcmUgc3RpbGwgdXNpbmcgVENQLVJFTk8sIHdo aWNoIG5laXRoZXIgTGludXggb3IgV2luZG93cyBvciBDaHJvbWUgZG9lcy4NCg0KLS0gQ2hyaXN0 aWFuIEh1aXRlbWENCg0KDQo= --_000_AM0PR07MB6114083DB6F3FBA98AA2281FB9130AM0PR07MB6114eurp_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy IDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpDb25zb2xhczsNCglwYW5vc2UtMToyIDEx IDYgOSAyIDIgNCAzIDIgNDt9DQovKiBTdHlsZSBEZWZpbml0aW9ucyAqLw0KcC5Nc29Ob3JtYWws IGxpLk1zb05vcm1hbCwgZGl2Lk1zb05vcm1hbA0KCXttYXJnaW46MGNtOw0KCW1hcmdpbi1ib3R0 b206LjAwMDFwdDsNCglmb250LXNpemU6MTEuMHB0Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixz YW5zLXNlcmlmO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNvLXN0eWxlLXByaW9y aXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQpwcmUN Cgl7bXNvLXN0eWxlLXByaW9yaXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOiJIVE1MIFByZWZvcm1h dHRlZCBDaGFyIjsNCgltYXJnaW46MGNtOw0KCW1hcmdpbi1ib3R0b206LjAwMDFwdDsNCglmb250 LXNpemU6MTAuMHB0Ow0KCWZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7fQ0Kc3Bhbi5IVE1MUHJl Zm9ybWF0dGVkQ2hhcg0KCXttc28tc3R5bGUtbmFtZToiSFRNTCBQcmVmb3JtYXR0ZWQgQ2hhciI7 DQoJbXNvLXN0eWxlLXByaW9yaXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOiJIVE1MIFByZWZvcm1h dHRlZCI7DQoJZm9udC1mYW1pbHk6Q29uc29sYXM7fQ0Kc3Bhbi5FbWFpbFN0eWxlMjENCgl7bXNv LXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMt c2VyaWY7DQoJY29sb3I6d2luZG93dGV4dDt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28tc3R5bGUt dHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdvcmRTZWN0aW9u MQ0KCXtzaXplOjYxMi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzIuMHB0IDcyLjBwdCA3Mi4wcHQg NzIuMHB0O30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0KLS0+PC9z dHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlZGVmYXVsdHMgdjpleHQ9ImVk aXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48IVtlbmRpZl0tLT48IS0tW2lmIGd0ZSBtc28g OV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0PSJlZGl0Ij4NCjxvOmlkbWFwIHY6ZXh0PSJl ZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwhW2VuZGlmXS0tPg0KPC9o ZWFkPg0KPGJvZHkgbGFuZz0iRU4tVVMiIGxpbms9ImJsdWUiIHZsaW5rPSJwdXJwbGUiPg0KPGRp diBjbGFzcz0iV29yZFNlY3Rpb24xIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkhpIENocmlzdGlh biw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+ PC9wPg0KPHA+Jmd0OyZndDtCeSB0aGUgd2F5LCBJIGRvIHRoaW5rIHRoYXQgTDRTIG1ha2VzIHRv byBtYW55IGh5cG90aGVzZXMgYWJvdXQgdGhlIENDLiBJIHdpc2ggdGhlIEFRTSB3YXMgYmFzZWQg b24gaW50cmluc2ljIHByb3BlcnRpZXMsIHN1Y2ggYXMgJnF1b3Q7dGhpcyBmbG93IGlzIHVzaW5n IG1vcmUgdGhhbiBpdHMgZmFpciBzaGFyZSBvZiBiYW5kd2lkdGgmcXVvdDsNCjxvOnA+PC9vOnA+ PC9wPg0KPHA+VGhpcyBsYXN0IHF1b3RlIGlzIGV4YWN0bHkgd2hhdCB0aGUgTDRTIG1hcmtpbmcg aXMgaW50ZW5kZWQgdG8gZG86IHRoZSBzbW9vdGhlZCBtYXJraW5nIHByb2JhYmlsaXR5IG92ZXIg YSBsb25nZXIgdGltZSBpbnRlcnZhbCAoYWJvdXQgMTAgUlRUcykgaW5kaWNhdGVzIHRoZSBmYWly IHRhcmdldCByYXRlLiBBc3N1bWluZyB3ZSBhZ3JlZSBvbiAxNW1zIGJlaW5nIHRoZSByZWZlcmVu Y2UgUlRULCB0aGVuIHRoZSBhdmVyYWdlIHJhdGUgY2FuIGJlDQogcj0yLyhwKjAuMDE1KT0xMzMv cCBwYWNrZXRzIHBlciBzZWNvbmQuIFNvIHlvdSBjYW4gZXZhbHVhdGUgeW91cnNlbGYgaWYgeW91 IHVzZSBtb3JlIG9yIGxlc3MgdGhhbiB0aGUgZmFpciBzaGFyZS4gSG93IHlvdSByZXNwb25kIHRv IHRoaXMga25vd2xlZGdlIGlzIHVwIHRvIHlvdXIgQ0MuPG86cD48L286cD48L3A+DQo8cD4mZ3Q7 Jmd0O29yICZxdW90O3RoaXMgbGluayBpcyBjb25nZXN0ZWQgYW5kIHRoZSB0cmFuc3BvcnQgc2hv dWxkIGJhY2sgb2ZmJnF1b3Q7PG86cD48L286cD48L3A+DQo8cD5MNFMgbWFya3MgYWxzbyBnaXZl IGEgdmVyeSBhY2N1cmF0ZSBxdWV1ZSBzaXplL2RlbGF5IGluZGljYXRpb24gKHRoZSBxdWV1ZSBz aXplL2RlbGF5IHRoYXQgdGhlIG5ldHdvcmsgcHJlZmVycykuIFNvIGtlZXBpbmcgdHJhY2sgb2Yg dGhlIFJUVCBvZiBtYXJrZWQgcGFja2V0cywgZXNwZWNpYWxseSB3aGVuIHRoZXkgY2hhbmdlIGZy b20gdW5tYXJrZWQgdG8gbWFya2VkIGFuZCB2L3YgaXMgYSBnb29kIGluZGljYXRvciBvZiB0aGUg dGFyZ2V0DQogUlRULCB3aXRob3V0IHRoZSBuZWVkIHRvIGFzc3VtZSBhIHVuaXZlcnNhbCBkZWxh eSB0YXJnZXQuPG86cD48L286cD48L3A+DQo8cD4mZ3Q7Jmd0OyByYXRoZXIgdGhhbiByZWFzb25p bmcgYXMgaWYgaW1wbGVtZW50YXRpb25zIHdlcmUgc3RpbGwgdXNpbmcgVENQLVJFTk8sIHdoaWNo IG5laXRoZXIgTGludXggb3IgV2luZG93cyBvciBDaHJvbWUgZG9lcy48bzpwPjwvbzpwPjwvcD4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPldlIG9ubHkgYXNzdW1lIFRDUC1mcmllbmRsaW5lc3MsIHdo aWNoIGlzIHR5cGljYWxseSBiYXNlZCBvbiB0aGUgUmVuby9DdWJpYyByZWZlcmVuY2UuIEkgYWdy ZWUsIHRoZXJlIHNob3VsZG7igJl0IGJlIGFueSBkZXBlbmRlbmN5IG9uIHRoZSBpbXBsZW1lbnRh dGlvbiBvZiB0aGUgY29uZ2VzdGlvbiBjb250cm9sIGl0c2VsZi48bzpwPjwvbzpwPjwvcD4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05v cm1hbCI+SSB3b3VsZCBhc3N1bWUgdGhhdCBCQlIgY2FuIG1vcmUgdGlnaHRseSBmb2xsb3cgdGhl IGF2YWlsYWJsZSBCVyBpZiBpdCByZWNlaXZlcyBDRSBtYXJrcywgYW5kIHRoYXQgdGhlIEJXIHBy b2JpbmcgaW5jcmVhc2UvZnJlcXVlbmN5L2R1cmF0aW9uIGluIHRoYXQgY2FzZSBjYW4gYmUgYWRh cHRlZCB0byB0cmlnZ2VyIHRoZSByaWdodCBtYXJraW5nIHByb2JhYmlsaXR5IG1hdGNoaW5nIHRo ZSBjdXJyZW50IEJXLg0KIElmIG5vIChvciBpbnN1ZmZpY2llbnQpIG1hcmtzIGFyZSByZWNlaXZl ZCwgdGhlIG5vcm1hbCBCQlIgQlcgcHJvYmluZyBjYW4gYmUgdXNlZC4gSWYgdG9vIG1hbnkgbWFy a3MgYXJlIHJlY2VpdmVkLCB0aGUgQlcgbmVlZHMgdG8gYmUgcmVkdWNlZCBhbmQgbm9ybWFsIEJX IHByb2JpbmcgY2FuIGJlIHN1cHByZXNzZWQvcmVkdWNlZOKApjxvOnA+PC9vOnA+PC9wPg0KPHAg Y2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9y bWFsIj5SZWdhcmRzLDxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+S29lbi48 bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9w Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8cCBjbGFzcz0i TXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxv OnA+Jm5ic3A7PC9vOnA+PC9wPg0KPGRpdj4NCjxkaXYgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRl ci10b3A6c29saWQgI0UxRTFFMSAxLjBwdDtwYWRkaW5nOjMuMHB0IDBjbSAwY20gMGNtIj4NCjxw IGNsYXNzPSJNc29Ob3JtYWwiPjxiPkZyb206PC9iPiBDaHJpc3RpYW4gSHVpdGVtYSAmbHQ7aHVp dGVtYUBodWl0ZW1hLm5ldCZndDsgPGJyPg0KPGI+U2VudDo8L2I+IFN1bmRheSwgTm92ZW1iZXIg MSwgMjAyMCA4OjQ4IFBNPGJyPg0KPGI+VG86PC9iPiBKb25hdGhhbiBNb3J0b24gJmx0O2Nocm9t YXRpeDk5QGdtYWlsLmNvbSZndDs8YnI+DQo8Yj5DYzo8L2I+IGljY3JnIElSVEYgbGlzdCAmbHQ7 aWNjcmdAaXJ0Zi5vcmcmZ3Q7OyBCb2IgQnJpc2NvZSAmbHQ7aWV0ZkBib2JicmlzY29lLm5ldCZn dDs7IHRzdndnIElFVEYgbGlzdCAmbHQ7dHN2d2dAaWV0Zi5vcmcmZ3Q7OyBEZSBTY2hlcHBlciwg S29lbiAoTm9raWEgLSBCRS9BbnR3ZXJwKSAmbHQ7a29lbi5kZV9zY2hlcHBlckBub2tpYS1iZWxs LWxhYnMuY29tJmd0OzsgVENQIFByYWd1ZSBMaXN0ICZsdDt0Y3BQcmFndWVAaWV0Zi5vcmcmZ3Q7 PGJyPg0KPGI+U3ViamVjdDo8L2I+IFJlOiBbdGNwUHJhZ3VlXSBbdHN2d2ddIGVjbi1sNHMtaWQ6 IFByb3Bvc2VkIENoYW5nZWQgdG8gTm9ybWF0aXZlIENsYXNzaWMgRUNOIGRldGVjdGlvbiBUZXh0 PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86 cD4mbmJzcDs8L286cD48L3A+DQo8cD48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxkaXY+DQo8cCBj bGFzcz0iTXNvTm9ybWFsIj5PbiAxMS8xLzIwMjAgOTowMCBBTSwgSm9uYXRoYW4gTW9ydG9uIHdy b3RlOjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0ibWFyZ2luLXRv cDo1LjBwdDttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxibG9ja3F1b3RlIHN0eWxlPSJtYXJnaW4t dG9wOjUuMHB0O21hcmdpbi1ib3R0b206NS4wcHQiPg0KPHByZT48c3BhbiBzdHlsZT0iY29sb3I6 YmxhY2siPk9uIDEgTm92LCAyMDIwLCBhdCAzOjA3IGFtLCBDaHJpc3RpYW4gSHVpdGVtYSA8YSBo cmVmPSJtYWlsdG86aHVpdGVtYUBodWl0ZW1hLm5ldCI+Jmx0O2h1aXRlbWFAaHVpdGVtYS5uZXQm Z3Q7PC9hPiB3cm90ZTo8bzpwPjwvbzpwPjwvc3Bhbj48L3ByZT4NCjxwcmU+PHNwYW4gc3R5bGU9 ImNvbG9yOmJsYWNrIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3ByZT4NCjxwcmU+PHNwYW4g c3R5bGU9ImNvbG9yOmJsYWNrIj5JIGFtIHJlYWRpbmcgdGhlIEw0UyBFQ04tQVFNIHByb3Bvc2Fs IHdpdGggYW4gZXllIG9uIHJlc3BvbmRpbmcgdG8gaXQgaW4gYW4gaW1wbGVtZW50YXRpb24gb2Yg UVVJQywgYW5kIEkgaGF2ZSBhIGNvdXBsZSBvZiBxdWVzdGlvbnMgcmVnYXJkaW5nIHVzZSBvZiBF Q04gbWFya2luZyB3aXRoIFFVSUMuIDxvOnA+PC9vOnA+PC9zcGFuPjwvcHJlPg0KPHByZT48c3Bh biBzdHlsZT0iY29sb3I6YmxhY2siPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcHJlPg0KPHBy ZT48c3BhbiBzdHlsZT0iY29sb3I6YmxhY2siPlRoZSBkb2N1bWVudCBkb2VzIG5vdCBtZW50aW9u IFFVSUMsIHlldCBRVUlDIGlzIGFscmVhZHkgdXNlZCBpbiBhIGxhcmdlIGZyYWN0aW9uIG9mIElu dGVybmV0IHRyYWZmaWMuIFFVSUMgZG9lcyBzcGVjaWZ5IHN1cHBvcnQgZm9yIEVDTiwgYW5kIFFV SUMgYWNrbm93bGVkZ2VtZW50cyBtYXkgY2FycnkgY291bnRzIG9mIGVhY2ggY2F0ZWdvcnkgb2Yg RUNOIG1hcmtzIHJlY2VpdmVkIGZyb20gdGhlIHBlZXIgLS0gdGhyZWUgY291bnRlcnMgZm9yIEVD VCgwKSwgRUNUKDEpIGFuZCBDRS4gSW4gdGhlb3J5LCBRVUlDIGltcGxlbWVudGF0aW9ucyBjb3Vs ZCB0YWtlIGFkdmFudGFnZSBvZiBMNFMgLS0gaW4gZmFjdCwgYXQgbGVhc3Qgb25lIGltcGxlbWVu dGF0aW9uIHN1cHBvcnRzIERDLVRDUCBsaWtlIENDIGFscmVhZHkuIElzIHRoZXJlIGludGVyZXN0 IGluIHNwZWNpZnlpbmcgTDRTIGZvciBRVUlDPzxvOnA+PC9vOnA+PC9zcGFuPjwvcHJlPg0KPHBy ZT48c3BhbiBzdHlsZT0iY29sb3I6YmxhY2siPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcHJl Pg0KPHByZT48c3BhbiBzdHlsZT0iY29sb3I6YmxhY2siPk15IG5leHQgcXVlc3Rpb24gcmVnYXJk cyB0aGUgaW50ZXJhY3Rpb24gb2YgdGhlIHByb3Bvc2VkIEw0UyBFQ04tQVFNIHdpdGggQ0MgYWxn b3JpdGhtcyBsaWtlIEJCUiB0aGF0IGF0dGVtcHQgdG8gZGlzY292ZXIgdGhlIGJvdHRsZW5lY2sg cGFja2V0IHJhdGUgZm9yIHRoZSBjb25uZWN0aW9uLCBhbmQgdXNlIHBhY2luZyB0byBzZW5kIHBh Y2tldHMgYXQgdGhhdCByYXRlLiBJIG9ic2VydmVkIHRoYXQgQkJSIGlzIG5ldmVyIG1lbnRpb25l ZCBpbiB0aGUgZHJhZnQsIHlldCBCQlIgaXMgdXNlZCBpbiBhIHNpemVhYmxlIHBhcnQgb2YgSW50 ZXJuZXQgdHJhZmZpYy4gRG8gd2UgaGF2ZSBkYXRhIG9uIGhvdyBhIG5vbi1MNFMgYXdhcmUgaW1w bGVtZW50YXRpb24gb2YgQkJSIGludGVyYWN0cyB3aXRoIHRoZSBwcm9wb3NlZCBMNFMgQVFNPzxv OnA+PC9vOnA+PC9zcGFuPjwvcHJlPg0KPHByZT48c3BhbiBzdHlsZT0iY29sb3I6YmxhY2siPjxv OnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcHJlPg0KPHByZT48c3BhbiBzdHlsZT0iY29sb3I6Ymxh Y2siPk15IGxhc3QgcXVlc3Rpb24gcmVnYXJkcyBwb3RlbnRpYWwgdXNlIG9mIEVDVCgxKSBtYXJr aW5nLiBNb3N0IGN1cnJlbnQgaW1wbGVtZW50YXRpb25zIHNldCBFQ1QoMCksIGJ1dCBzZXR0aW5n IEVDVCgxKSBpbnN0ZWFkIGlzIHRyaXZpYWwuIFRoaXMgc2hvdWxkIGVsaWNpdCBhbiBMNFMgY29t cGF0aWJsZSByZXNwb25zZSBpbiBMNFMtQVFNLCBhbmQgdGhlIEJCUiBpbXBsZW1lbnRhdGlvbiBt aWdodCBiZSBtb2RpZmllZCB0byB1c2UgdGhlIHNpZ25hbHMgYXMgcGFydCBvZiB0aGUgYm90dGxl bmVjayBiYW5kd2lkdGggdHJhY2tpbmcuIEJ1dCB0aGVyZSBpcyBhIHNtYWxsIGlzc3VlIHRoZXJl LiBXaXRoIEJCUiwgUVVJQyBwYWNrZXRzIGFyZSBzdXBwb3NlZGx5IHBhY2VkIGF0IGp1c3QgdW5k ZXIgdGhlIGJvdHRsZW5lY2sgcmF0ZSwgZXhjZXB0IGR1cmluZyAmcXVvdDtwcm9iZSZxdW90OyBw ZXJpb2RzIGluIHdoaWNoIHRoZXkgcHJvYmUgZm9yIDEgUlRUIGF0IGEgc2xpZ2h0bHkgaGlnaGVy IHJhdGUuIFRoZSBMNFMgQUdNIG1pZ2h0IGRlZ2VuZXJhdGUgaW4gYSBmb3JtIG9mIE9OLU9GRiBj b250cm9sIC0tIG5vIGZlZWRiYWNrIGF0IGFsbCBtb3N0IG9mIHRoZSB0aW1lLCB0aGVuIGEgYnVu Y2ggb2YgQ0UgbWFya3MgaWYgdGhlIHByb2JlIHJhdGUgZXhjZWVkcyB0aGUgYm90dGxlbmVjayBi YW5kd2lkdGguIEFzIGFueW9uZSBleHBlcmltZW50ZWQgd2l0aCB0aGF0PzxvOnA+PC9vOnA+PC9z cGFuPjwvcHJlPg0KPC9ibG9ja3F1b3RlPg0KPHByZT5JbiBteSB2aWV3LCB0aGUgYmlnZ2VzdCBx dWVzdGlvbiB5b3Ugc2hvdWxkIGJlIGFza2luZyBpcyBob3cgUVVJQyB3aWxsIGRpc3Rpbmd1aXNo IGJldHdlZW4gQ0UgbWFya3MgYXBwbGllZCBieSBhbiBMNFMgQVFNIG9uIHRoZSBwYXRoLCBhbmQg dGhvc2UgYXBwbGllZCBieSBhbiBSRkMtMzE2OCBBUU0gb24gdGhlIHBhdGguJm5ic3A7IFRoZSBs YXR0ZXIgd2lsbCB0cmVhdCBFQ1QoMSkgbWFya2VkIHBhY2tldHMgdGhlIHNhbWUgYXMgRUNUKDAp LCBhbmQgZXhwZWN0cyB0aGUgc2FtZSBtdWx0aXBsaWNhdGl2ZS1kZWNyZWFzZSBjb25nZXN0aW9u IHJlc3BvbnNlLCBidXQgTDRTIGV4cGVjdHMgYSBxdWFsaXRhdGl2ZWx5IGRpZmZlcmVudCBsaW5l YXIgcmVzcG9uc2UuPG86cD48L286cD48L3ByZT4NCjwvYmxvY2txdW90ZT4NCjxwPlllcywgdGhh dCdzIGEgc2lnbmlmaWNhbnQgaXNzdWUuIEFzIGl0IGlzLCBub2RlcyBjYW4gb25seSB3b3JrIHdp dGggTDRTIG9uIHNwZWNpZmljIHBhdGhzIHRoYXQgYXJlIHNvbWVob3cgZ3VhcmFudGVlZCB0byBu b3QgbWl4IHRoZSAmcXVvdDtjbGFzc2ljJnF1b3Q7IGFuZCAmcXVvdDtMNFMmcXVvdDsgYmVoYXZp b3IuIEZvciBub3csIHRoZSBvbmx5IHdheSBJIHdvdWxkIGltcGxlbWVudCBpdCBpcyBieSBhZGRp bmcgYW4gJnF1b3Q7bDRzJnF1b3Q7IG9wdGlvbiwgdGhhdCB3b3VsZCBvbmx5IGJlDQogdHVybmVk IG9uIGJ5IGRlZmF1bHQgd2hlbiBleHBlcmltZW50aW5nIHdpdGggTDRTIGJlaGF2aW9yLiBJZiB3 ZSB3YW50IGFueSBsYXJnZSBzY2FsZSBkZXBsb3ltZW50LCB3ZSBoYXZlIHRvIGdldCBhbiBJRVRG IGNvbnNlbnN1cyBvbiB0aGUgZXZvbHV0aW9uIG9mIEVDTiAtLSBzb21ldGhpbmcgcHJlc2NyaXB0 aXZlLCBub3QganVzdCB0aGUgJnF1b3Q7bWF5IGV4cGVyaW1lbnQmcXVvdDsgb2YgUkZDIDgzMTEu IEN1cnJlbnRseSwgaW1wbGVtZW50YXRpb25zIGNhbg0KIG9ubHkgdHJlYXQgQ0UgbWFya3MgYXMg YW4gaW5kaWNhdGlvbiB0aGF0IHRoZXkgYXJlIHByb2JhYmx5IHNlbmRpbmcgZmFzdGVyIHRoYW4g dGhlIG5ldHdvcmsgYWxsb3dzLjxvOnA+PC9vOnA+PC9wPg0KPGJsb2NrcXVvdGUgc3R5bGU9Im1h cmdpbi10b3A6NS4wcHQ7bWFyZ2luLWJvdHRvbTo1LjBwdCI+DQo8cHJlPjxvOnA+Jm5ic3A7PC9v OnA+PC9wcmU+DQo8cHJlPllvdSBzaG91bGQgcHJvYmFibHkgbm90IGRvIGFueSBzdWJzdGFudGlh bCB3b3JrIG9uIGludGVncmF0aW5nIEw0UyBpbnRvIFFVSUMgdW50aWwgeW91IGhhdmUgYSBnb29k IGFuc3dlciB0byB0aGF0IHF1ZXN0aW9uLiZuYnNwOyBBbHRlcm5hdGl2ZSBhcHByb2FjaGVzIHRv IGhpZ2gtZmlkZWxpdHkgY29uZ2VzdGlvbiBjb250cm9sIGV4aXN0IHdoaWNoIHJlc29sdmUgdGhh dCBwYXJ0aWN1bGFyIGNvbnVuZHJ1bSBlYXNpbHkuJm5ic3A7IEluIHBhcnRpY3VsYXIsIHRoZSBF Q04gZmllbGQgZmVlZGJhY2sgbWVjaGFuaXNtIGluIFFVSUMgY2FuIGFsc28gc3VwcG9ydCBTQ0Uu PG86cD48L286cD48L3ByZT4NCjwvYmxvY2txdW90ZT4NCjxwPlllcywgUVVJQyBpbXBsZW1lbnRh dGlvbnMgY291bGQgaW5kZWVkIHN1cHBvcnQgU0NFLiBCdXQgdGhleSBjb3VsZCBub3Qgc3VwcG9y dCBib3RoIFNDRSBhbmQgTDRTLCBhbmQgdGhhdCdzIGEgYmlnIGJ1bW1lci48bzpwPjwvbzpwPjwv cD4NCjxwPkkgd29uZGVyIHdoZXRoZXIgdGhlcmUgaXMgYSB3YXkgdG8gdW5pZnkgdGhlIEw0UyBh bmQgU0NFIGJlaGF2aW9ycy4gVHJhbnNwb3J0IHByb3RvY29sIGRldmVsb3BlcnMgYXJlIHVubGlr ZWx5IHRvIHdpZGVseSBkZXZlbG9wIGVpdGhlciBvcHRpb24gdW5sZXNzIHRoZSBJRVRGIGNvbWVz IHRvIGEgY29uc2Vuc3VzLiBUaGlzIGlzIGEgc2hhbWUsIGJlY2F1c2UgRUNOIGJhc2VkIGNvbmdl c3Rpb24gY29udHJvbCBkb2VzIG5vdCBoYXZlIHRoZSBhbWJpZ3VpdHkNCiBvZiBsb3NzLWJhc2Vk IENDOiBFQ04gbWFya3MgYXJlIG5vdCBhbWJpZ3VvdXMsIHRoZXkgYXJlIGNsZWFybHkgc2lnbmFs cyBmcm9tIHRoZSBuZXR3b3JrLCB0aGUgYXBwbGljYXRpb24gZG9lcyBub3QgbmVlZCB0byBidWls ZCBsb2dpYyB0byBkaWZmZXJlbnRpYXRlIGJldHdlZW4gY29uZ2VzdGlvbiBpbmR1Y2VkIGxvc3Nl cyBhbmQgcmFuZG9tIGxvc3NlcyBkdWUgdG8sIGZvciBleGFtcGxlLCByYWRpbyBldmVudC4gRmFp bGluZyB0aGF0IEVDTiB1bmlmaWNhdGlvbg0KIENDIGFsZ29yaXRobXMgZW5kIHVwIHJlbHlpbmcg YSBsb3Qgb24gYmFuZHdpZHRoIGFuZCBkZWxheSBtZWFzdXJlbWVudHMsIGJ1dCB0aGF0IGlzIHN1 Yi1vcHRpbWFsIGJlY2F1c2Ugc29tZSByYWRpbyBsaW5rcyBkbyBoYXZlIHZhcmlhYmxlIGJhbmR3 aWR0aCBhbmQgdmFyaWFibGUgZGVsYXlzLjxvOnA+PC9vOnA+PC9wPg0KPHA+QnkgdGhlIHdheSwg SSBkbyB0aGluayB0aGF0IEw0UyBtYWtlcyB0b28gbWFueSBoeXBvdGhlc2VzIGFib3V0IHRoZSBD Qy4gSSB3aXNoIHRoZSBBUU0gd2FzIGJhc2VkIG9uIGludHJpbnNpYyBwcm9wZXJ0aWVzLCBzdWNo IGFzICZxdW90O3RoaXMgZmxvdyBpcyB1c2luZyBtb3JlIHRoYW4gaXRzIGZhaXIgc2hhcmUgb2Yg YmFuZHdpZHRoJnF1b3Q7IG9yICZxdW90O3RoaXMgbGluayBpcyBjb25nZXN0ZWQgYW5kIHRoZSB0 cmFuc3BvcnQgc2hvdWxkIGJhY2sgb2ZmJnF1b3Q7IHJhdGhlcg0KIHRoYW4gcmVhc29uaW5nIGFz IGlmIGltcGxlbWVudGF0aW9ucyB3ZXJlIHN0aWxsIHVzaW5nIFRDUC1SRU5PLCB3aGljaCBuZWl0 aGVyIExpbnV4IG9yIFdpbmRvd3Mgb3IgQ2hyb21lIGRvZXMuPG86cD48L286cD48L3A+DQo8cD4t LSBDaHJpc3RpYW4gSHVpdGVtYTxvOnA+PC9vOnA+PC9wPg0KPHA+PG86cD4mbmJzcDs8L286cD48 L3A+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg== --_000_AM0PR07MB6114083DB6F3FBA98AA2281FB9130AM0PR07MB6114eurp_-- From nobody Sun Nov 1 15:42:10 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C7863A0AB8; Sun, 1 Nov 2020 15:42:03 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.346 X-Spam-Level: X-Spam-Status: No, score=-2.346 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.247, SPF_HELO_FAIL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jiRcIC_EI8Mu; Sun, 1 Nov 2020 15:42:02 -0800 (PST) Received: from cl3.bcs-hosting.net (cl3.bcs-hosting.net [3.11.37.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D2EB13A0AB9; Sun, 1 Nov 2020 15:42:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Transfer-Encoding:Content-Type: In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=Q7uBGqy8uMNT8mxVHHK9Q8PnBpCzuTdLb8YMKDMmTrQ=; b=nN27gGkTkVTGk315DoSGdc35Rs 4x+4uQIDAtUCqFfO+4xL9FqWuQ1ALrLiwVNJXe94CFaX2uWMiDCv6nsm7/rrxhhyZX3zKlY7ONNHA Plj6BQKGgpW2qFG8O7jTkS0eRyLOn9t8kK3fFU4+sDaKcKbXlZQ3V8bJj/WchNzZeUY/ckhwDcATn R0BpfFoRAL64jXAZp+fdzvzPRzDNTdxWjScLVRS8AWyEDlm9nqecy3krJ6ho3fU6GiL1rqjS6V2DR Z+/K7hQB80a1wAf4/BBcUt2htaN9jDzSjAMOLK0L35w+D0gVaDZ0l9Y6hFAEet0/scDX0HnFEQSNz FaYARzPA==; Received: from 67.153.238.178.in-addr.arpa ([178.238.153.67]:36340 helo=[192.168.1.3]) by cl3.bcs-hosting.net with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1kZMz1-00EvzK-NI; Sun, 01 Nov 2020 23:41:59 +0000 To: Christian Huitema , Quentin De Coninck , "Tilmans, Olivier (Nokia - BE/Antwerp)" Cc: tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> <12c3cebf-e6e1-7c11-38cb-44c49c4e1d6e@huitema.net> From: Bob Briscoe Message-ID: Date: Sun, 1 Nov 2020 23:41:59 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <12c3cebf-e6e1-7c11-38cb-44c49c4e1d6e@huitema.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cl3.bcs-hosting.net X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: cl3.bcs-hosting.net: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: cl3.bcs-hosting.net: in@bobbriscoe.net X-Source: X-Source-Args: X-Source-Dir: Archived-At: Subject: Re: [tcpPrague] L4S and QUIC X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 23:42:03 -0000 Christian, [Changed subject line again] On 01/11/2020 20:02, Christian Huitema wrote: > On 11/1/2020 5:30 AM, Bob Briscoe wrote: > >> Quentin De Coninck wrote an initial implementation of a "QUIC Prague" >> congestion control based on your picoQUIC code. It can be accessed via >> the L4S landing page here: >>     https://riteproject.eu/dctth/#code >> He started it at an IETF hackathon. I don't think it's been maintained >> since it was first written, but it might be a start. > Care doing a PR so this could be added to the Picoquic distribution? [BB] Would need to ask Quentin. This was some time ago; I have no idea what he's doing these days. I meant to add him. I have now (don't know whether the @ will still reach him). I've also included Olivier, who might be in touch with him. Bob > > -- Christian Huitema > > -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ From nobody Sun Nov 1 15:56:12 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1CFEF3A0ADD for ; Sun, 1 Nov 2020 15:56:05 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.135 X-Spam-Level: X-Spam-Status: No, score=-2.135 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.247, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AxNbgznyXFlr for ; Sun, 1 Nov 2020 15:56:02 -0800 (PST) Received: from mx43-out1.antispamcloud.com (mx43-out1.antispamcloud.com [138.201.61.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0459E3A0AD6 for ; Sun, 1 Nov 2020 15:56:01 -0800 (PST) Received: from xse48.mail2web.com ([66.113.196.48] helo=xse.mail2web.com) by mx36.antispamcloud.com with esmtp (Exim 4.92) (envelope-from ) id 1kZNCU-0001oF-5Z for tcpPrague@ietf.org; Mon, 02 Nov 2020 00:55:59 +0100 Received: from xsmtp22.mail2web.com (unknown [10.100.68.61]) by xse.mail2web.com (Postfix) with ESMTPS id 4CPXxL4nHZzLjZ for ; Sun, 1 Nov 2020 15:55:50 -0800 (PST) Received: from [10.5.2.15] (helo=xmail05.myhosting.com) by xsmtp22.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1kZNCQ-0004sL-IJ for tcpPrague@ietf.org; Sun, 01 Nov 2020 15:55:50 -0800 Received: (qmail 14517 invoked from network); 1 Nov 2020 23:55:50 -0000 Received: from unknown (HELO [192.168.1.107]) (Authenticated-user:_huitema@huitema.net@[172.58.43.139]) (envelope-sender ) by xmail05.myhosting.com (qmail-ldap-1.03) with ESMTPA for ; 1 Nov 2020 23:55:50 -0000 To: "De Schepper, Koen (Nokia - BE/Antwerp)" , Jonathan Morton Cc: iccrg IRTF list , Bob Briscoe , tsvwg IETF list , TCP Prague List References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> From: Christian Huitema Autocrypt: addr=huitema@huitema.net; prefer-encrypt=mutual; keydata= mDMEXtavGxYJKwYBBAHaRw8BAQdA1ou9A5MHTP9N3jfsWzlDZ+jPnQkusmc7sfLmWVz1Rmu0 J0NocmlzdGlhbiBIdWl0ZW1hIDxodWl0ZW1hQGh1aXRlbWEubmV0PoiWBBMWCAA+FiEEw3G4 Nwi4QEpAAXUUELAmqKBYtJQFAl7WrxsCGwMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgEC F4AACgkQELAmqKBYtJQbMwD/ebj/qnSbthC/5kD5DxZ/Ip0CGJw5QBz/+fJp3R8iAlsBAMjK r2tmyWyJz0CUkVG24WaR5EAJDvgwDv8h22U6QVkAuDgEXtavGxIKKwYBBAGXVQEFAQEHQJoM 6MUAIqpoqdCIiACiEynZf7nlJg2Eu0pXIhbUGONdAwEIB4h+BBgWCAAmFiEEw3G4Nwi4QEpA AXUUELAmqKBYtJQFAl7WrxsCGwwFCQlmAYAACgkQELAmqKBYtJRm2wD7BzeK5gEXSmBcBf0j BYdSaJcXNzx4yPLbP4GnUMAyl2cBAJzcsR4RkwO4dCRqM9CHpVJCwHtbUDJaa55//E0kp+gH Message-ID: <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> Date: Sun, 1 Nov 2020 15:55:50 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------BE98C98514FF49A3DB80CDC5" Content-Language: en-US X-Originating-IP: 66.113.196.48 X-Spampanel-Domain: xsmtpout.mail2web.com X-Spampanel-Username: 66.113.196.48/32 Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.196.48/32@xsmtpout.mail2web.com X-Spampanel-Outgoing-Class: unsure X-Spampanel-Outgoing-Evidence: Combined (0.15) X-Recommended-Action: accept X-Filter-ID: Mvzo4OR0dZXEDF/gcnlw0Z1apovzGPsYhEeBL1aoZmqpSDasLI4SayDByyq9LIhVUZbR67CQ7/vm /hHDJU4RXkTNWdUk1Ol2OGx3IfrIJKywOmJyM1qr8uRnWBrbSAGD42+pgDSJdnVPeXOEK1m6uDbl kDZNUkuHiojIVh7uAUvRxM+xBZ1w6pGno+CK6C/mUZq99/v+OWrp+SY7iSuea8EuNfnjQsRqWXYP fdewBkdPR9o014ICOs9pIWX8OtLJqUALHGqxBO534sbPOpB/O8xKTagmQ3xMyVZ07UERWzx6BTXs FxcSzhpFHKuVDd8Suy8lNDS0QWWkADhl7glCR9PbrhSmW22tW1yBxgRT8bmxZJSIFVPkVVALPRKr lHlM3kWCH4Q79vaQ+COHDJAgLHQOD0r6/AaHZiEtdTMtMlgTBUa5LSawQcdT80HH17nNg8oiq9mz mwrbQbTulSg7juWBOXp8nHKe0R+FkIqN7hkFZqA6TBkpoO/ktnXt0JlLIRFsicyJMEhQFtD8PLoi nuxTyssp4L0plUGigax8zy4LpVxP5YFZg5fgueXLf6LKHDJ71JSXKkUqfqsTqwEEUOidX4Ts4xdG +C13IyWeZaLC2lfTiRPZMteoXs+GRoRWGBfTiFUgMuua665ZNZolbYOTnT6wN9gOMciuJ4QlThoh 1KxQEOw1WgWLtkQSpIiO5CAOwzF4KlWKogaY4h5dvpXked+P+aSZU/EB7YnRWs2LBDMrD7q/cJog wbqzsuokd3KVbf1SW+BycALUr5UaiYaPmF/7MAKyW1Kb4FKGpjRUxjUEXi0B+fEhaFh69PSnZtIW M4JzTFxRH3Dz7iuUi5v6swd5IHP4xIKORT1wivuloQCszo/2qS3jMvYFyyWx9R/2gMGq0KWAzmMf +ibVDpdplkxcBm4XM6d7s4Bx3w1WbaUe4g0kgaInvdEp64qlVpe//bVkg87Xe61e30HXuSERbInM iTBIUBbQ/Dy6Ip4D1rnEhdYtY/lMQX5s39oH5ijcGdSK77ViXbmzTYWgl82XucjoLWQ7++7jcUS/ T5w= X-Report-Abuse-To: spam@quarantine11.antispamcloud.com Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2020 23:56:06 -0000 This is a multi-part message in MIME format. --------------BE98C98514FF49A3DB80CDC5 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/1/2020 1:53 PM, De Schepper, Koen (Nokia - BE/Antwerp) wrote: > > >>By the way, I do think that L4S makes too many hypotheses about the > CC. I wish the AQM was based on intrinsic properties, such as "this > flow is using more than its fair share of bandwidth" > > This last quote is exactly what the L4S marking is intended to do: the > smoothed marking probability over a longer time interval (about 10 > RTTs) indicates the fair target rate. Assuming we agree on 15ms being > the reference RTT, then the average rate can be r=3D2/(p*0.015)=3D133/p= > packets per second. So you can evaluate yourself if you use more or > less than the fair share. How you respond to this knowledge is up to > your CC. > I think that part of the complexity in L4S AQM comes from trying to achieve fair sharing of resource without actually implementing fair queuing. You end up expecting that implementations apply a formula and derive their packet sending rate from a smoothed average of the packet marking rate. It may work well in controlled environments, but I am afraid that very few implementations are ever going to do that in practic= e. Congestion Control algorithms are much more likely to treat packet losses, markings, or delay increases as signals that they are sending too fast, while considering their absence or lower frequency as signal that they might be able to send faster. Implementations also are likely to take into account the recent history, for example monitoring the min RTT (LEDBAT, BBR), or monitoring the recent available bandwidth (BBR). They also typically filter signals, e.g., consider only one packet loss or ECN mark per RTT (New Reno, Cubic, etc.). This is expected, because we do see packet losses arriving in batches, and can reasonably expect ECN marks to have the same behavior. But implementations are inherently selfish, do not really trust the signals from the network, and are very unlikely to end up applying the formula that you suggest. In any case, you have a scaling issue. Let's consider a 1.5Gbps link, with 15 ms delay and 1500 bytes packets. The nominal sending rate is 125,000 packets per second. The marking rate with your formula shall be p =3D 2/(r*0.015), which is about 0.0008%. Over the last 10 RTT, the connection will on average see 0.14 marks -- that is, no mark over the last 10 RTT 86% of the time. This falls well short of the requirement to provide frequent feedback! If you give up the formula, you could end up with a significantly simpler event based congestion control, in which applications slow down if they see too many CE marks, accelerate if they only see a very low marking rate, and stay put if they see an infrequent marking rate. I can see applications doing that, resulting in a fairly stable network and low delays. Of course, you will have to do some form of fair share enforcement to catch violators, but I am sure you can come up with something. -- Christian Huitema --------------BE98C98514FF49A3DB80CDC5 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit


On 11/1/2020 1:53 PM, De Schepper, Koen (Nokia - BE/Antwerp) wrote:

>>By the way, I do think that L4S makes too many hypotheses about the CC. I wish the AQM was based on intrinsic properties, such as "this flow is using more than its fair share of bandwidth"

This last quote is exactly what the L4S marking is intended to do: the smoothed marking probability over a longer time interval (about 10 RTTs) indicates the fair target rate. Assuming we agree on 15ms being the reference RTT, then the average rate can be r=2/(p*0.015)=133/p packets per second. So you can evaluate yourself if you use more or less than the fair share. How you respond to this knowledge is up to your CC.

I think that part of the complexity in L4S AQM comes from trying to achieve fair sharing of resource without actually implementing fair queuing. You end up expecting that implementations apply a formula and derive their packet sending rate from a smoothed average of the packet marking rate. It may work well in controlled environments, but I am afraid that very few implementations are ever going to do that in practice.

Congestion Control algorithms are much more likely to treat packet losses, markings, or delay increases as signals that they are sending too fast, while considering their absence or lower frequency as signal that they might be able to send faster. Implementations also are likely to take into account the recent history, for example monitoring the min RTT (LEDBAT, BBR), or monitoring the recent available bandwidth (BBR). They also typically filter signals, e.g., consider only one packet loss or ECN mark per RTT (New Reno, Cubic, etc.). This is expected, because we do see packet losses arriving in batches, and can reasonably expect ECN marks to have the same behavior. But implementations are inherently selfish, do not really trust the signals from the network, and are very unlikely to end up applying the formula that you suggest.

In any case, you have a scaling issue. Let's consider a 1.5Gbps link, with 15 ms delay and 1500 bytes packets. The nominal sending rate is 125,000 packets per second. The marking rate with your formula shall be p = 2/(r*0.015), which is about 0.0008%. Over the last 10 RTT, the connection will on average see 0.14 marks -- that is, no mark over the last 10 RTT 86% of the time. This falls well short of the requirement to provide frequent feedback!

If you give up the formula, you could end up with a significantly simpler event based congestion control, in which applications slow down if they see too many CE marks, accelerate if they only see a very low marking rate, and stay put if they see an infrequent marking rate. I can see applications doing that, resulting in a fairly stable network and low delays. Of course, you will have to do some form of fair share enforcement to catch violators, but I am sure you can come up with something.

-- Christian Huitema



--------------BE98C98514FF49A3DB80CDC5-- From nobody Sun Nov 1 16:11:05 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DADDB3A0ADD for ; Sun, 1 Nov 2020 16:11:00 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.134 X-Spam-Level: X-Spam-Status: No, score=-2.134 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.247, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AJFhKzjrs8kj for ; Sun, 1 Nov 2020 16:10:59 -0800 (PST) Received: from mx43-out1.antispamcloud.com (mx43-out1.antispamcloud.com [138.201.61.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ADE123A0ADE for ; Sun, 1 Nov 2020 16:10:58 -0800 (PST) Received: from xse427.mail2web.com ([66.113.197.173] helo=xse.mail2web.com) by mx165.antispamcloud.com with esmtp (Exim 4.92) (envelope-from ) id 1kZNQy-0016Pp-8r for tcpPrague@ietf.org; Mon, 02 Nov 2020 01:10:55 +0100 Received: from xsmtp21.mail2web.com (unknown [10.100.68.60]) by xse.mail2web.com (Postfix) with ESMTPS id 4CPYGf09q4zPLs for ; Sun, 1 Nov 2020 16:10:50 -0800 (PST) Received: from [10.5.2.16] (helo=xmail06.myhosting.com) by xsmtp21.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1kZNQv-0000ff-T1 for tcpPrague@ietf.org; Sun, 01 Nov 2020 16:10:49 -0800 Received: (qmail 16962 invoked from network); 2 Nov 2020 00:10:49 -0000 Received: from unknown (HELO [192.168.1.107]) (Authenticated-user:_huitema@huitema.net@[172.58.43.139]) (envelope-sender ) by xmail06.myhosting.com (qmail-ldap-1.03) with ESMTPA for ; 2 Nov 2020 00:10:49 -0000 From: Christian Huitema To: "De Schepper, Koen (Nokia - BE/Antwerp)" , Jonathan Morton Cc: iccrg IRTF list , Bob Briscoe , tsvwg IETF list , TCP Prague List References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> Autocrypt: addr=huitema@huitema.net; prefer-encrypt=mutual; keydata= mDMEXtavGxYJKwYBBAHaRw8BAQdA1ou9A5MHTP9N3jfsWzlDZ+jPnQkusmc7sfLmWVz1Rmu0 J0NocmlzdGlhbiBIdWl0ZW1hIDxodWl0ZW1hQGh1aXRlbWEubmV0PoiWBBMWCAA+FiEEw3G4 Nwi4QEpAAXUUELAmqKBYtJQFAl7WrxsCGwMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgEC F4AACgkQELAmqKBYtJQbMwD/ebj/qnSbthC/5kD5DxZ/Ip0CGJw5QBz/+fJp3R8iAlsBAMjK r2tmyWyJz0CUkVG24WaR5EAJDvgwDv8h22U6QVkAuDgEXtavGxIKKwYBBAGXVQEFAQEHQJoM 6MUAIqpoqdCIiACiEynZf7nlJg2Eu0pXIhbUGONdAwEIB4h+BBgWCAAmFiEEw3G4Nwi4QEpA AXUUELAmqKBYtJQFAl7WrxsCGwwFCQlmAYAACgkQELAmqKBYtJRm2wD7BzeK5gEXSmBcBf0j BYdSaJcXNzx4yPLbP4GnUMAyl2cBAJzcsR4RkwO4dCRqM9CHpVJCwHtbUDJaa55//E0kp+gH Message-ID: Date: Sun, 1 Nov 2020 16:10:49 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> Content-Type: multipart/alternative; boundary="------------72566BE0D78938900CD90A3C" Content-Language: en-US X-Originating-IP: 66.113.197.173 X-Spampanel-Domain: xsmtpout.mail2web.com X-Spampanel-Username: 66.113.197.0/24 Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.197.0/24@xsmtpout.mail2web.com X-Spampanel-Outgoing-Class: unsure X-Spampanel-Outgoing-Evidence: Combined (0.15) X-Recommended-Action: accept X-Filter-ID: Mvzo4OR0dZXEDF/gcnlw0XvADx2zSFwG+3csxFBPBHmpSDasLI4SayDByyq9LIhVUZbR67CQ7/vm /hHDJU4RXkTNWdUk1Ol2OGx3IfrIJKywOmJyM1qr8uRnWBrbSAGD42+pgDSJdnVPeXOEK1m6uDbl kDZNUkuHiojIVh7uAUtUS5yfxPR0LUnCfc2etzVZUZq99/v+OWrp+SY7iSuea8EuNfnjQsRqWXYP fdewBkdPR9o014ICOs9pIWX8OtLJfIWPzE7MiabwbSDoRIQJQtbNhTNyx2YOvjKTJ3Ps/x2i3GSv wir0OshyEkOwfCYvxs5/T0oXYyKdvWabEYxQFC5Ano+acIK7aoZBBKN3kq8lTiPCf+PwTb/RKAdw /TswIg1fYmNb1kgzViAoNrSrXN1jhnM/Mbva2XLV/LIEzaJm4kYWBU//ZZ8pZ1xZfmaG1NRsUdzW awx6dX0NJ8Bzt99fxN2oReTDHAyOynaY0ClENaQq/2aUAxcG3yLqjApZmdySlZou9qHIGOZDEEo7 O58ZQzrOqjAERHu4pt/Ia6wELzcGxDgkPe7eR6qspNNQGjLhGMBSrFdf8dBbPvtqJwEiRQv+PVjj wa+Z5RFCOMR3ix2Ng3ctwEt0ycHrTvD9oBpdWj8QLYr5sN4Ugz0teygCLYuflBwW1CjUUQdx8quh nJWj57cNhf7KkzliAIHWGlU4PSngKJoJ6shQV+KVUWkvZ+pWP1s35neRYWMQUWZErSs0X3oyoTc8 j/o7qulxDJ14J/ZpV+AwLTzPkgkfF2re/hsBBxzR0ZxLcHZ9dOizQO9KaEkxrOQpyJAFamkd8jAz /walAdU1F5uBDCIEXwEH5tktsnhMr4gG+2qXrJ2i00qa8JieZgni7Zekc3P+9R/2gMGq0KWAzmMf +ibVDpdplkxcBm4XM6d7s4Bx3w1WbaUe4g0kgaInvdEp64qlVpe//bVkg87Xe61e30HXuSERbInM iTBIUBbQ/Dy6Ip4D1rnEhdYtY/lMQX5s39oH5ijcGdSK77ViXbmzTYWgl82XucjoLWQ7++7jcUS/ T5w= X-Report-Abuse-To: spam@quarantine11.antispamcloud.com Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 00:11:01 -0000 This is a multi-part message in MIME format. --------------72566BE0D78938900CD90A3C Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 11/1/2020 3:55 PM, Christian Huitema wrote: > > > On 11/1/2020 1:53 PM, De Schepper, Koen (Nokia - BE/Antwerp) wrote: >> >> >>By the way, I do think that L4S makes too many hypotheses about the >> CC. I wish the AQM was based on intrinsic properties, such as "this >> flow is using more than its fair share of bandwidth" >> >> This last quote is exactly what the L4S marking is intended to do: >> the smoothed marking probability over a longer time interval (about >> 10 RTTs) indicates the fair target rate. Assuming we agree on 15ms >> being the reference RTT, then the average rate can be >> r=2/(p*0.015)=133/p packets per second. So you can evaluate yourself >> if you use more or less than the fair share. How you respond to this >> knowledge is up to your CC. >> > I think that part of the complexity in L4S AQM comes from trying to > achieve fair sharing of resource without actually implementing fair > queuing. You end up expecting that implementations apply a formula and > derive their packet sending rate from a smoothed average of the packet > marking rate. It may work well in controlled environments, but I am > afraid that very few implementations are ever going to do that in > practice. > > Congestion Control algorithms are much more likely to treat packet > losses, markings, or delay increases as signals that they are sending > too fast, while considering their absence or lower frequency as signal > that they might be able to send faster. Implementations also are > likely to take into account the recent history, for example monitoring > the min RTT (LEDBAT, BBR), or monitoring the recent available > bandwidth (BBR). They also typically filter signals, e.g., consider > only one packet loss or ECN mark per RTT (New Reno, Cubic, etc.). This > is expected, because we do see packet losses arriving in batches, and > can reasonably expect ECN marks to have the same behavior. But > implementations are inherently selfish, do not really trust the > signals from the network, and are very unlikely to end up applying the > formula that you suggest. > > In any case, you have a scaling issue. Let's consider a 1.5Gbps link, > with 15 ms delay and 1500 bytes packets. The nominal sending rate is > 125,000 packets per second. The marking rate with your formula shall > be p = 2/(r*0.015), which is about 0.0008%. Over the last 10 RTT, the > connection will on average see 0.14 marks -- that is, no mark over the > last 10 RTT 86% of the time. This falls well short of the requirement > to provide frequent feedback! > Sorry, bug in my spreadsheet. The marking rate is actually 0.1%, about 2 packets per RTT. Still not frequent enough for my taste, but much better than 0.0008%. Nevertheless, the inverse relation between marking rate and data rate is not great. > If you give up the formula, you could end up with a significantly > simpler event based congestion control, in which applications slow > down if they see too many CE marks, accelerate if they only see a very > low marking rate, and stay put if they see an infrequent marking rate. > I can see applications doing that, resulting in a fairly stable > network and low delays. Of course, you will have to do some form of > fair share enforcement to catch violators, but I am sure you can come > up with something. > > -- Christian Huitema > > > > > _______________________________________________ > tcpPrague mailing list > tcpPrague@ietf.org > https://www.ietf.org/mailman/listinfo/tcpprague --------------72566BE0D78938900CD90A3C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit


On 11/1/2020 3:55 PM, Christian Huitema wrote:


On 11/1/2020 1:53 PM, De Schepper, Koen (Nokia - BE/Antwerp) wrote:

>>By the way, I do think that L4S makes too many hypotheses about the CC. I wish the AQM was based on intrinsic properties, such as "this flow is using more than its fair share of bandwidth"

This last quote is exactly what the L4S marking is intended to do: the smoothed marking probability over a longer time interval (about 10 RTTs) indicates the fair target rate. Assuming we agree on 15ms being the reference RTT, then the average rate can be r=2/(p*0.015)=133/p packets per second. So you can evaluate yourself if you use more or less than the fair share. How you respond to this knowledge is up to your CC.

I think that part of the complexity in L4S AQM comes from trying to achieve fair sharing of resource without actually implementing fair queuing. You end up expecting that implementations apply a formula and derive their packet sending rate from a smoothed average of the packet marking rate. It may work well in controlled environments, but I am afraid that very few implementations are ever going to do that in practice.

Congestion Control algorithms are much more likely to treat packet losses, markings, or delay increases as signals that they are sending too fast, while considering their absence or lower frequency as signal that they might be able to send faster. Implementations also are likely to take into account the recent history, for example monitoring the min RTT (LEDBAT, BBR), or monitoring the recent available bandwidth (BBR). They also typically filter signals, e.g., consider only one packet loss or ECN mark per RTT (New Reno, Cubic, etc.). This is expected, because we do see packet losses arriving in batches, and can reasonably expect ECN marks to have the same behavior. But implementations are inherently selfish, do not really trust the signals from the network, and are very unlikely to end up applying the formula that you suggest.

In any case, you have a scaling issue. Let's consider a 1.5Gbps link, with 15 ms delay and 1500 bytes packets. The nominal sending rate is 125,000 packets per second. The marking rate with your formula shall be p = 2/(r*0.015), which is about 0.0008%. Over the last 10 RTT, the connection will on average see 0.14 marks -- that is, no mark over the last 10 RTT 86% of the time. This falls well short of the requirement to provide frequent feedback!

Sorry, bug in my spreadsheet. The marking rate is actually 0.1%, about 2 packets per RTT. Still not frequent enough for my taste, but much better than 0.0008%. Nevertheless, the inverse relation between marking rate and data rate is not great.

If you give up the formula, you could end up with a significantly simpler event based congestion control, in which applications slow down if they see too many CE marks, accelerate if they only see a very low marking rate, and stay put if they see an infrequent marking rate. I can see applications doing that, resulting in a fairly stable network and low delays. Of course, you will have to do some form of fair share enforcement to catch violators, but I am sure you can come up with something.

-- Christian Huitema




_______________________________________________
tcpPrague mailing list
tcpPrague@ietf.org
https://www.ietf.org/mailman/listinfo/tcpprague
--------------72566BE0D78938900CD90A3C-- From nobody Sun Nov 1 16:52:20 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 572663A0B84; Sun, 1 Nov 2020 16:52:12 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.848 X-Spam-Level: X-Spam-Status: No, score=-1.848 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cxy4WbWImxha; Sun, 1 Nov 2020 16:52:10 -0800 (PST) Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com [IPv6:2a00:1450:4864:20::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 52AAD3A0BA3; Sun, 1 Nov 2020 16:52:10 -0800 (PST) Received: by mail-lj1-x22d.google.com with SMTP id m8so7105635ljj.0; Sun, 01 Nov 2020 16:52:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=xOgHW4UnCQg5rS6VD+i5ChctdQSlkmtLnKxsUBK78PU=; b=V/9Ugq6hOE4tGBgCaY4/cPtomgAvyVP5po5PtO0rWXt6XoA8nK0dxdrVgUkJF8ly+M xZ82fz+MC1fUIdaDauJc1Z1ON2FSIU6ZWJ+in3LoSC3nkcM/x0LeN85aDsJPV03neta7 T7KcLGwNYkb8x+M81v3HjLo6oPZ7I+Wly/YGqOLWR6GGwMt/tzf2ZSnOSrRI6uFplttl lS5Iy54azFsM6M2xRS2oFhDfH1UvTs3O82irpijcjW7SUGjj/QlBG0zs3bV2QPpU+myS I2+u0jgIKXEDdoklRPiAu02CzrcDuadwZx+lXLznwNnbi8ywdoV//6tFmVTHTr9AtW8D Fxbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=xOgHW4UnCQg5rS6VD+i5ChctdQSlkmtLnKxsUBK78PU=; b=IHJ/UonPp1q8cT7ifi0y+IxycveX9iSQWd/xMVnfPLgq7K6RMlpMzRSlMB3oDwds+7 V3wshmxs7g04pMlSiFundSJumrhELWagpevm8VPr5R8KzI0u4bJpptooBnsABfy1ArhR iWja/SlH7ge9rEcPgzuNZi5jQZbC6Rkj/h/QLpd/gIDN0OtWr3aIdYAWoHyub2Gj8izb U76zMHxgHA8kwF+ub6OuBC+HptiuJpq3BHq2PYasWe42QYebMy/ROmJFJZx6yxnBJUjF uOQVnEJndhWR/DaJo+jLR5KVVKtUKiFTiqVTPkHZHPrmEMcRMJnzybOTJWZz2OARMJZK 6WqA== X-Gm-Message-State: AOAM5336EvpVtcBdYKysnUexCNnebSdS8CsVSB6v3oL3B4SSWeTpyDFi ztxG3t0CJNJA3Zz6lTHXdfc= X-Google-Smtp-Source: ABdhPJxhakLehS9eCJG9/D08dOGNZaIN8iFvSfQcrahvCommSeQxzjvvBLAerDQJnrBjvvxkIcpz3w== X-Received: by 2002:a2e:b1c6:: with SMTP id e6mr1996719lja.108.1604278328301; Sun, 01 Nov 2020 16:52:08 -0800 (PST) Received: from jonathartonsmbp.lan (178-55-170-35.bb.dnainternet.fi. [178.55.170.35]) by smtp.gmail.com with ESMTPSA id b25sm1617776lfa.32.2020.11.01.16.52.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 01 Nov 2020 16:52:07 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.7\)) From: Jonathan Morton In-Reply-To: Date: Mon, 2 Nov 2020 02:52:06 +0200 Cc: "De Schepper, Koen (Nokia - BE/Antwerp)" , iccrg IRTF list , Bob Briscoe , tsvwg IETF list , TCP Prague List Content-Transfer-Encoding: quoted-printable Message-Id: <84E29EA3-67DF-4130-8E92-940D5A4B19DA@gmail.com> References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> To: Christian Huitema X-Mailer: Apple Mail (2.3445.9.7) Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 00:52:19 -0000 > On 2 Nov, 2020, at 2:10 am, Christian Huitema = wrote: >=20 >> In any case, you have a scaling issue. Let's consider a 1.5Gbps link, = with 15 ms delay and 1500 bytes packets. The nominal sending rate is = 125,000 packets per second. The marking rate with your formula shall be = p =3D 2/(r*0.015), which is about 0.0008%. Over the last 10 RTT, the = connection will on average see 0.14 marks -- that is, no mark over the = last 10 RTT 86% of the time. This falls well short of the requirement to = provide frequent feedback! >=20 > Sorry, bug in my spreadsheet. The marking rate is actually 0.1%, about = 2 packets per RTT. Still not frequent enough for my taste, but much = better than 0.0008%. Nevertheless, the inverse relation between marking = rate and data rate is not great. A property of both Reno and DCTCP is that a particular CE marking = probability results in a particular average cwnd, over some reasonably = long period. The relationship formula is of course very different for = each one, but that qualitative rule happens to be the same. It is a = property that DualQ's coupled AQM design relies on. CUBIC is a little = different in its high-BDP regime, but acts like Reno otherwise. What this means is that competing flows experiencing the same marking = probability will converge to the same cwnd, and their relative = throughputs will be inversely proportional to their effective RTTs. = This convergence is not necessarily very fast, but given a single queue = and a single AQM, it's about as good as you can expect. When you have an AQM per flow, you can do a bit better by applying a = different marking probability to each flow. This makes convergence = faster, and you can make them converge to the same throughput, not = merely the same cwnd. When you have a separate queue per flow, you can = additionally prevent one "fat" flow from affecting the latency of = sparser flows, and *enforce* the throughput equality on short = timescales, even with flows that are unresponsive to congestion signals. = This is all established practice. And that is the key to achieving RTT independence with the minimum of = added complexity. - Jonathan Morton= From nobody Sun Nov 1 21:58:24 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 997E43A0DBF; Sun, 1 Nov 2020 21:58:18 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.1 X-Spam-Level: X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cablelabs.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LU3CXyETOtAo; Sun, 1 Nov 2020 21:58:16 -0800 (PST) Received: from NAM04-CO1-obe.outbound.protection.outlook.com (mail-eopbgr690119.outbound.protection.outlook.com [40.107.69.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 982393A0EA1; Sun, 1 Nov 2020 21:58:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T6odPAaz7IKg6u78tDOSxFYTs1mto8hGX0M/uHh1cYkKS5BInmyp0U9v+zGcJ+R/KVkVRGlyN9hg3jV7quYdD6aOhHsXBbDAwGKjIdzDL8G2aYVvjUGOEVW8huBaISqDmU4kK5OAwYLrwa5YmzCATnwNBNvQmeK66KIEOoECWT1fiUnaORk+NXJXkk0ghKx4UwFq2+3WugtxHEukmNqWGU9c7OKkmPLRJjfpwWLkZyNblmWg2dVrKdVUY16M2fLErbdxtoSFd3Nb7tDt5LglR+WV54vxYO2gA4FDLmXcXm2DvFgXzsCq8HJNehiSmdkLjZq7TxeaP87Z7bfHFYKLsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ybMrz8xsxQ8n6e+CflRofkVzD/6gzXtyKpyTrdEpJos=; b=SGjEIuRrgFPeN3SrGBhTdXaDIgOQc4e+A8s4Nxinl4uoQ10qA5Xg9l1yR76ZCAiMmtLlXxgLrKvNi7BzYCkdfegcfLbR3P3olEyN1UEzzf+Kzeq/M8himvwAB7Ia2d+pF//jsiqs8VDo0kqhpTxJt+/KmfjBOTwcjXgJMITrbTtuiHGk4Xiyp0BIegZ+61pU/SfhuraS8wFQaB0xr+FZZMVpngFDAziof8FG0MHz/T+IpZlmgNWkBaINcE9DHnxACfJTF9XjVeCEkRVP1tLrgCAQWEh4jSV+qg0LNWNjeoZB/hn/9/kaVHUHZ6K+Z4nM8LOWNp5xPDeeF9jSJ03G6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cablelabs.com; dmarc=pass action=none header.from=cablelabs.com; dkim=pass header.d=cablelabs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cablelabs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ybMrz8xsxQ8n6e+CflRofkVzD/6gzXtyKpyTrdEpJos=; b=EIU8osQwjx03p9jXYtgisVERMZ7APJI01fU32hOoO6aXFuvgJ1Zan/xSBBy16ed44mgGVMYztRkZsWQOuftZYNR3kw9oaPJqkWawwBYD/4Xgl1lEQNMeY38ldiNlgEwqO5gjdZM83sFaw3/Ke7pnhhQS8m70mWV26gqIJEcdccsYQXAhQTWV7K4YEFQnWAjegBXHQ4gAQqp05nLVRQGVX0bK3L0/5huZyPDxqBcOojpJPBIo5z3cxQyZdQMv7M9k8M5EXCwGW1ALoSXgPun/0wtyi1bpc7UOaHjc2e9vx4gixC5j7OwTfe5d6YSRSdOGkn8PQFdjinKLjHURJqI0Og== Received: from CY4PR0601MB3713.namprd06.prod.outlook.com (2603:10b6:910:92::10) by CY4PR06MB2375.namprd06.prod.outlook.com (2603:10b6:903:15::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3499.28; Mon, 2 Nov 2020 05:58:01 +0000 Received: from CY4PR0601MB3713.namprd06.prod.outlook.com ([fe80::e03a:dba6:a8d0:17e0]) by CY4PR0601MB3713.namprd06.prod.outlook.com ([fe80::e03a:dba6:a8d0:17e0%7]) with mapi id 15.20.3499.029; Mon, 2 Nov 2020 05:58:01 +0000 From: Greg White To: Christian Huitema , "De Schepper, Koen (Nokia - BE/Antwerp)" , Jonathan Morton CC: iccrg IRTF list , tsvwg IETF list , TCP Prague List Thread-Topic: [tsvwg] [tcpPrague] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text Thread-Index: AQHWsKqTEvlUwocdKECuNOe/y5VAQqmz9+eA///rqAA= Date: Mon, 2 Nov 2020 05:58:00 +0000 Message-ID: <35DF946B-06F0-44DD-A220-40D74777385D@cablelabs.com> References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/16.42.20100402 authentication-results: huitema.net; dkim=none (message not signed) header.d=none;huitema.net; dmarc=none action=none header.from=CableLabs.com; x-originating-ip: [98.245.82.7] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a083fd2f-bbcc-45df-674f-08d87ef44191 x-ms-traffictypediagnostic: CY4PR06MB2375: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Sl2ttzuqgceSrTnuq3zbYSiykOlf3uPEZj5EJOPhVAccNPI4onwgIexbBy/8867lNJ75f0XUt/Xu8k0s7R6MDpJziU/L5/kkksI7ZN78EBHfnpzFNBMrRYCsvqWHW98tAToedeBzZJZkASR8gq+TAbYR8W2+S2ZCu8Vfp4t6ZbUkTCGWgv5L1CabAYIlZhxs91R2pLZXHn2SrZ+ck62ZOaM8vARDm1pfIWIKmvrm0T4AX7PkF5blvZd+0oVI7uKufflDSbF+H0KI/bXalZU/S1Vv3gTgfbKQro4d3kGernbj19IzopnDO/9n4zNvCiDE3lAKqOSjhE71BOi2EN62iwrZyIVGkxxKO8VxKkDCxYF1lun8Et24/XPhK+uS6PDYBBlvexIy+AvLWWIYpNUWoOR31egh2W3ExInhyrVv565gtjGyS6jKw+C6cb6LWKygWvOfokE0oeOxMy4GaWEdxA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CY4PR0601MB3713.namprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(396003)(346002)(376002)(136003)(366004)(39830400003)(26005)(6486002)(186003)(53546011)(478600001)(71200400001)(83380400001)(6512007)(2906002)(66446008)(64756008)(66556008)(66476007)(5660300002)(6506007)(66946007)(2616005)(166002)(76116006)(54906003)(966005)(8936002)(316002)(33656002)(110136005)(86362001)(36756003)(8676002)(4326008)(85282002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: UAgNwlVr1sJ9KLALO271/uPB39kklH4Wtz50s8O/zqki6rCbX4MNnE30Q/JZuthDvZJdLmSSr+Yj+iWYQUuKUr3TQFmisDiDaK8gMFO2odnH5iSL+Q2gkOwEX7iHT6rwxFpwEbEMmbKh5PkjQXebMYSH1nn3YLNW9CBMbC/dlUuTyvkuZruhpri5OPx5Fom41rzn7dhAtWfF/lHJ3bBB1wWDKrI3X4XFCJGr35XXJXZAm6tTgN/QpmiQacokZCinNSiWNKVUI3GE6P9UAe325TYBV02MyLCc0XsKHDnl2cGtDBF5yuzqbnLdPuVuQXwc1FQQlAVIzRJf5tqhFePGjLxefzp630nKXswssWe5IqGua1ldBIRHOneVeBUmBpZHAnMoKTkHm4M6QKy/YPK5Ppjq/BmMwuJCUOsJvsOiRZLNiV4fPwTeyQh2TGT+IS/SrNXCz7RO1Wj8NODmV51ZHInrl22jK9O/WnXGCVBRAvKpLGaf4diTzk8I0QH0Y7PLw+PFi60RTgCe+HDyL3hT4DntflxH7d2nmzuruSanDPAySSG4XeMFE5eY1COeaN0KqjYIF2GgZ435KklC7Pk0/TiY0DDoFx4EXkvqphuhtRz1FPOCg5vARJaCqPVq6MX+BhOZuhUd0Yo/1rGmIr9v9g== x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_35DF946B06F044DDA22040D74777385Dcablelabscom_" MIME-Version: 1.0 X-OriginatorOrg: cablelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR0601MB3713.namprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a083fd2f-bbcc-45df-674f-08d87ef44191 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Nov 2020 05:58:00.8784 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ce4fbcd1-1d81-4af0-ad0b-2998c441e160 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: o0DRITjiQJhYsVaWZ+iZU+Upos0GdbmCGT6zNnMzyMZM2koLeSUHkOOnzbKmTkuSFa2FZrv25LhM22prQsHEzA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR06MB2375 Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 05:58:19 -0000 --_000_35DF946B06F044DDA22040D74777385Dcablelabscom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 T24gMTEvMS8yMDIwIDM6NTUgUE0sIENocmlzdGlhbiBIdWl0ZW1hIHdyb3RlOg0KT24gMTEvMS8y MDIwIDE6NTMgUE0sIERlIFNjaGVwcGVyLCBLb2VuIChOb2tpYSAtIEJFL0FudHdlcnApIHdyb3Rl Og0KDQo+PkJ5IHRoZSB3YXksIEkgZG8gdGhpbmsgdGhhdCBMNFMgbWFrZXMgdG9vIG1hbnkgaHlw b3RoZXNlcyBhYm91dCB0aGUgQ0MuIEkgd2lzaCB0aGUgQVFNIHdhcyBiYXNlZCBvbiBpbnRyaW5z aWMgcHJvcGVydGllcywgc3VjaCBhcyAidGhpcyBmbG93IGlzIHVzaW5nIG1vcmUgdGhhbiBpdHMg ZmFpciBzaGFyZSBvZiBiYW5kd2lkdGgiDQoNClRoaXMgbGFzdCBxdW90ZSBpcyBleGFjdGx5IHdo YXQgdGhlIEw0UyBtYXJraW5nIGlzIGludGVuZGVkIHRvIGRvOiB0aGUgc21vb3RoZWQgbWFya2lu ZyBwcm9iYWJpbGl0eSBvdmVyIGEgbG9uZ2VyIHRpbWUgaW50ZXJ2YWwgKGFib3V0IDEwIFJUVHMp IGluZGljYXRlcyB0aGUgZmFpciB0YXJnZXQgcmF0ZS4gQXNzdW1pbmcgd2UgYWdyZWUgb24gMTVt cyBiZWluZyB0aGUgcmVmZXJlbmNlIFJUVCwgdGhlbiB0aGUgYXZlcmFnZSByYXRlIGNhbiBiZSBy PTIvKHAqMC4wMTUpPTEzMy9wIHBhY2tldHMgcGVyIHNlY29uZC4gU28geW91IGNhbiBldmFsdWF0 ZSB5b3Vyc2VsZiBpZiB5b3UgdXNlIG1vcmUgb3IgbGVzcyB0aGFuIHRoZSBmYWlyIHNoYXJlLiBI b3cgeW91IHJlc3BvbmQgdG8gdGhpcyBrbm93bGVkZ2UgaXMgdXAgdG8geW91ciBDQy4NCg0KSSB0 aGluayB0aGF0IHBhcnQgb2YgdGhlIGNvbXBsZXhpdHkgaW4gTDRTIEFRTSBjb21lcyBmcm9tIHRy eWluZyB0byBhY2hpZXZlIGZhaXIgc2hhcmluZyBvZiByZXNvdXJjZSB3aXRob3V0IGFjdHVhbGx5 IGltcGxlbWVudGluZyBmYWlyIHF1ZXVpbmcuIFlvdSBlbmQgdXAgZXhwZWN0aW5nIHRoYXQgaW1w bGVtZW50YXRpb25zIGFwcGx5IGEgZm9ybXVsYSBhbmQgZGVyaXZlIHRoZWlyIHBhY2tldCBzZW5k aW5nIHJhdGUgZnJvbSBhIHNtb290aGVkIGF2ZXJhZ2Ugb2YgdGhlIHBhY2tldCBtYXJraW5nIHJh dGUuIEl0IG1heSB3b3JrIHdlbGwgaW4gY29udHJvbGxlZCBlbnZpcm9ubWVudHMsIGJ1dCBJIGFt IGFmcmFpZCB0aGF0IHZlcnkgZmV3IGltcGxlbWVudGF0aW9ucyBhcmUgZXZlciBnb2luZyB0byBk byB0aGF0IGluIHByYWN0aWNlLg0KDQpDb25nZXN0aW9uIENvbnRyb2wgYWxnb3JpdGhtcyBhcmUg bXVjaCBtb3JlIGxpa2VseSB0byB0cmVhdCBwYWNrZXQgbG9zc2VzLCBtYXJraW5ncywgb3IgZGVs YXkgaW5jcmVhc2VzIGFzIHNpZ25hbHMgdGhhdCB0aGV5IGFyZSBzZW5kaW5nIHRvbyBmYXN0LCB3 aGlsZSBjb25zaWRlcmluZyB0aGVpciBhYnNlbmNlIG9yIGxvd2VyIGZyZXF1ZW5jeSBhcyBzaWdu YWwgdGhhdCB0aGV5IG1pZ2h0IGJlIGFibGUgdG8gc2VuZCBmYXN0ZXIuIEltcGxlbWVudGF0aW9u cyBhbHNvIGFyZSBsaWtlbHkgdG8gdGFrZSBpbnRvIGFjY291bnQgdGhlIHJlY2VudCBoaXN0b3J5 LCBmb3IgZXhhbXBsZSBtb25pdG9yaW5nIHRoZSBtaW4gUlRUIChMRURCQVQsIEJCUiksIG9yIG1v bml0b3JpbmcgdGhlIHJlY2VudCBhdmFpbGFibGUgYmFuZHdpZHRoIChCQlIpLiBUaGV5IGFsc28g dHlwaWNhbGx5IGZpbHRlciBzaWduYWxzLCBlLmcuLCBjb25zaWRlciBvbmx5IG9uZSBwYWNrZXQg bG9zcyBvciBFQ04gbWFyayBwZXIgUlRUIChOZXcgUmVubywgQ3ViaWMsIGV0Yy4pLiBUaGlzIGlz IGV4cGVjdGVkLCBiZWNhdXNlIHdlIGRvIHNlZSBwYWNrZXQgbG9zc2VzIGFycml2aW5nIGluIGJh dGNoZXMsIGFuZCBjYW4gcmVhc29uYWJseSBleHBlY3QgRUNOIG1hcmtzIHRvIGhhdmUgdGhlIHNh bWUgYmVoYXZpb3IuIEJ1dCBpbXBsZW1lbnRhdGlvbnMgYXJlIGluaGVyZW50bHkgc2VsZmlzaCwg ZG8gbm90IHJlYWxseSB0cnVzdCB0aGUgc2lnbmFscyBmcm9tIHRoZSBuZXR3b3JrLCBhbmQgYXJl IHZlcnkgdW5saWtlbHkgdG8gZW5kIHVwIGFwcGx5aW5nIHRoZSBmb3JtdWxhIHRoYXQgeW91IHN1 Z2dlc3QuDQoNCkluIGFueSBjYXNlLCB5b3UgaGF2ZSBhIHNjYWxpbmcgaXNzdWUuIExldCdzIGNv bnNpZGVyIGEgMS41R2JwcyBsaW5rLCB3aXRoIDE1IG1zIGRlbGF5IGFuZCAxNTAwIGJ5dGVzIHBh Y2tldHMuIFRoZSBub21pbmFsIHNlbmRpbmcgcmF0ZSBpcyAxMjUsMDAwIHBhY2tldHMgcGVyIHNl Y29uZC4gVGhlIG1hcmtpbmcgcmF0ZSB3aXRoIHlvdXIgZm9ybXVsYSBzaGFsbCBiZSBwID0gMi8o ciowLjAxNSksIHdoaWNoIGlzIGFib3V0IDAuMDAwOCUuIE92ZXIgdGhlIGxhc3QgMTAgUlRULCB0 aGUgY29ubmVjdGlvbiB3aWxsIG9uIGF2ZXJhZ2Ugc2VlIDAuMTQgbWFya3MgLS0gdGhhdCBpcywg bm8gbWFyayBvdmVyIHRoZSBsYXN0IDEwIFJUVCA4NiUgb2YgdGhlIHRpbWUuIFRoaXMgZmFsbHMg d2VsbCBzaG9ydCBvZiB0aGUgcmVxdWlyZW1lbnQgdG8gcHJvdmlkZSBmcmVxdWVudCBmZWVkYmFj ayENCg0KU29ycnksIGJ1ZyBpbiBteSBzcHJlYWRzaGVldC4gVGhlIG1hcmtpbmcgcmF0ZSBpcyBh Y3R1YWxseSAwLjElLCBhYm91dCAyIHBhY2tldHMgcGVyIFJUVC4gU3RpbGwgbm90IGZyZXF1ZW50 IGVub3VnaCBmb3IgbXkgdGFzdGUsIGJ1dCBtdWNoIGJldHRlciB0aGFuIDAuMDAwOCUuIE5ldmVy dGhlbGVzcywgdGhlIGludmVyc2UgcmVsYXRpb24gYmV0d2VlbiBtYXJraW5nIHJhdGUgYW5kIGRh dGEgcmF0ZSBpcyBub3QgZ3JlYXQuDQoNCltHV10gQnV0LCB0aGlzIGFsd2F5cyB3b3JrcyBvdXQg dG8gMiBtYXJrcyBwZXIgUlRUIG9yIDIgbWFya3MgcGVyIDE1bXMgKHdoaWNoZXZlciBpcyBzbG93 ZXIpLCBzbyBpbiB0aGF0IHNlbnNlIGl0IGRvZXNu4oCZdCBkZXBlbmQgb24gZGF0YSByYXRlIGF0 IGFsbC4gIFRoZSBzZW5kZXIgc2xvd3MgZG93biBpZiBpdCBzZWVzIG1vcmUgZnJlcXVlbnQgbWFy a3MgdGhhbiB0aGlzLCBhY2NlbGVyYXRlcyBpZiBpdCBzZWVzIGZld2VyIG1hcmtzIHRoYW4gdGhp cywgYW5kIHN0YXlzIHB1dCBpZiBpdCBzZWVzIG1hcmtpbmcgYXQgdGhpcyByYXRlLg0KDQpJZiB5 b3UgZ2l2ZSB1cCB0aGUgZm9ybXVsYSwgeW91IGNvdWxkIGVuZCB1cCB3aXRoIGEgc2lnbmlmaWNh bnRseSBzaW1wbGVyIGV2ZW50IGJhc2VkIGNvbmdlc3Rpb24gY29udHJvbCwgaW4gd2hpY2ggYXBw bGljYXRpb25zIHNsb3cgZG93biBpZiB0aGV5IHNlZSB0b28gbWFueSBDRSBtYXJrcywgYWNjZWxl cmF0ZSBpZiB0aGV5IG9ubHkgc2VlIGEgdmVyeSBsb3cgbWFya2luZyByYXRlLCBhbmQgc3RheSBw dXQgaWYgdGhleSBzZWUgYW4gaW5mcmVxdWVudCBtYXJraW5nIHJhdGUuIEkgY2FuIHNlZSBhcHBs aWNhdGlvbnMgZG9pbmcgdGhhdCwgcmVzdWx0aW5nIGluIGEgZmFpcmx5IHN0YWJsZSBuZXR3b3Jr IGFuZCBsb3cgZGVsYXlzLiBPZiBjb3Vyc2UsIHlvdSB3aWxsIGhhdmUgdG8gZG8gc29tZSBmb3Jt IG9mIGZhaXIgc2hhcmUgZW5mb3JjZW1lbnQgdG8gY2F0Y2ggdmlvbGF0b3JzLCBidXQgSSBhbSBz dXJlIHlvdSBjYW4gY29tZSB1cCB3aXRoIHNvbWV0aGluZy4NCg0KLS0gQ2hyaXN0aWFuIEh1aXRl bWENCg0KDQoNCg0KDQoNCg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18NCg0KdGNwUHJhZ3VlIG1haWxpbmcgbGlzdA0KDQp0Y3BQcmFndWVAaWV0Zi5vcmc8 bWFpbHRvOnRjcFByYWd1ZUBpZXRmLm9yZz4NCg0KaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1h bi9saXN0aW5mby90Y3BwcmFndWUNCg== --_000_35DF946B06F044DDA22040D74777385Dcablelabscom_ Content-Type: text/html; charset="utf-8" Content-ID: <8BA0F7694219514EA7ED6A0D1F47142F@namprd06.prod.outlook.com> Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4 bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l dGEgbmFtZT0iR2VuZXJhdG9yIiBjb250ZW50PSJNaWNyb3NvZnQgV29yZCAxNSAoZmlsdGVyZWQg bWVkaXVtKSI+DQo8c3R5bGU+PCEtLQ0KLyogRm9udCBEZWZpbml0aW9ucyAqLw0KQGZvbnQtZmFj ZQ0KCXtmb250LWZhbWlseToiQ2FtYnJpYSBNYXRoIjsNCglwYW5vc2UtMToyIDQgNSAzIDUgNCA2 IDMgMiA0O30NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsNCglwYW5vc2UtMToy IDE1IDUgMiAyIDIgNCAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNvbnNvbGFz Ow0KCXBhbm9zZS0xOjIgMTEgNiA5IDIgMiA0IDMgMiA0O30NCi8qIFN0eWxlIERlZmluaXRpb25z ICovDQpwLk1zb05vcm1hbCwgbGkuTXNvTm9ybWFsLCBkaXYuTXNvTm9ybWFsDQoJe21hcmdpbjow aW47DQoJZm9udC1zaXplOjExLjBwdDsNCglmb250LWZhbWlseToiQ2FsaWJyaSIsc2Fucy1zZXJp Zjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsN Cgljb2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7fQ0KcHJlDQoJe21zby1z dHlsZS1wcmlvcml0eTo5OTsNCgltc28tc3R5bGUtbGluazoiSFRNTCBQcmVmb3JtYXR0ZWQgQ2hh ciI7DQoJbWFyZ2luOjBpbjsNCglmb250LXNpemU6MTAuMHB0Ow0KCWZvbnQtZmFtaWx5OiJDb3Vy aWVyIE5ldyI7fQ0Kc3Bhbi5IVE1MUHJlZm9ybWF0dGVkQ2hhcg0KCXttc28tc3R5bGUtbmFtZToi SFRNTCBQcmVmb3JtYXR0ZWQgQ2hhciI7DQoJbXNvLXN0eWxlLXByaW9yaXR5Ojk5Ow0KCW1zby1z dHlsZS1saW5rOiJIVE1MIFByZWZvcm1hdHRlZCI7DQoJZm9udC1mYW1pbHk6Q29uc29sYXM7fQ0K c3Bhbi5FbWFpbFN0eWxlMjENCgl7bXNvLXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9u dC1mYW1pbHk6IkNhbGlicmkiLHNhbnMtc2VyaWY7DQoJY29sb3I6d2luZG93dGV4dDt9DQouTXNv Q2hwRGVmYXVsdA0KCXttc28tc3R5bGUtdHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAu MHB0O30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0KCXtzaXplOjguNWluIDExLjBpbjsNCgltYXJnaW46 MS4waW4gMS4waW4gMS4waW4gMS4waW47fQ0KZGl2LldvcmRTZWN0aW9uMQ0KCXtwYWdlOldvcmRT ZWN0aW9uMTt9DQotLT48L3N0eWxlPg0KPC9oZWFkPg0KPGJvZHkgbGFuZz0iRU4tVVMiIGxpbms9 ImJsdWUiIHZsaW5rPSJwdXJwbGUiIHN0eWxlPSJ3b3JkLXdyYXA6YnJlYWstd29yZCI+DQo8ZGl2 IGNsYXNzPSJXb3JkU2VjdGlvbjEiPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxl PSJtYXJnaW4tbGVmdDouNWluIj5PbiAxMS8xLzIwMjAgMzo1NSBQTSwgQ2hyaXN0aWFuIEh1aXRl bWEgd3JvdGU6PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJtYXJn aW4tdG9wOjUuMHB0O21hcmdpbi1ib3R0b206NS4wcHQiPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O b3JtYWwiIHN0eWxlPSJtYXJnaW4tbGVmdDouNWluIj5PbiAxMS8xLzIwMjAgMTo1MyBQTSwgRGUg U2NoZXBwZXIsIEtvZW4gKE5va2lhIC0gQkUvQW50d2VycCkgd3JvdGU6PG86cD48L286cD48L3A+ DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJtYXJnaW4tdG9wOjUuMHB0O21hcmdpbi1ib3R0 b206NS4wcHQiPg0KPHAgc3R5bGU9Im1hcmdpbi1sZWZ0Oi41aW4iPiZndDsmZ3Q7QnkgdGhlIHdh eSwgSSBkbyB0aGluayB0aGF0IEw0UyBtYWtlcyB0b28gbWFueSBoeXBvdGhlc2VzIGFib3V0IHRo ZSBDQy4gSSB3aXNoIHRoZSBBUU0gd2FzIGJhc2VkIG9uIGludHJpbnNpYyBwcm9wZXJ0aWVzLCBz dWNoIGFzICZxdW90O3RoaXMgZmxvdyBpcyB1c2luZyBtb3JlIHRoYW4gaXRzIGZhaXIgc2hhcmUg b2YgYmFuZHdpZHRoJnF1b3Q7DQo8bzpwPjwvbzpwPjwvcD4NCjxwIHN0eWxlPSJtYXJnaW4tbGVm dDouNWluIj5UaGlzIGxhc3QgcXVvdGUgaXMgZXhhY3RseSB3aGF0IHRoZSBMNFMgbWFya2luZyBp cyBpbnRlbmRlZCB0byBkbzogdGhlIHNtb290aGVkIG1hcmtpbmcgcHJvYmFiaWxpdHkgb3ZlciBh IGxvbmdlciB0aW1lIGludGVydmFsIChhYm91dCAxMCBSVFRzKSBpbmRpY2F0ZXMgdGhlIGZhaXIg dGFyZ2V0IHJhdGUuIEFzc3VtaW5nIHdlIGFncmVlIG9uIDE1bXMgYmVpbmcgdGhlIHJlZmVyZW5j ZSBSVFQsIHRoZW4NCiB0aGUgYXZlcmFnZSByYXRlIGNhbiBiZSByPTIvKHAqMC4wMTUpPTEzMy9w IHBhY2tldHMgcGVyIHNlY29uZC4gU28geW91IGNhbiBldmFsdWF0ZSB5b3Vyc2VsZiBpZiB5b3Ug dXNlIG1vcmUgb3IgbGVzcyB0aGFuIHRoZSBmYWlyIHNoYXJlLiBIb3cgeW91IHJlc3BvbmQgdG8g dGhpcyBrbm93bGVkZ2UgaXMgdXAgdG8geW91ciBDQy48bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2tx dW90ZT4NCjxwIHN0eWxlPSJtYXJnaW4tbGVmdDouNWluIj5JIHRoaW5rIHRoYXQgcGFydCBvZiB0 aGUgY29tcGxleGl0eSBpbiBMNFMgQVFNIGNvbWVzIGZyb20gdHJ5aW5nIHRvIGFjaGlldmUgZmFp ciBzaGFyaW5nIG9mIHJlc291cmNlIHdpdGhvdXQgYWN0dWFsbHkgaW1wbGVtZW50aW5nIGZhaXIg cXVldWluZy4gWW91IGVuZCB1cCBleHBlY3RpbmcgdGhhdCBpbXBsZW1lbnRhdGlvbnMgYXBwbHkg YSBmb3JtdWxhIGFuZCBkZXJpdmUgdGhlaXIgcGFja2V0IHNlbmRpbmcNCiByYXRlIGZyb20gYSBz bW9vdGhlZCBhdmVyYWdlIG9mIHRoZSBwYWNrZXQgbWFya2luZyByYXRlLiBJdCBtYXkgd29yayB3 ZWxsIGluIGNvbnRyb2xsZWQgZW52aXJvbm1lbnRzLCBidXQgSSBhbSBhZnJhaWQgdGhhdCB2ZXJ5 IGZldyBpbXBsZW1lbnRhdGlvbnMgYXJlIGV2ZXIgZ29pbmcgdG8gZG8gdGhhdCBpbiBwcmFjdGlj ZS4NCjxvOnA+PC9vOnA+PC9wPg0KPHAgc3R5bGU9Im1hcmdpbi1sZWZ0Oi41aW4iPkNvbmdlc3Rp b24gQ29udHJvbCBhbGdvcml0aG1zIGFyZSBtdWNoIG1vcmUgbGlrZWx5IHRvIHRyZWF0IHBhY2tl dCBsb3NzZXMsIG1hcmtpbmdzLCBvciBkZWxheSBpbmNyZWFzZXMgYXMgc2lnbmFscyB0aGF0IHRo ZXkgYXJlIHNlbmRpbmcgdG9vIGZhc3QsIHdoaWxlIGNvbnNpZGVyaW5nIHRoZWlyIGFic2VuY2Ug b3IgbG93ZXIgZnJlcXVlbmN5IGFzIHNpZ25hbCB0aGF0IHRoZXkgbWlnaHQgYmUgYWJsZQ0KIHRv IHNlbmQgZmFzdGVyLiBJbXBsZW1lbnRhdGlvbnMgYWxzbyBhcmUgbGlrZWx5IHRvIHRha2UgaW50 byBhY2NvdW50IHRoZSByZWNlbnQgaGlzdG9yeSwgZm9yIGV4YW1wbGUgbW9uaXRvcmluZyB0aGUg bWluIFJUVCAoTEVEQkFULCBCQlIpLCBvciBtb25pdG9yaW5nIHRoZSByZWNlbnQgYXZhaWxhYmxl IGJhbmR3aWR0aCAoQkJSKS4gVGhleSBhbHNvIHR5cGljYWxseSBmaWx0ZXIgc2lnbmFscywgZS5n LiwgY29uc2lkZXIgb25seSBvbmUgcGFja2V0DQogbG9zcyBvciBFQ04gbWFyayBwZXIgUlRUIChO ZXcgUmVubywgQ3ViaWMsIGV0Yy4pLiBUaGlzIGlzIGV4cGVjdGVkLCBiZWNhdXNlIHdlIGRvIHNl ZSBwYWNrZXQgbG9zc2VzIGFycml2aW5nIGluIGJhdGNoZXMsIGFuZCBjYW4gcmVhc29uYWJseSBl eHBlY3QgRUNOIG1hcmtzIHRvIGhhdmUgdGhlIHNhbWUgYmVoYXZpb3IuIEJ1dCBpbXBsZW1lbnRh dGlvbnMgYXJlIGluaGVyZW50bHkgc2VsZmlzaCwgZG8gbm90IHJlYWxseSB0cnVzdCB0aGUgc2ln bmFscw0KIGZyb20gdGhlIG5ldHdvcmssIGFuZCBhcmUgdmVyeSB1bmxpa2VseSB0byBlbmQgdXAg YXBwbHlpbmcgdGhlIGZvcm11bGEgdGhhdCB5b3Ugc3VnZ2VzdC48bzpwPjwvbzpwPjwvcD4NCjxw IHN0eWxlPSJtYXJnaW4tbGVmdDouNWluIj5JbiBhbnkgY2FzZSwgeW91IGhhdmUgYSBzY2FsaW5n IGlzc3VlLiBMZXQncyBjb25zaWRlciBhIDEuNUdicHMgbGluaywgd2l0aCAxNSBtcyBkZWxheSBh bmQgMTUwMCBieXRlcyBwYWNrZXRzLiBUaGUgbm9taW5hbCBzZW5kaW5nIHJhdGUgaXMgMTI1LDAw MCBwYWNrZXRzIHBlciBzZWNvbmQuIFRoZSBtYXJraW5nIHJhdGUgd2l0aCB5b3VyIGZvcm11bGEg c2hhbGwgYmUgcCA9IDIvKHIqMC4wMTUpLCB3aGljaA0KIGlzIGFib3V0IDAuMDAwOCUuIE92ZXIg dGhlIGxhc3QgMTAgUlRULCB0aGUgY29ubmVjdGlvbiB3aWxsIG9uIGF2ZXJhZ2Ugc2VlIDAuMTQg bWFya3MgLS0gdGhhdCBpcywgbm8gbWFyayBvdmVyIHRoZSBsYXN0IDEwIFJUVCA4NiUgb2YgdGhl IHRpbWUuIFRoaXMgZmFsbHMgd2VsbCBzaG9ydCBvZiB0aGUgcmVxdWlyZW1lbnQgdG8gcHJvdmlk ZSBmcmVxdWVudCBmZWVkYmFjayE8bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2txdW90ZT4NCjxwIHN0 eWxlPSJtYXJnaW4tbGVmdDouNWluIj5Tb3JyeSwgYnVnIGluIG15IHNwcmVhZHNoZWV0LiBUaGUg bWFya2luZyByYXRlIGlzIGFjdHVhbGx5IDAuMSUsIGFib3V0IDIgcGFja2V0cyBwZXIgUlRULiBT dGlsbCBub3QgZnJlcXVlbnQgZW5vdWdoIGZvciBteSB0YXN0ZSwgYnV0IG11Y2ggYmV0dGVyIHRo YW4gMC4wMDA4JS4gTmV2ZXJ0aGVsZXNzLCB0aGUgaW52ZXJzZSByZWxhdGlvbiBiZXR3ZWVuIG1h cmtpbmcgcmF0ZSBhbmQgZGF0YSByYXRlDQogaXMgbm90IGdyZWF0LjxvOnA+PC9vOnA+PC9wPg0K PHA+W0dXXSBCdXQsIHRoaXMgYWx3YXlzIHdvcmtzIG91dCB0byAyIG1hcmtzIHBlciBSVFQgb3Ig MiBtYXJrcyBwZXIgMTVtcyAod2hpY2hldmVyIGlzIHNsb3dlciksIHNvIGluIHRoYXQgc2Vuc2Ug aXQgZG9lc27igJl0IGRlcGVuZCBvbiBkYXRhIHJhdGUgYXQgYWxsLiZuYnNwOyBUaGUgc2VuZGVy IHNsb3dzIGRvd24gaWYgaXQgc2VlcyBtb3JlIGZyZXF1ZW50IG1hcmtzIHRoYW4gdGhpcywgYWNj ZWxlcmF0ZXMgaWYgaXQgc2VlcyBmZXdlciBtYXJrcyB0aGFuDQogdGhpcywgYW5kIHN0YXlzIHB1 dCBpZiBpdCBzZWVzIG1hcmtpbmcgYXQgdGhpcyByYXRlLjxvOnA+PC9vOnA+PC9wPg0KPGJsb2Nr cXVvdGUgc3R5bGU9Im1hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLWJvdHRvbTo1LjBwdCI+DQo8cCBz dHlsZT0ibWFyZ2luLWxlZnQ6LjVpbiI+SWYgeW91IGdpdmUgdXAgdGhlIGZvcm11bGEsIHlvdSBj b3VsZCBlbmQgdXAgd2l0aCBhIHNpZ25pZmljYW50bHkgc2ltcGxlciBldmVudCBiYXNlZCBjb25n ZXN0aW9uIGNvbnRyb2wsIGluIHdoaWNoIGFwcGxpY2F0aW9ucyBzbG93IGRvd24gaWYgdGhleSBz ZWUgdG9vIG1hbnkgQ0UgbWFya3MsIGFjY2VsZXJhdGUgaWYgdGhleSBvbmx5IHNlZSBhIHZlcnkg bG93IG1hcmtpbmcgcmF0ZSwgYW5kIHN0YXkNCiBwdXQgaWYgdGhleSBzZWUgYW4gaW5mcmVxdWVu dCBtYXJraW5nIHJhdGUuIEkgY2FuIHNlZSBhcHBsaWNhdGlvbnMgZG9pbmcgdGhhdCwgcmVzdWx0 aW5nIGluIGEgZmFpcmx5IHN0YWJsZSBuZXR3b3JrIGFuZCBsb3cgZGVsYXlzLiBPZiBjb3Vyc2Us IHlvdSB3aWxsIGhhdmUgdG8gZG8gc29tZSBmb3JtIG9mIGZhaXIgc2hhcmUgZW5mb3JjZW1lbnQg dG8gY2F0Y2ggdmlvbGF0b3JzLCBidXQgSSBhbSBzdXJlIHlvdSBjYW4gY29tZSB1cCB3aXRoIHNv bWV0aGluZy48bzpwPjwvbzpwPjwvcD4NCjxwIHN0eWxlPSJtYXJnaW4tbGVmdDouNWluIj4tLSBD aHJpc3RpYW4gSHVpdGVtYTxvOnA+PC9vOnA+PC9wPg0KPHAgc3R5bGU9Im1hcmdpbi1sZWZ0Oi41 aW4iPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgc3R5bGU9Im1hcmdpbi1sZWZ0Oi41aW4iPjxv OnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1s ZWZ0Oi41aW4iPjxicj4NCjxicj4NCjxvOnA+PC9vOnA+PC9wPg0KPHByZSBzdHlsZT0ibWFyZ2lu LWxlZnQ6LjVpbiI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X188bzpwPjwvbzpwPjwvcHJlPg0KPHByZSBzdHlsZT0ibWFyZ2luLWxlZnQ6LjVpbiI+dGNwUHJh Z3VlIG1haWxpbmcgbGlzdDxvOnA+PC9vOnA+PC9wcmU+DQo8cHJlIHN0eWxlPSJtYXJnaW4tbGVm dDouNWluIj48YSBocmVmPSJtYWlsdG86dGNwUHJhZ3VlQGlldGYub3JnIj50Y3BQcmFndWVAaWV0 Zi5vcmc8L2E+PG86cD48L286cD48L3ByZT4NCjxwcmUgc3R5bGU9Im1hcmdpbi1sZWZ0Oi41aW4i PjxhIGhyZWY9Imh0dHBzOi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwcHJhZ3Vl Ij5odHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFuL2xpc3RpbmZvL3RjcHByYWd1ZTwvYT48bzpw PjwvbzpwPjwvcHJlPg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2JvZHk+DQo8L2h0bWw+DQo= --_000_35DF946B06F044DDA22040D74777385Dcablelabscom_-- From nobody Sun Nov 1 22:50:50 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 904E63A0E0D for ; Sun, 1 Nov 2020 22:50:47 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.135 X-Spam-Level: X-Spam-Status: No, score=-2.135 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.247, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xKtkvTFVtNCG for ; Sun, 1 Nov 2020 22:50:44 -0800 (PST) Received: from mx36-out10.antispamcloud.com (mx36-out10.antispamcloud.com [209.126.121.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B82AB3A0E02 for ; Sun, 1 Nov 2020 22:50:44 -0800 (PST) Received: from xse466.mail2web.com ([66.113.197.212] helo=xse.mail2web.com) by mx128.antispamcloud.com with esmtp (Exim 4.92) (envelope-from ) id 1kZTft-0015HR-UN for tcpPrague@ietf.org; Mon, 02 Nov 2020 07:50:43 +0100 Received: from xsmtp21.mail2web.com (unknown [10.100.68.60]) by xse.mail2web.com (Postfix) with ESMTPS id 4CPk805pgsz8lm for ; Sun, 1 Nov 2020 22:50:40 -0800 (PST) Received: from [10.5.2.14] (helo=xmail04.myhosting.com) by xsmtp21.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1kZTfs-0000O3-Ln for tcpPrague@ietf.org; Sun, 01 Nov 2020 22:50:40 -0800 Received: (qmail 17649 invoked from network); 2 Nov 2020 06:50:40 -0000 Received: from unknown (HELO [192.168.1.107]) (Authenticated-user:_huitema@huitema.net@[172.58.43.139]) (envelope-sender ) by xmail04.myhosting.com (qmail-ldap-1.03) with ESMTPA for ; 2 Nov 2020 06:50:40 -0000 To: Greg White , "De Schepper, Koen (Nokia - BE/Antwerp)" , Jonathan Morton Cc: iccrg IRTF list , tsvwg IETF list , TCP Prague List References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> <35DF946B-06F0-44DD-A220-40D74777385D@cablelabs.com> From: Christian Huitema Autocrypt: addr=huitema@huitema.net; prefer-encrypt=mutual; keydata= mDMEXtavGxYJKwYBBAHaRw8BAQdA1ou9A5MHTP9N3jfsWzlDZ+jPnQkusmc7sfLmWVz1Rmu0 J0NocmlzdGlhbiBIdWl0ZW1hIDxodWl0ZW1hQGh1aXRlbWEubmV0PoiWBBMWCAA+FiEEw3G4 Nwi4QEpAAXUUELAmqKBYtJQFAl7WrxsCGwMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgEC F4AACgkQELAmqKBYtJQbMwD/ebj/qnSbthC/5kD5DxZ/Ip0CGJw5QBz/+fJp3R8iAlsBAMjK r2tmyWyJz0CUkVG24WaR5EAJDvgwDv8h22U6QVkAuDgEXtavGxIKKwYBBAGXVQEFAQEHQJoM 6MUAIqpoqdCIiACiEynZf7nlJg2Eu0pXIhbUGONdAwEIB4h+BBgWCAAmFiEEw3G4Nwi4QEpA AXUUELAmqKBYtJQFAl7WrxsCGwwFCQlmAYAACgkQELAmqKBYtJRm2wD7BzeK5gEXSmBcBf0j BYdSaJcXNzx4yPLbP4GnUMAyl2cBAJzcsR4RkwO4dCRqM9CHpVJCwHtbUDJaa55//E0kp+gH Message-ID: Date: Sun, 1 Nov 2020 22:50:39 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: <35DF946B-06F0-44DD-A220-40D74777385D@cablelabs.com> Content-Type: multipart/alternative; boundary="------------86283D5CADDA09379546A1CF" Content-Language: en-US X-Originating-IP: 66.113.197.212 X-Spampanel-Domain: xsmtpout.mail2web.com X-Spampanel-Username: 66.113.197.0/24 Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.197.0/24@xsmtpout.mail2web.com X-Spampanel-Outgoing-Class: unsure X-Spampanel-Outgoing-Evidence: Combined (0.15) X-Recommended-Action: accept X-Filter-ID: Mvzo4OR0dZXEDF/gcnlw0XvADx2zSFwG+3csxFBPBHmpSDasLI4SayDByyq9LIhVUZbR67CQ7/vm /hHDJU4RXkTNWdUk1Ol2OGx3IfrIJKywOmJyM1qr8uRnWBrbSAGD42+pgDSJdnVPeXOEK1m6uDbl kDZNUkuHiojIVh7uAUtMmQhgAlZ6hsyieRVSg8TDUZq99/v+OWrp+SY7iSuea8EuNfnjQsRqWXYP fdewBkdPR9o014ICOs9pIWX8OtLJdwmMW2Igo3slaTKxvBsN69bNhTNyx2YOvjKTJ3Ps/x2i3GSv wir0OshyEkOwfCYvxs5/T0oXYyKdvWabEYxQFC5Ano+acIK7aoZBBKN3kq8lTiPCf+PwTb/RKAdw /TswIg1fYmNb1kgzViAoNrSrXN1jhnM/Mbva2XLV/LIEzaJm4kYWBU//ZZ8pZ1xZfmaG1NRsUdzW awx6dX0NJ8Bzt99fxN2oReTDHAyOynaY0ClENaQq/2aUAxcG3yLqjApZmdySlZou9qHIGOZDEEo7 O58ZQzrOqjAERHu4pt/Ia6wELzcGxDgkPe7eR6qspNNQGjLhGMBSrFdf8dBbPvtqJwEiRQv+PVjj wa+Z5RFCOMQa1Vus00n+QjkVg8Y9yuTVR7/dvDWlsWWmYvtoo/dCwrK6HflbskIiJsH3WXieD/iI 9kVgrP8d/S99xR9aHf3T0ryXeQDqqkgHDboVrOy1pkXw8BMIsfBCWTvqwtHGsGbsxBoFbdIyTZwD 0tTyhMSsHScFlGkjbz0OvhDlTjXQPYaPmF/7MAKyW1Kb4FKGpjSTHLN3gEp08lb+iT+JYfiyGO8m 5xifdQ6v1Zv869jbIA2MtuoXMRfSG2WhEZRm53tcFvYT7FcUL8JfnIu/hBVs647lNwN4qOsSZg+f YhVZG3Fpf9P2WwOdPRT3EWlMd754CMBw7+ErSKQN7lw8ITGaNmGHCZF9ZXtfSxIkZLsbHZnckpWa LvahyBjmQxBKOzsmHW0N+NPiHPgn+dyxhvT1Z5EPOIlpyGKEA1x50oTQDUtseoetsROuM82YMzIj 8EU= X-Report-Abuse-To: spam@quarantine11.antispamcloud.com Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 06:50:48 -0000 This is a multi-part message in MIME format. --------------86283D5CADDA09379546A1CF Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/1/2020 9:58 PM, Greg White wrote: > > On 11/1/2020 3:55 PM, Christian Huitema wrote: > > On 11/1/2020 1:53 PM, De Schepper, Koen (Nokia - BE/Antwerp) wrote:= > > >>By the way, I do think that L4S makes too many hypotheses > about the CC. I wish the AQM was based on intrinsic > properties, such as "this flow is using more than its fair > share of bandwidth" > > This last quote is exactly what the L4S marking is intended to > do: the smoothed marking probability over a longer time > interval (about 10 RTTs) indicates the fair target rate. > Assuming we agree on 15ms being the reference RTT, then the > average rate can be r=3D2/(p*0.015)=3D133/p packets per second.= So > you can evaluate yourself if you use more or less than the > fair share. How you respond to this knowledge is up to your CC.= > > I think that part of the complexity in L4S AQM comes from trying > to achieve fair sharing of resource without actually implementing > fair queuing. You end up expecting that implementations apply a > formula and derive their packet sending rate from a smoothed > average of the packet marking rate. It may work well in controlled > environments, but I am afraid that very few implementations are > ever going to do that in practice. > > Congestion Control algorithms are much more likely to treat packet > losses, markings, or delay increases as signals that they are > sending too fast, while considering their absence or lower > frequency as signal that they might be able to send faster. > Implementations also are likely to take into account the recent > history, for example monitoring the min RTT (LEDBAT, BBR), or > monitoring the recent available bandwidth (BBR). They also > typically filter signals, e.g., consider only one packet loss or > ECN mark per RTT (New Reno, Cubic, etc.). This is expected, > because we do see packet losses arriving in batches, and can > reasonably expect ECN marks to have the same behavior. But > implementations are inherently selfish, do not really trust the > signals from the network, and are very unlikely to end up applying > the formula that you suggest. > > In any case, you have a scaling issue. Let's consider a 1.5Gbps > link, with 15 ms delay and 1500 bytes packets. The nominal sending > rate is 125,000 packets per second. The marking rate with your > formula shall be p =3D 2/(r*0.015), which is about 0.0008%. Over th= e > last 10 RTT, the connection will on average see 0.14 marks -- that > is, no mark over the last 10 RTT 86% of the time. This falls well > short of the requirement to provide frequent feedback! > > Sorry, bug in my spreadsheet. The marking rate is actually 0.1%, about > 2 packets per RTT. Still not frequent enough for my taste, but much > better than 0.0008%. Nevertheless, the inverse relation between > marking rate and data rate is not great. > > [GW] But, this always works out to 2 marks per RTT or 2 marks per 15ms > (whichever is slower), so in that sense it doesn=E2=80=99t depend on da= ta rate > at all.=C2=A0 The sender slows down if it sees more frequent marks than= > this, accelerates if it sees fewer marks than this, and stays put if > it sees marking at this rate. > Yes, there is that. But even if the rate is large enough, the CC implementations can only program such a constant if they have assurance that every bottleneck that they encounter will use the same time constant. I don't think we are ready to make such assumptions outside of controlled experiments. I would expect a mix of L4S-AQM, CODEL and PIE variants, classic ECN, etc. The host implementation has to be ready to handle all that, and trying to use a time constant there is very challenging. > If you give up the formula, you could end up with a significantly > simpler event based congestion control, in which applications slow > down if they see too many CE marks, accelerate if they only see a > very low marking rate, and stay put if they see an infrequent > marking rate. I can see applications doing that, resulting in a > fairly stable network and low delays. Of course, you will have to > do some form of fair share enforcement to catch violators, but I > am sure you can come up with something. > I still believe that this kind of "natural signaling" is easier to handle than the formulaic behavior expected with L4S. It can easily deliver the "low queing delay" goal of L4S. It cannot guarantee "fair sharing of bandwidth", but my thesis is that such fair sharing requires per-flow enforcement. -- Christian Huitema --------------86283D5CADDA09379546A1CF Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit


On 11/1/2020 9:58 PM, Greg White wrote:

On 11/1/2020 3:55 PM, Christian Huitema wrote:

On 11/1/2020 1:53 PM, De Schepper, Koen (Nokia - BE/Antwerp) wrote:

>>By the way, I do think that L4S makes too many hypotheses about the CC. I wish the AQM was based on intrinsic properties, such as "this flow is using more than its fair share of bandwidth"

This last quote is exactly what the L4S marking is intended to do: the smoothed marking probability over a longer time interval (about 10 RTTs) indicates the fair target rate. Assuming we agree on 15ms being the reference RTT, then the average rate can be r=2/(p*0.015)=133/p packets per second. So you can evaluate yourself if you use more or less than the fair share. How you respond to this knowledge is up to your CC.

I think that part of the complexity in L4S AQM comes from trying to achieve fair sharing of resource without actually implementing fair queuing. You end up expecting that implementations apply a formula and derive their packet sending rate from a smoothed average of the packet marking rate. It may work well in controlled environments, but I am afraid that very few implementations are ever going to do that in practice.

Congestion Control algorithms are much more likely to treat packet losses, markings, or delay increases as signals that they are sending too fast, while considering their absence or lower frequency as signal that they might be able to send faster. Implementations also are likely to take into account the recent history, for example monitoring the min RTT (LEDBAT, BBR), or monitoring the recent available bandwidth (BBR). They also typically filter signals, e.g., consider only one packet loss or ECN mark per RTT (New Reno, Cubic, etc.). This is expected, because we do see packet losses arriving in batches, and can reasonably expect ECN marks to have the same behavior. But implementations are inherently selfish, do not really trust the signals from the network, and are very unlikely to end up applying the formula that you suggest.

In any case, you have a scaling issue. Let's consider a 1.5Gbps link, with 15 ms delay and 1500 bytes packets. The nominal sending rate is 125,000 packets per second. The marking rate with your formula shall be p = 2/(r*0.015), which is about 0.0008%. Over the last 10 RTT, the connection will on average see 0.14 marks -- that is, no mark over the last 10 RTT 86% of the time. This falls well short of the requirement to provide frequent feedback!

Sorry, bug in my spreadsheet. The marking rate is actually 0.1%, about 2 packets per RTT. Still not frequent enough for my taste, but much better than 0.0008%. Nevertheless, the inverse relation between marking rate and data rate is not great.

[GW] But, this always works out to 2 marks per RTT or 2 marks per 15ms (whichever is slower), so in that sense it doesn’t depend on data rate at all.  The sender slows down if it sees more frequent marks than this, accelerates if it sees fewer marks than this, and stays put if it sees marking at this rate.

Yes, there is that. But even if the rate is large enough, the CC implementations can only program such a constant if they have assurance that every bottleneck that they encounter will use the same time constant. I don't think we are ready to make such assumptions outside of controlled experiments. I would expect a mix of L4S-AQM, CODEL and PIE variants, classic ECN, etc. The host implementation has to be ready to handle all that, and trying to use a time constant there is very challenging.

If you give up the formula, you could end up with a significantly simpler event based congestion control, in which applications slow down if they see too many CE marks, accelerate if they only see a very low marking rate, and stay put if they see an infrequent marking rate. I can see applications doing that, resulting in a fairly stable network and low delays. Of course, you will have to do some form of fair share enforcement to catch violators, but I am sure you can come up with something.

I still believe that this kind of "natural signaling" is easier to handle than the formulaic behavior expected with L4S. It can easily deliver the "low queing delay" goal of L4S. It cannot guarantee "fair sharing of bandwidth", but my thesis is that such fair sharing requires per-flow enforcement.

-- Christian Huitema

--------------86283D5CADDA09379546A1CF-- From nobody Mon Nov 2 00:30:28 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5E8193A0A6F; Mon, 2 Nov 2020 00:30:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.102 X-Spam-Level: X-Spam-Status: No, score=-2.102 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VZ1PLqEn3aYJ; Mon, 2 Nov 2020 00:30:22 -0800 (PST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130070.outbound.protection.outlook.com [40.107.13.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 17B583A0A6C; Mon, 2 Nov 2020 00:30:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nRb003sF5umqFZoONSvOfOrYZtRq6nQT0PcZ0iGXXKjPmu3Pye3UuRBqdCQKjge0An3yhDwp0l+tEXDXNbkrCDYmH/F2bj/LhzOvLJdd91pPh3s9mzNrYCG680KHV3EMSJ8maJsbtT/edSn0mI+cg5r9/zOq0H0YCExL8vFtcyk3aCwFXddgA6bk9Vvk47tz0q5c1q/uaPwuIsPpRwLVR6t/x1bxcapp7MitIaUt42lapaGpMxd1W23AdlrR+BVSnK2AgBa9+/hCJa4YfFtPbigpTdUJxWtyEIaIzCrhSjHceHNl5RMKSm0cgwZpSx8Q9MhNPXx6yWRNwP7YwClSaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vFjdvOYh59q8Q2gEjuEqPkBBywubzkXkTOFaVndgG9Q=; b=Jisttw2FSm7QR0Ccs2Rjc7M7wFvuguH7ONcIbSOojSmGnZZKxFySATIecugT1u67FTGz0oCEg4LB24SzsEFhFeek+5sJ5VDGazBhDa7ybb1893dnvPtF6W5Ey9GYhQnC+WZtJv1SmKHM5Ae5EM0bbEStmT4/o2tv6USIUDlrtj9IXBAW4lW39Ro5xvq30Y5xK7RKzbgpQRShcUXiU1mPP3o7G95EaMYiSy+tmXR6k7nx/zxJ3TG5c5FFvV0RSdDib3bP5zRsyg8l6bUU+C7eWG35JcynPRy99nPvdcBJ36/6RoOMeR4XRfF85W5PWVsJODHjf2ZTo1uzJD2S/eIWFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vFjdvOYh59q8Q2gEjuEqPkBBywubzkXkTOFaVndgG9Q=; b=CtaRe7O6u/Vdb5b2Y5T8NkQNC4+emNDW0iTOY3pmFkkKuaFELQ9gaDeHgmhYRZx/3sagDiQCkfZeP9iFWEi0v0IcmEDWgr2fh5baENI1TUu3cF7OOj5Zb12JO5squKZV3xpfPT7q9UKTr68SnqIsmWpCAJJxCh9E79cwiYME3bY= Received: from HE1PR0701MB2876.eurprd07.prod.outlook.com (2603:10a6:3:56::8) by HE1PR07MB4156.eurprd07.prod.outlook.com (2603:10a6:7:9b::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.10; Mon, 2 Nov 2020 08:30:15 +0000 Received: from HE1PR0701MB2876.eurprd07.prod.outlook.com ([fe80::a4a1:d6b5:1254:923a]) by HE1PR0701MB2876.eurprd07.prod.outlook.com ([fe80::a4a1:d6b5:1254:923a%11]) with mapi id 15.20.3541.011; Mon, 2 Nov 2020 08:30:15 +0000 From: Ingemar Johansson S To: "Rodney W. Grimes" , Bob Briscoe CC: Christian Huitema , tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" , Ingemar Johansson S Thread-Topic: [iccrg] Realistic Queue delay targets Thread-Index: AQHWsPJjPxAUbDfzuUaS/obDyN9yVQ== Date: Mon, 2 Nov 2020 08:30:15 +0000 Message-ID: References: <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> <202011012023.0A1KNkue011764@gndrsh.dnsmgr.net> In-Reply-To: <202011012023.0A1KNkue011764@gndrsh.dnsmgr.net> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: authentication-results: gndrsh.dnsmgr.net; dkim=none (message not signed) header.d=none;gndrsh.dnsmgr.net; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [83.227.122.88] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 36fd5d5e-9c57-4f2e-61df-08d87f09866a x-ms-traffictypediagnostic: HE1PR07MB4156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: LydnCF40qvWbZFi2G2PntyakvWPy8xovcsfhv849uzFabUaG2fezcO+2PUENdOAPFX+18BG2Idy0kW0eDQT9Rxe0l7Wvp+pKdXqKMeeaBD1ly35FUG2x2UnqBVVzAIMe7SAmRtNOxO6YDfINwg9/MumrQmz2a86AgwsjL9rwFv9G8hGqbl4GNny4mah+mMLlVgZQ9u8XynWq/WEswXWawjgeIJe1sPBg/7xNAgyX67CeX/c4O8nwPfnUoFdw0bHA2MCp9R3UpDC75bFcduEu2emMxtIKy3EZXiKBEEFG/N0Beb9Kl7p3A6Ezf1sgY15mqkKEZU1UXBO7gKfH0XcgTCa7YiJOe2XCgTOQ637Qd051AMSkgiRb3YaRbHlYadYBlQED+3aGPIxfL+iiCLsquQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:HE1PR0701MB2876.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(376002)(346002)(366004)(396003)(33656002)(86362001)(4326008)(8676002)(8936002)(9686003)(5660300002)(71200400001)(7696005)(478600001)(966005)(55016002)(66574015)(83380400001)(66616009)(6506007)(53546011)(110136005)(186003)(99936003)(26005)(54906003)(107886003)(66556008)(66476007)(66946007)(52536014)(2906002)(76116006)(64756008)(66446008)(316002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: bhjcO6ERRooPjYBK4uGwoqcgZe0XjFyZDeKAQVP3NP3c81nKhJxuPeWa5JWT8/LySEM0RPZ+sZvgKVMZkxMPed8+9ttlqUjnOenWOQG7NQtY4XSZalQnPIElr4TEwR+VWJtu4ypQdtIINOK8L1nemYH50CTkmSz8oGC2v8fDmfj2afxDSgiKs9Pp9iIg/hR0FDJWLJ975cEVepiSvW2I+0jG3M/uQxc8AB5QgGTwc5hnGodAeORpzfAX14VaPPqgbzYfznwShscrLrdgK/eCcla72ZmbMmGORJjs2nL0YSCOK4yInwQNvQ3Hw2cUYwJNWdCAC6bLrKPQsOZEjbzI9lTsqcWLUNe9DwMo3Wn00PRVZhiwOiVU9sXNiSEmjQeWqtsp6CSor7oDOYhzLyQBw5YvwwKieK/eEkIEoz1/2qmYjhE2j1pnlU63p5m4yl2++N7VYYMW3iVjCnyVMtiT0NPQ7zzsMDniGnVaP32LiBoAtQSiFI2LyJT7TJVYW87wmXkcqOITcdxhFtK4RYdwfd4GOIkxm2DZQ2+E47ASTulECRVHlFOYKTY+eF2Ban4cRPCd/zY8pCRgoBpLWSa9n+1ctHpCmsLgobAaYz3htjpbzUzSu+eHLcSvuGx/KncEVVnhldH/zmToRpzer5QbLQ== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_002D_01D6B0FA.C4CBA000" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: HE1PR0701MB2876.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 36fd5d5e-9c57-4f2e-61df-08d87f09866a X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Nov 2020 08:30:15.7388 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ca+LPbhRVXFMyLBg2J//qn8NJlKMAaxWCd6ucaHVITWMvY00+e/TN0X17rkh09szxyfedgbFB2XeV4WV17PHKXwOeGCqN/eo75+N+Hr9FgkSXUUkRS3M+I8touqThi/Q X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR07MB4156 Archived-At: Subject: Re: [tcpPrague] [iccrg] Realistic Queue delay targets X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 08:30:25 -0000 ------=_NextPart_000_002D_01D6B0FA.C4CBA000 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi Rodney I believe part of the answer to your question is found in https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-6.3 . The fact that we begin to see the scheduling jitter does not invalidate L4S as a means to reduce the congestion related jitter. Scheduling jitter exist also in cellular technology. Scheduling jitter is however generally not evidence of congestion. L4S congestion marking should therefore be designed to not trigger on this. The master thesis** the L4S marking threshold was set sufficiently high to avoid this but it should be understood that L4S marking in cellular can be implemented in smarter ways. In the master thesis, the L4S marker only inspected the queue on the RLC layer. There are other causes to delay jitter in cellular, HARQ retransmissions and handover delays are two examples. There is work in progress to reduce all kinds on delay jitter, L4S is a tool to reduce congestion related jitter. In any case, non-congestion related jitter is not an argument against L4S /Ingemar ** https://kth.diva-portal.org/smash/record.jsf?dswid=-6303&pid=diva2%3A1484466 &c=1&searchType=SIMPLE&language=en&query=brunello&af=%5B%5D&aq=%5B%5B%5D%5D& aq2=%5B%5B%5D%5D&aqe=%5B%5D&noOfRows=50&sortOrder=author_sort_asc&sortOrder2 =title_sort_asc&onlyFullText=false&sf=all > -----Original Message----- > From: Rodney W. Grimes > Sent: den 1 november 2020 21:24 > To: Bob Briscoe > Cc: Christian Huitema ; tsvwg IETF list > ; iccrg IRTF list ; TCP Prague List > ; De Schepper, Koen (Koen) > > Subject: [iccrg] Realistic Queue delay targets > > Bob, > > > Christian, > > > > I've changed the subject line given it's no longer appropriate. > > See inline tagged [BB]... > > And I am changing it again... as only addressing a statement that I have great > concerns about. > > See inline ragged [RWG] > > > On 01/11/2020 01:07, Christian Huitema wrote: > ... [ much text deleted ] ... > > > [BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'allowed' > > to send packets over the Internet as ECT(1). Then, yes, an L4S DualQ > > Coupled AQM will classify BBRv2 packets into the L4S queue. This > > should have a very shallow ECN marking threshold (500us - 1ms), so > > even if the flow (whether QUIC or TCP) is flying just under the > > available capacity, bunching of packets means it is unlikely to > > completely avoid ECN marking between probes. If it could avoid ECN > > marking, you are right that it would get a bump of ECN marks during > > the probe. I haven't studied the code, but when it experiences ECN > > marking I believe it switches into an L4S ECN mode for a while, and > > uses ECN rather than delay probing to track available capacity. I > > assume it switches back to BBR's delay probing mode if it gets no ECN > > for a while (e.g. the bottleneck might have moved). But I haven't looked at > BBRv2's ECN behaviour in detail. > > [RWG] I have great concern about this 500uS to 1mS ECN marking threshold > given that I have recently learned the latest WiFi standards actually run with a > packet aggregation time of 4mS thus making it probably impossible to have > such traffic work in such a targeted low latency queue. > > [RWG] Has any consideration been given to what is already deployed on the > Internet as link layer technologies that would preclude the operation of the > L4S low latency queue? > > Regards, > -- > Rod Grimes rgrimes@freebsd.org > ------=_NextPart_000_002D_01D6B0FA.C4CBA000 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIVfDCCAyAw ggIIoAMCAQICAR0wDQYJKoZIhvcNAQEFBQAwOTELMAkGA1UEBhMCRkkxDzANBgNVBAoTBlNvbmVy YTEZMBcGA1UEAxMQU29uZXJhIENsYXNzMiBDQTAeFw0wMTA0MDYwNzI5NDBaFw0yMTA0MDYwNzI5 NDBaMDkxCzAJBgNVBAYTAkZJMQ8wDQYDVQQKEwZTb25lcmExGTAXBgNVBAMTEFNvbmVyYSBDbGFz czIgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCQF0o1ncrwDZbHRPoWN/xIvb1/ gC01O+FvqGepvwMcTYxvMkfVQWikEwTBNQyahEP8XB3/ibPoFxjNkV/7iePqv05dfBsm03V57eaE 41flrSnE9Doo56V7hDZps/1edr2jLZnTkE4jKH0YY/FUOyaddluXQrL/rvBO7N05lU6DBn/nSUDI xQGyVFpmHT38+ek8Cp6BuHDwAYvkI1R8yK74kB4AlnLUVM9hI7zq+50CldG2uXE6aQg/D7ThQseI 9T+YqKe6HOBxce9YV4FQelxrdEYOgwOYw46obvJ2Mm4ng8Jz89wY6LST6nVEawRgIHFXh53zvqCQ Iz2KJOHaIdvDAgMBAAGjMzAxMA8GA1UdEwEB/wQFMAMBAf8wEQYDVR0OBAoECEqgqliE0148MAsG A1UdDwQEAwIBBjANBgkqhkiG9w0BAQUFAAOCAQEAWs6H+RZyFVdLHdmb56ImMOyTZ9/WLdI0r/c4 pc6rFrmrL3w1y6zQD7RMK/yA72uMkV82dvfbsxsZ6vSyEf1hcUS/KLM6Hb+zQ+ifv9wxCHGwnY3W NEcykMZlJPegSnwEc485bxeMcrW9S8h6+HuDwyhOnAnqZz+yZwQbwxTa+OdJJJHQHWr6YTnva+ch dQYH2BK0ISBwQnGB2jyaNr6mWw1qbJofkXv5+e9Cuk5OnswMjZTc2UWcXuxCUGOu9F3EsRLcyjuo Lp0UWgV1t+zXY+K6NbYECJHo2p2c9ma1GKwKplQmNDPSG8HUfxo6jguqMm7b/E8ln9kyx5ZacKzf TDCCBX0wggRloAMCAQICEQCH7S4aKCZKxRmqOuu5DaLLMA0GCSqGSIb3DQEBCwUAMDkxCzAJBgNV BAYTAkZJMQ8wDQYDVQQKEwZTb25lcmExGTAXBgNVBAMTEFNvbmVyYSBDbGFzczIgQ0EwHhcNMTQx MjA1MDgxOTE1WhcNMjEwNDA1MTAyOTAwWjA3MRQwEgYDVQQKDAtUZWxpYVNvbmVyYTEfMB0GA1UE AwwWVGVsaWFTb25lcmEgUm9vdCBDQSB2MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB AMK+6yfwIaPzaSZVfp3FVRaRXP3vIb9TgHot0pGMYzHw7CTww6XScnwQbfQ3t+XmfHnqjLWCi65I tqwA3GV17CpNX8GH9SBlK4GoRz6JI5UwFpB/6FcHSOcZrr9FZ7E3GwYq/t75rH2D+1665I+XZ75L jo1kB1c4VWk0Nj0TSO9P4tNmHqTPGrdeNjPUtAa9GAH9d4RQAEX1jF3oI7x+/jXh7VB7qTCNGdMJ jmhnXb88lxhTuylixcpecsHHltTbLaC0H2kD7OriUPEMPPCs81Mt8Bz17Ww5OXOAFshSsCPN4D7c 3TxHoLs1iuKYaIu+5b9y7tL6pe0S7fyYGKkmdtwoSxAgHNN/Fnct7W+A90m7UwW7XWjH1Mh1Fj+J Wov3F0fUTPHSiXk+TT2YqGHeOh7S+F4D4MHJHIzTjU3TlTazN19jY5szFPAtJmtTfImMMsJu7D0h ADnJoWjiUIMusDor8zagrC/kb2HCUQk5PotTubtn2txTuXZZNp1D5SDgPTJghSJRt8czu90VL6R4 pgd7gUY2BIbdeTXHlSw7sKMXNeVzH7RcWe/a6hBle3rQf5+ztCo3O3CLm1u5K7fsslESl1MpWtTw EhDcTwK7EpIvYtQ/aUN8Ddb8WHUBiJ1YFkveupD/RwGJBmr2X7KQarMCpgKIv7NHfirZ1fpoeDVN AgMBAAGjggGAMIIBfDBOBggrBgEFBQcBAQRCMEAwPgYIKwYBBQUHMAKGMmh0dHA6Ly9jYS50cnVz dC50ZWxpYXNvbmVyYS5jb20vc29uZXJhY2xhc3MyY2EuY2VyMA8GA1UdEwEB/wQFMAMBAf8wGQYD VR0gBBIwEDAOBgwrBgEEAYIPAgMBAQIwDgYDVR0PAQH/BAQDAgEGMB0GA1UdDgQWBBTwj1k4ALP1 j5qWDNXr+nuqF+gTEjCBuQYDVR0fBIGxMIGuMG+gbaBrhmlsZGFwOi8vY3JsLTEudHJ1c3QudGVs aWFzb25lcmEuY29tL2NuPVNvbmVyYSUyMENsYXNzMiUyMENBLG89U29uZXJhLGM9Rkk/Y2VydGlm aWNhdGVyZXZvY2F0aW9ubGlzdDtiaW5hcnkwO6A5oDeGNWh0dHA6Ly9jcmwtMi50cnVzdC50ZWxp YXNvbmVyYS5jb20vc29uZXJhY2xhc3MyY2EuY3JsMBMGA1UdIwQMMAqACEqgqliE0148MA0GCSqG SIb3DQEBCwUAA4IBAQAQ1elFTM6fGkQ/aRKdkUZicO3Cb9uzBJOpOtFctw+1El0/17lsjoVvJkZB D3KnUobnrriFdAa+7FAN55KLmZeB/3Y2bG0bB4toSyaVHjOQnQY9M0dv8U852w0Q7GwchKfebLUI bh9TMt2hI3Xc6j4knFTBUo7C1WAfO51K4bn1irmX6/Ej2VTgiOFsvOAny28W6enFSEQpSHw60VhN fSttSqTOxyrRR/7kW7Y8yb/3DZDZ/dH6ZCfx/y+BNIv2NuSd85M9HXUzplXXohti4Ql/qeaMn6by Ius6XlMWZZfkdVRvTuk2PkeC7UmAJ2+/DUWOPpawaytMXVfF4Hvxk34NMIIGDTCCA/WgAwIBAgIQ Us79UtuT8yEU6XIq+TroQjANBgkqhkiG9w0BAQsFADBHMQswCQYDVQQGEwJTRTERMA8GA1UECgwI RXJpY3Nzb24xJTAjBgNVBAMMHEVyaWNzc29uIE5MIEluZGl2aWR1YWwgQ0EgdjMwHhcNMTcxMTI3 MTAxMzQ4WhcNMjAxMTI3MTAxMzQ3WjB0MREwDwYDVQQKDAhFcmljc3NvbjEcMBoGA1UEAwwTSW5n ZW1hciBKb2hhbnNzb24gUzEvMC0GCSqGSIb3DQEJARYgaW5nZW1hci5zLmpvaGFuc3NvbkBlcmlj c3Nvbi5jb20xEDAOBgNVBAUTB0VQTElKT0gwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB AQCvxK7X3xRTAzY+JaEVXSFcykcRDnb9xJCcMYXRb7WCh0HpVZ7PThGgp0ZUNQQJ9ypm7YcIcwRD kYdD4fgxGLU1bIUC4GcH7rKiuf93b+Ec3jsrkSEkCsXkO7ROjlS9+7y7H/qC8bB8D4CTQNs15o3J 0njOaYngdmLGb8bwYV47hv3sAiyqoY0aZeGZyk2a/D9Kc28b+towJxQrGnekMxynyQQesKF7mFaA gtlEa5bNbLEfIki5tZWceSZEd5xXxuib7vv0Rb/Gd+Q+jyvobAEyd5RDB6i9HAA/FFPZIJTgM+k9 7yIFPPhpLjjcjeowqH9PlEaody2drdP67tBDnNQTAgMBAAGjggHGMIIBwjBIBgNVHR8EQTA/MD2g O6A5hjdodHRwOi8vY3JsLnRydXN0LnRlbGlhLmNvbS9lcmljc3Nvbm5saW5kaXZpZHVhbGNhdjMu Y3JsMIGCBggrBgEFBQcBAQR2MHQwKAYIKwYBBQUHMAGGHGh0dHA6Ly9vY3NwMi50cnVzdC50ZWxp YS5jb20wSAYIKwYBBQUHMAKGPGh0dHA6Ly9jYS50cnVzdC50ZWxpYXNvbmVyYS5jb20vZXJpY3Nz b25ubGluZGl2aWR1YWxjYXYzLmNlcjArBgNVHREEJDAigSBpbmdlbWFyLnMuam9oYW5zc29uQGVy aWNzc29uLmNvbTBVBgNVHSAETjBMMEoGDCsGAQQBgg8CAwEBEjA6MDgGCCsGAQUFBwIBFixodHRw czovL3JlcG9zaXRvcnkudHJ1c3QudGVsaWFzb25lcmEuY29tL0NQUzAdBgNVHSUEFjAUBggrBgEF BQcDBAYIKwYBBQUHAwIwHQYDVR0OBBYEFH5/jYsQz4BrrLa7pwQSkqtzL4EOMB8GA1UdIwQYMBaA FBx7GZ6XnHasID3Y3OORauPbLaZTMA4GA1UdDwEB/wQEAwIFoDANBgkqhkiG9w0BAQsFAAOCAgEA uwe9UtkD6Q6/dW/uQOCxTOaGCX76lOy+a0Fqlyh+ZmlS8qWj8YTCtD866gJOXMzYUMVFrWoeggvP 9j/MBmkDDuuJMgCkN3TEuAu4AF9ekcbm/yjWOUFx3kje0SYL+GNwN50He1tuOlmHrsHo7S89kOL6 Pq8+QMRSESzEqva7nC6sJdYxXGyWAagM1ZWFnJTtFCzsW6Tm6P6rpit4yCPYvPSgNPhPRj7UbHon vU4uwGBiO15aKLwg0jfoJn6wF/CZqtGeza+ucmcuEKA+y/IRnQ2pA5+p3QnX41eypQ7OJfCG5U6k fXjxyu7PGMompQQutZk75tw76wvE8kWj5VpDNf0dRVhq/PQugzqiRjrF5tFSQbOnq1GGQ4X3PhJ8 TBaoC4otM95ZguVQrM8wdYMSxeinbJY4T91eVXYltNgtUd84zbmL/1JgVa7PN3wASXPhguc0WZu3 OWNn31yKlqL/QmgDEoOs8UsNqYU7ngsg620TE9tta4S96LXRhXhBiQCDKR16ZhEITbaJPT7KgoMO 30YCHQa0K7ah5UxC9S6+E6cPK+uKyHvzXzjNdILMh6DXQFMblld6ax+to7tQQEkv1Xr4tjCN2H/z elNX2LGB1lO2qra3MfgaENY5SMk6sIi4Y4QDrVEBnrZeKstS9Qd3TCvc+fHu6bt3wU9Tk3K0dOAw ggbCMIIEqqADAgECAhBTuH6D4ZyZKJOwm0kc7LjrMA0GCSqGSIb3DQEBCwUAMDcxFDASBgNVBAoM C1RlbGlhU29uZXJhMR8wHQYDVQQDDBZUZWxpYVNvbmVyYSBSb290IENBIHYxMB4XDTE1MTAyNzEy MTY0NloXDTI1MTAyNzEyMTY0NlowRzELMAkGA1UEBhMCU0UxETAPBgNVBAoMCEVyaWNzc29uMSUw IwYDVQQDDBxFcmljc3NvbiBOTCBJbmRpdmlkdWFsIENBIHYzMIICIjANBgkqhkiG9w0BAQEFAAOC Ag8AMIICCgKCAgEA7PLfAAC4UPKnu9hUt8aT9+PBqjvUw0Y0tLPOXkO2NC0y2XZks9nJfpWKrNM3 0k5vu5norG4ZKlF5C+3xc6HuIiGQof1bmFGluNOwmZQwl3rOJ+E6k0rqJJTerjj4WOxAvWVW1yC5 S4Ubppk3Q3cYVVuC3qNGsBIXy3/fDL1sc8Ah8zI/JumDpjY8fn/U3CRN6mgNKYrr0sZX6VXYgrpT 05ZrJldkUgUgMKgbIWWEXEASA36pnb5GqD/RMzSgIe8o7YQtIaYB2cmTCLNHjaOL9j1JhNK4bvmb NJ7o58IZYzwNv/G/L/bRosQ9c27U+86DNjrdZnpyaRaeMyVUn3SlYLaFqoObdh/xNF2NS8CXs/PV tO57HBKHMgZqQvsyQJisSocxFqiMj9VK2WhCBbvoTvrNDZvLDlDGuE5RuKwFIpHOVOU5lCBgUUBs bpWIXwM6kmH/KC1DC5MtQzmvXkbt7KdBXUAxM0JZxf4dS+ACtTDpF9b0vny4DrwaOS0VNXyz1GUO xSqw1wup5dpXbxLZYx1rLRgZqr9uWhLwAPsq66ZQof5GL0gY72Ym8/Tm28MeMqku+/zRzdYsmclT 9rOdgdgS3b6OMoc5Op0ZPEv/Mx2lFJAVK674ozw2hiuRTVUmoqBr5AuyCoqCEyn32C7U/V7oqyqx 5Yd1c5GsxuOqQFcCAwEAAaOCAbgwggG0MIGKBggrBgEFBQcBAQR+MHwwLQYIKwYBBQUHMAGGIWh0 dHA6Ly9vY3NwLnRydXN0LnRlbGlhc29uZXJhLmNvbTBLBggrBgEFBQcwAoY/aHR0cDovL3JlcG9z aXRvcnkudHJ1c3QudGVsaWFzb25lcmEuY29tL3RlbGlhc29uZXJhcm9vdGNhdjEuY2VyMBIGA1Ud EwEB/wQIMAYBAf8CAQAwVQYDVR0gBE4wTDBKBgwrBgEEAYIPAgMBAQIwOjA4BggrBgEFBQcCARYs aHR0cHM6Ly9yZXBvc2l0b3J5LnRydXN0LnRlbGlhc29uZXJhLmNvbS9DUFMwSwYDVR0fBEQwQjBA oD6gPIY6aHR0cDovL2NybC0zLnRydXN0LnRlbGlhc29uZXJhLmNvbS90ZWxpYXNvbmVyYXJvb3Rj YXYxLmNybDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwQwDgYDVR0PAQH/BAQDAgEGMB0G A1UdDgQWBBQcexmel5x2rCA92NzjkWrj2y2mUzAfBgNVHSMEGDAWgBTwj1k4ALP1j5qWDNXr+nuq F+gTEjANBgkqhkiG9w0BAQsFAAOCAgEAUFhr8dWMO7Quq1dDyIynw8sWmpyF/jWSxBjpHUCyhlto FS7Q1CUBD0bOULWmYjmzRwme5pkjTFXpOJZLf9Han1SBbrVcP0JMhRsAvfWZjcF0l/c/jqDMqBAR xr8OUWOr0ZWa49Lir3QEs2C+CjGge5tzcLqzQ5pjWxudrLkSGe+sAThDnXUWXGYk8udGZAamJ55d rdw96AV9jWQkMrLIVHKkXVG5Etdx0wiAoTLk1fVtLcz11DiaCZSZVPZ3fdSIpIRhDqz8H4sVprPg vLBdK/ajdbiRsehCzzohay3zbXDDTDGwKkR8KUi8Xt8HDZCRsb/U/C7MC4tVK0SEPOQCo6swZy0r I0RoGzICfsSrZ4JrxANeeSZqCn1A+w0Wz+iqdeP2PVxW0f1rg4/OG2DSl3uB3Q3NT/lDGJtepti+ i5CCKEZcdAOZoviu43sLhqsxSpGjzZidESwovuHeP+O2bNwwtz1DTsXThBB3+JJHVjmkiLo900GI Tb/i7IBdLoo4gZms9s1BQ2tm3CJCmpA2XwBTOB6B8/CtgWUWhyloXd3Wbmv7ZUoqqJFBV9g8Zh5m dZ+RzPTomgCFz/2aNsddI/2G9ZjN4tG6hmocZR2M5f0MhBv3bo6d5XsLlYwiNJjw5GRqYb8cqqeC aPKkveBJzqgb8ToH7WLoOzmPRCmPlpAxggMCMIIC/gIBATBbMEcxCzAJBgNVBAYTAlNFMREwDwYD VQQKDAhFcmljc3NvbjElMCMGA1UEAwwcRXJpY3Nzb24gTkwgSW5kaXZpZHVhbCBDQSB2MwIQUs79 UtuT8yEU6XIq+TroQjAJBgUrDgMCGgUAoIIBfDAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwG CSqGSIb3DQEJBTEPFw0yMDExMDIwODMwMTRaMCMGCSqGSIb3DQEJBDEWBBTCW6/k9Mrpr/XDKX2w Af3M7jUVhzBDBgkqhkiG9w0BCQ8xNjA0MAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggq hkiG9w0DAgIBQDAHBgUrDgMCGjBqBgkrBgEEAYI3EAQxXTBbMEcxCzAJBgNVBAYTAlNFMREwDwYD VQQKDAhFcmljc3NvbjElMCMGA1UEAwwcRXJpY3Nzb24gTkwgSW5kaXZpZHVhbCBDQSB2MwIQUs79 UtuT8yEU6XIq+TroQjBsBgsqhkiG9w0BCRACCzFdoFswRzELMAkGA1UEBhMCU0UxETAPBgNVBAoM CEVyaWNzc29uMSUwIwYDVQQDDBxFcmljc3NvbiBOTCBJbmRpdmlkdWFsIENBIHYzAhBSzv1S25Pz IRTpcir5OuhCMA0GCSqGSIb3DQEBAQUABIIBABbNWd7z9tgGi2WvVKZ8J4HD3gBIKs3US0SfQaUE 6mfHjIOKUP1xyacncx3yo6KsCdqdnS0d2f5fpTOIekQLXPBCIRHY0Ueq2Q2wADhzsTRsv/heeGNe kJHd6YVTE1J1qyqTxE9GligbE3tx2PotCaUrDncX3nQLoDTPc4tWNUtMhFY/lEwP3SaI5z7aa05o oCd0yVkS7oHvLsDKuNV5wfA+z3Ue6bmQ2gBo5D5//UfEOmqLFG4H1vpEsS/WImTXaY0UnYNJs2S2 7mhKbCHz+PEuw3eY8BV5k3Yt1B0IsFoxpn/y1HAfenhX82cWxHfDhZdJlqeqQQSuVv66RcUoY1cA AAAAAAA= ------=_NextPart_000_002D_01D6B0FA.C4CBA000-- From nobody Mon Nov 2 00:47:42 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 693D13A0D33; Mon, 2 Nov 2020 00:47:29 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CsfYJMvNswhT; Mon, 2 Nov 2020 00:47:27 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DF91C3A0CFA; Mon, 2 Nov 2020 00:47:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1604306796; bh=Hn09OegG8Vr6OUWgBzDRanhigKcOZPef+PdWbYWBR+0=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=j6v8WXLIoj7FYsuOGmQroEU+bdqsfp6xPMLMpY7w4jnbOzergVNEg/NLtzCgE623W qNBCAmRv7TybJAdC+/IMKh64wIzX4cHK2jmoyFITA7ewRAHIZeRArV1NrN4buVzK0F 3IcYvglkcOLcSNrugicOR82c/bPSUve2gdQiL3Vk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [134.76.241.253] ([134.76.241.253]) by web-mail.gmx.net (3c-app-gmx-bap61.server.lan [172.19.172.131]) (via HTTP); Mon, 2 Nov 2020 09:46:36 +0100 MIME-Version: 1.0 Message-ID: From: Sebastian Moeller To: Ingemar Johansson S Cc: "Rodney W. Grimes" , Bob Briscoe , tsvwg IETF list , iccrg IRTF list , Christian Huitema , TCP Prague List , "De Schepper, Koen (Koen)" Content-Type: text/plain; charset=UTF-8 Date: Mon, 2 Nov 2020 09:46:36 +0100 Importance: normal Sensitivity: Normal In-Reply-To: References: <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> <202011012023.0A1KNkue011764@gndrsh.dnsmgr.net> X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:wopUGQ2PPi6PDX/JCZrd8YGKkeR4ldvwKTuFBYmhysrbDjFLx50KYE7OjWKVFX/Us1MAZ az/e+NsE2yNYWESxgb9yZhPcVPEE273nUy1YQ+RTEsOI8+9SkydAkCMXe3CjV5Ab1PXcaasUuCGD ihY8bM+UXoSYdt7w+qq3eLr2AaJdlCnt/efac6I8cv7/2SqWEftCFWFDS8bRxMzEDNcXvVYP2f80 cJIfyHLExPsS64OghsjvZcokxmTQYocZkcFT9wSE5wy+0lx5QG6jGN6kK+/JO6ZUwTtrKCDXrwBy jc= X-UI-Out-Filterresults: notjunk:1;V03:K0:TU7WgeaHMuc=:HxEtn+AjktqqDVQd5xb48z GeAHTpgoT6BqtxsNt0XFeuX5cRq6G1BMN6UHzZHgzBAZRovlmn51v4W/J8EZ2cJEf8VGHaFzX DZRvJbycWcxHo3UQofTxfPHjt5g3sKz9JEwxH/uHG8ezs9knPXyU/oIaNEGsrVVaubN2QBHss ktZLNMYhfGlI6TsXWuLtM4VtDQrS6JLgFhT/r3Adymqgz+7z6/ZLlCrMoZNn6Yqp0W+ahzODn z67HtTgMT1w/m7mN4KVojxjuQ44SaMKEdoC/qy3KRYUWm4/9HoaHBdZfn6MSOmAATYwT+6dP3 wLgEfk/T6l0pFlmLYwacDrdvBXbF8S1lgwqWKT8JHA4z2itQvgnw2SDUGkPWwgcDi3z4NZ+hh G7F/KMEUeOZfQLzCMPEBHpxEqjcjy+O3svrkEAxd0XEwsvU93r4ILkpJkoAhg4MJvpsvm5TEj ykC7eI2yYBFXAYwhBYFblTdy2psFptTHA+YyPTs0BA0M25TeyUIK7HQ550l4NBMNVAPdPGcAK JQhXmjGpwZy0FnPOHkzX25WDNXE57Fa3KOxxJXnHo4FGxmw1095+a+XjM7jiUsQXmOjRSy6zK y3l4230bTwbUY= Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [tcpPrague] [tsvwg] [iccrg] Realistic Queue delay targets X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 08:47:37 -0000 Hi Ingemar, more below in-line, prefixed [SM]. > Gesendet: Montag, 02. November 2020 um 09:30 Uhr > Von: "Ingemar Johansson S" > An: "Rodney W. Grimes" , "Bob Briscoe" > Cc: "tsvwg IETF list" , "iccrg IRTF list" , "Christian Huitema" , "TCP Prague List" , "De Schepper, Koen (Koen)" > Betreff: Re: [tsvwg] [iccrg] Realistic Queue delay targets > > Hi Rodney > > I believe part of the answer to your question is found in > https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-6.3 . T= he > fact that we begin to see the scheduling jitter does not invalidate L4S = as a > means to reduce the congestion related jitter. [SM] As so often that section does contain some theoretical observa= tions, but fails to empirically show which level of burstyness L4S can dea= l with gracefully and what the consequences are (on the bursty flow itself= , other flows in the shallow-queue and and flows in the deep queue). That = would be meaningful responses to Rodney's concern. As is that section basi= cally just claims without supporting data, that even in the face of bursty= flows, L4S should still be tried, but that must be compared to a FiFO, be= cause compared to say fq_codel with a 5ms default target, traffic with > 5= ms jitter will probably look much better after fq_codel than after L4S (L4= S will conserve the burstyness much more, while fq_codel will smooth it ou= t a bit). I wonder why such verbose but essentially non-actionable (almost u= seless) sections keep getting added to already quite long and verbose int= ernet drafts in the first place? Regards Sebastian > > Scheduling jitter exist also in cellular technology. Scheduling jitter i= s > however generally not evidence of congestion. L4S congestion marking sho= uld > therefore be designed to not trigger on this. The master thesis** the L4= S > marking threshold was set sufficiently high to avoid this but it should = be > understood that L4S marking in cellular can be implemented in smarter wa= ys. > In the master thesis, the L4S marker only inspected the queue on the RLC > layer. > There are other causes to delay jitter in cellular, HARQ retransmissions= and > handover delays are two examples. There is work in progress to reduce al= l > kinds on delay jitter, L4S is a tool to reduce congestion related jitter= . > In any case, non-congestion related jitter is not an argument against L4= S > > /Ingemar > > ** > https://kth.diva-portal.org/smash/record.jsf?dswid=3D-6303&pid=3Ddiva2%3= A1484466 > &c=3D1&searchType=3DSIMPLE&language=3Den&query=3Dbrunello&af=3D%5B%5D&aq= =3D%5B%5B%5D%5D& > aq2=3D%5B%5B%5D%5D&aqe=3D%5B%5D&noOfRows=3D50&sortOrder=3Dauthor_sort_as= c&sortOrder2 > =3Dtitle_sort_asc&onlyFullText=3Dfalse&sf=3Dall > > > -----Original Message----- > > From: Rodney W. Grimes > > Sent: den 1 november 2020 21:24 > > To: Bob Briscoe > > Cc: Christian Huitema ; tsvwg IETF list > > ; iccrg IRTF list ; TCP Prague List > > ; De Schepper, Koen (Koen) > > > > Subject: [iccrg] Realistic Queue delay targets > > > > Bob, > > > > > Christian, > > > > > > I've changed the subject line given it's no longer appropriate. > > > See inline tagged [BB]... > > > > And I am changing it again... as only addressing a statement that I ha= ve > great > > concerns about. > > > > See inline ragged [RWG] > > > > > On 01/11/2020 01:07, Christian Huitema wrote: > > ... [ much text deleted ] ... > > > > > [BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'allowe= d' > > > to send packets over the Internet as ECT(1). Then, yes, an L4S DualQ > > > Coupled AQM will classify BBRv2 packets into the L4S queue. This > > > should have a very shallow ECN marking threshold (500us - 1ms), so > > > even if the flow (whether QUIC or TCP) is flying just under the > > > available capacity, bunching of packets means it is unlikely to > > > completely avoid ECN marking between probes. If it could avoid ECN > > > marking, you are right that it would get a bump of ECN marks during > > > the probe. I haven't studied the code, but when it experiences ECN > > > marking I believe it switches into an L4S ECN mode for a while, and > > > uses ECN rather than delay probing to track available capacity. I > > > assume it switches back to BBR's delay probing mode if it gets no EC= N > > > for a while (e.g. the bottleneck might have moved). But I haven't lo= oked > at > > BBRv2's ECN behaviour in detail. > > > > [RWG] I have great concern about this 500uS to 1mS ECN marking thresh= old > > given that I have recently learned the latest WiFi standards actually = run > with a > > packet aggregation time of 4mS thus making it probably impossible to h= ave > > such traffic work in such a targeted low latency queue. > > > > [RWG] Has any consideration been given to what is already deployed on= the > > Internet as link layer technologies that would preclude the operation = of > the > > L4S low latency queue? > > > > Regards, > > -- > > Rod Grimes > rgrimes@freebsd.org > > > > From nobody Mon Nov 2 01:51:13 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CDD7A3A0D6C; Mon, 2 Nov 2020 01:51:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.102 X-Spam-Level: X-Spam-Status: No, score=-2.102 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6mrXXlz-ltbr; Mon, 2 Nov 2020 01:51:05 -0800 (PST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50047.outbound.protection.outlook.com [40.107.5.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 73C9B3A0DE8; Mon, 2 Nov 2020 01:51:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=er7Oik0WlxTUbQ71y5G32Bfg5brp747gJFIW+BF/sO7mHSm2lZvgoArux8ae777wBVdIo3yY+dcu1VC+ICvlrtwWk2VoBadiy7X5Y+gn6jBodRSs6hIFVvRrU0MrAaPJgXZA1dKJSLjDb/wTPdGLPLPmnq3GWDfYA5R2nMBkFd+DlCkYRRH8Oj47oPDYmDF/Pjnm1Yd5pkMy6W3hQmuARrku+Ygp5gmJqF263fCyi/DdmwDyWz73al/U/pnrnrNgBXLhZJPY/tQwiZPdzaQTOmLhAX2Hj187VUieA7Xss7LDJzORZEJfPCkfdpwnFOcsL2cRQZ0Ta0WoLRFimQ2vpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sYVU53BrtTrIHsaYdlM3hLNx1/0ciCuMTnBvbNkrwo0=; b=RGxKiRwIUhL7VSb8YdJtAwhFkjjGDR82iLaORn9/h69XFpV7mFwGuaAAE3d71qZA79tXZYrsHFXXBoGOzA3UOm9JCo/hhHM368vuH2do9Eob4Vnokh7kMr2YxwtLy+vdecLzyvYGKUKlGpIzrBKe6NzXL1CPDkbXoIINtp1r4keCizdJRL8cdgiR+HyEhOvExk0Yt/cGaahbj60OpPttl5uLPeOm78tMEkhwGKUv4eBKRxn0GKQkA77PZw/IlR+6VOxQ0GBgaY7swsHGL+vGjb1mhr9O6DWdfXWfjs3FOMAoRlOlNcwKTcd0Sf9nlVqkAxhzexNc0qAp4bwImjWXFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sYVU53BrtTrIHsaYdlM3hLNx1/0ciCuMTnBvbNkrwo0=; b=Pv03hTeizC/N1vxLw3H2x7h9i41/gQFDs+3yM5Ikby6/+sGOonTbJb//0+08K0QFcVthCsryREegJyrZ5rp3biDlko2xVwXBNU5PF2MGEDg8UAo6qOEkm05dZK5/tR0Z5/HagKOV7gDsMk63FNZIO7iqtOFOeei/LUSqlSqBNQE= Received: from HE1PR0701MB2876.eurprd07.prod.outlook.com (2603:10a6:3:56::8) by HE1PR0701MB2987.eurprd07.prod.outlook.com (2603:10a6:3:51::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.10; Mon, 2 Nov 2020 09:50:55 +0000 Received: from HE1PR0701MB2876.eurprd07.prod.outlook.com ([fe80::a4a1:d6b5:1254:923a]) by HE1PR0701MB2876.eurprd07.prod.outlook.com ([fe80::a4a1:d6b5:1254:923a%11]) with mapi id 15.20.3541.011; Mon, 2 Nov 2020 09:50:55 +0000 From: Ingemar Johansson S To: Sebastian Moeller CC: "Rodney W. Grimes" , Bob Briscoe , tsvwg IETF list , iccrg IRTF list , Christian Huitema , TCP Prague List , "De Schepper, Koen (Koen)" , Ingemar Johansson S Thread-Topic: Re: [tsvwg] [iccrg] Realistic Queue delay targets Thread-Index: AQHWsPJjPxAUbDfzuUaS/obDyN9yVam0h3MAgAAOk0A= Date: Mon, 2 Nov 2020 09:50:55 +0000 Message-ID: References: <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> <202011012023.0A1KNkue011764@gndrsh.dnsmgr.net> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: authentication-results: gmx.de; dkim=none (message not signed) header.d=none;gmx.de; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [83.227.122.88] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1d0acf14-4fad-4435-6f58-08d87f14cb09 x-ms-traffictypediagnostic: HE1PR0701MB2987: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: yB1JaYNs6sad+x6/7u2dtVDuL/AlAOoEh51U7OmmeDhJ16tUCcsrpQkRJin+LUsFQzicmUKTFyU2LvNiDF8bXGSS0SfxIha/V54R3T5f0HHpLaEn1FltM0UcW3uOVDJklMPu+PUEOpWM346pegFfCf8NFze2vob7QyPOE1G8VR55nWCic5f0QD+uJOWv6KkOsOpQ3Gupqy696k7mpn25mlR/REiQOvdaefU5JqY+YMFB57W2mvHdmsXDqMtiUAvZWgdj37bG07Pm/KJhUFWdMhDA/FNCuNHQFThdm1nHDjLaMHlRozsJrlUkjfNOGrB2/vA+ubDZ+YVkvVrRIBPsKdoCiyBIQKASt/GseJTq5DZo70mldbRXI0CbscDxFrDaPuyMG03XMnjFKZw9+Fc2iQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:HE1PR0701MB2876.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(366004)(346002)(136003)(396003)(66946007)(966005)(66446008)(66556008)(66616009)(55016002)(64756008)(66476007)(2906002)(52536014)(6916009)(316002)(9686003)(76116006)(478600001)(5660300002)(54906003)(99936003)(8936002)(66574015)(83380400001)(4326008)(71200400001)(7696005)(33656002)(86362001)(26005)(186003)(53546011)(6506007)(107886003)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: YGSr8G6X0sI0LUpoyUNKHNJA66mmlOnqIl5PJQSf5EyPtN5larHoDtwNOpD396ThfoJTcQRTdQIrnQMSap6rlbvKOs4WgVmcqGtmUjpWZAcYv3csAwRviVnPzELyKEpUiteLeD8k9aAx8hIhlddOM2TeVvB7neqvb9dcUPkTW+6COu8LRC/wux7WN3W09nYsD3+8RPQDUXNyoga+CqBOul/fygcsMARDFpGBBYztGPz7utboBF6Z3LSP04UKzvWcmVf+U6pE6fMk6Apfu5QrkH2m3qbhZcP2lk2Bv7f+GI4qa4o4355VJ7OG2/HzDj5cTdBng6Rd9lmvWttW7ZXRPGPiWjk4aKyc95PcRwSM9PZVyxkU6obISmp45WYXLr1mYc2/PUKHmx8pUI9lZ4EnHGxCpeo8fHmYo1ijuRFa0DqltDZmXRK+mAUC6ljiYUloN+wiEoFDLHFfsWFYQN1XkvLLsUv4qkTO7Kt2FLD7cLlr7Zg1PBBPCWvteo40uYGuwJMQOEHi3s80khhwkA+Ue/Xd5ho7gtScjoasOJTGaT0WjfdJNdJPtB6PKsJtY1nrR85wykJeDSOifiUJmzAAF2iPilYjMH90bjmewV5e6CddMzTAZnFjdR9H8SBTpjrVPBbU8NYnNmHZsJ4wtOHjYQ== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_00DB_01D6B106.09411A70" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: HE1PR0701MB2876.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1d0acf14-4fad-4435-6f58-08d87f14cb09 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Nov 2020 09:50:55.3722 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: LpNwOgLmCyTacGdbJOzatL8Qdw8/yChFHbjyQQwQXbP42ABnpaP6LYbgZdyPn25AIvh7BGcKISOwO2iVKy0PUZJXxvHzQeEQBKEuZlq0AuUnTbbbdtucd9LnSFjC7Qpj X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0701MB2987 Archived-At: Subject: Re: [tcpPrague] [tsvwg] [iccrg] Realistic Queue delay targets X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 09:51:09 -0000 ------=_NextPart_000_00DB_01D6B106.09411A70 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Hi I don't anymore know where all these discussions are heading. Seems like we always end up in infinite discussions that lead nowhere!. Network implementers will implement whatever queue management they find useful, be it dualQ, fq-codel or whatever. Yes, there are ways to deal with jitter, some are implemented, some are in the pipe, some are on the drawing board and as I mentioned several times: It is not very fruitful to take todays internet jitter as a litmus test on how good L4S will behave in the future. As examples, 5G has a shorter HARQ retransmission loop than 4G, this reduces jitter, there is also work to reduce handover delay, shorter TTIs etc. It is therefore quite meaningless to produce data on how well L4S will fare in the precense of jitter. Leave that to the experiments. /Ingemar > -----Original Message----- > From: Sebastian Moeller > Sent: den 2 november 2020 09:47 > To: Ingemar Johansson S > Cc: Rodney W. Grimes ; Bob Briscoe > ; tsvwg IETF list ; iccrg IRTF list > ; Christian Huitema ; TCP Prague List > ; De Schepper, Koen (Koen) > > Subject: Aw: Re: [tsvwg] [iccrg] Realistic Queue delay targets > > Hi Ingemar, > > more below in-line, prefixed [SM]. > > > Gesendet: Montag, 02. November 2020 um 09:30 Uhr > > Von: "Ingemar Johansson S" > > > > An: "Rodney W. Grimes" , "Bob Briscoe" > > > > Cc: "tsvwg IETF list" , "iccrg IRTF list" > > , "Christian Huitema" , "TCP > > Prague List" , "De Schepper, Koen (Koen)" > > > > Betreff: Re: [tsvwg] [iccrg] Realistic Queue delay targets > > > > Hi Rodney > > > > I believe part of the answer to your question is found in > > https://tools.ietf.org/html/draft-ietf-tsvwg-l4s-arch-07#section-6.3 . > > The fact that we begin to see the scheduling jitter does not > > invalidate L4S as a means to reduce the congestion related jitter. > > [SM] As so often that section does contain some theoretical > observations, > but fails to empirically show which level of burstyness L4S can deal with > gracefully and what the consequences are (on the bursty flow itself, other > flows in the shallow-queue and and flows in the deep queue). That would be > meaningful responses to Rodney's concern. As is that section basically just > claims without supporting data, that even in the face of bursty flows, L4S > should still be tried, but that must be compared to a FiFO, because compared > to say fq_codel with a 5ms default target, traffic with > 5ms jitter will > probably > look much better after fq_codel than after L4S (L4S will conserve the > burstyness much more, while fq_codel will smooth it out a bit). > I wonder why such verbose but essentially non-actionable (almost > useless) sections keep getting added to already quite long and verbose > internet drafts in the first place? > > Regards > Sebastian > > > > > Scheduling jitter exist also in cellular technology. Scheduling jitter > > is however generally not evidence of congestion. L4S congestion > > marking should therefore be designed to not trigger on this. The > > master thesis** the L4S marking threshold was set sufficiently high to > > avoid this but it should be understood that L4S marking in cellular can be > implemented in smarter ways. > > In the master thesis, the L4S marker only inspected the queue on the > > RLC layer. > > There are other causes to delay jitter in cellular, HARQ > > retransmissions and handover delays are two examples. There is work in > > progress to reduce all kinds on delay jitter, L4S is a tool to reduce > congestion related jitter. > > In any case, non-congestion related jitter is not an argument against > > L4S > > > > /Ingemar > > > > ** > > https://kth.diva-portal.org/smash/record.jsf?dswid=-6303&pid=diva2%3A1 > > 484466 > > > &c=1&searchType=SIMPLE&language=en&query=brunello&af=%5B%5D&aq= > %5B%5B% > > 5D%5D& > > > aq2=%5B%5B%5D%5D&aqe=%5B%5D&noOfRows=50&sortOrder=author_sor > t_asc&sort > > Order2 =title_sort_asc&onlyFullText=false&sf=all > > > > > -----Original Message----- > > > From: Rodney W. Grimes > > > Sent: den 1 november 2020 21:24 > > > To: Bob Briscoe > > > Cc: Christian Huitema ; tsvwg IETF list > > > ; iccrg IRTF list ; TCP Prague List > > > ; De Schepper, Koen (Koen) > > > > > > Subject: [iccrg] Realistic Queue delay targets > > > > > > Bob, > > > > > > > Christian, > > > > > > > > I've changed the subject line given it's no longer appropriate. > > > > See inline tagged [BB]... > > > > > > And I am changing it again... as only addressing a statement that I > > > have > > great > > > concerns about. > > > > > > See inline ragged [RWG] > > > > > > > On 01/11/2020 01:07, Christian Huitema wrote: > > > ... [ much text deleted ] ... > > > > > > > [BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'allowed' > > > > to send packets over the Internet as ECT(1). Then, yes, an L4S > > > > DualQ Coupled AQM will classify BBRv2 packets into the L4S queue. > > > > This should have a very shallow ECN marking threshold (500us - > > > > 1ms), so even if the flow (whether QUIC or TCP) is flying just > > > > under the available capacity, bunching of packets means it is > > > > unlikely to completely avoid ECN marking between probes. If it > > > > could avoid ECN marking, you are right that it would get a bump of > > > > ECN marks during the probe. I haven't studied the code, but when > > > > it experiences ECN marking I believe it switches into an L4S ECN > > > > mode for a while, and uses ECN rather than delay probing to track > > > > available capacity. I assume it switches back to BBR's delay > > > > probing mode if it gets no ECN for a while (e.g. the bottleneck > > > > might have moved). But I haven't looked > > at > > > BBRv2's ECN behaviour in detail. > > > > > > [RWG] I have great concern about this 500uS to 1mS ECN marking > > > threshold given that I have recently learned the latest WiFi > > > standards actually run > > with a > > > packet aggregation time of 4mS thus making it probably impossible to > > > have such traffic work in such a targeted low latency queue. > > > > > > [RWG] Has any consideration been given to what is already deployed > > > on the Internet as link layer technologies that would preclude the > > > operation of > > the > > > L4S low latency queue? > > > > > > Regards, > > > -- > > > Rod Grimes > > rgrimes@freebsd.org > > > > > > > ------=_NextPart_000_00DB_01D6B106.09411A70 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIVfDCCAyAw ggIIoAMCAQICAR0wDQYJKoZIhvcNAQEFBQAwOTELMAkGA1UEBhMCRkkxDzANBgNVBAoTBlNvbmVy YTEZMBcGA1UEAxMQU29uZXJhIENsYXNzMiBDQTAeFw0wMTA0MDYwNzI5NDBaFw0yMTA0MDYwNzI5 NDBaMDkxCzAJBgNVBAYTAkZJMQ8wDQYDVQQKEwZTb25lcmExGTAXBgNVBAMTEFNvbmVyYSBDbGFz czIgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCQF0o1ncrwDZbHRPoWN/xIvb1/ gC01O+FvqGepvwMcTYxvMkfVQWikEwTBNQyahEP8XB3/ibPoFxjNkV/7iePqv05dfBsm03V57eaE 41flrSnE9Doo56V7hDZps/1edr2jLZnTkE4jKH0YY/FUOyaddluXQrL/rvBO7N05lU6DBn/nSUDI xQGyVFpmHT38+ek8Cp6BuHDwAYvkI1R8yK74kB4AlnLUVM9hI7zq+50CldG2uXE6aQg/D7ThQseI 9T+YqKe6HOBxce9YV4FQelxrdEYOgwOYw46obvJ2Mm4ng8Jz89wY6LST6nVEawRgIHFXh53zvqCQ Iz2KJOHaIdvDAgMBAAGjMzAxMA8GA1UdEwEB/wQFMAMBAf8wEQYDVR0OBAoECEqgqliE0148MAsG A1UdDwQEAwIBBjANBgkqhkiG9w0BAQUFAAOCAQEAWs6H+RZyFVdLHdmb56ImMOyTZ9/WLdI0r/c4 pc6rFrmrL3w1y6zQD7RMK/yA72uMkV82dvfbsxsZ6vSyEf1hcUS/KLM6Hb+zQ+ifv9wxCHGwnY3W NEcykMZlJPegSnwEc485bxeMcrW9S8h6+HuDwyhOnAnqZz+yZwQbwxTa+OdJJJHQHWr6YTnva+ch dQYH2BK0ISBwQnGB2jyaNr6mWw1qbJofkXv5+e9Cuk5OnswMjZTc2UWcXuxCUGOu9F3EsRLcyjuo Lp0UWgV1t+zXY+K6NbYECJHo2p2c9ma1GKwKplQmNDPSG8HUfxo6jguqMm7b/E8ln9kyx5ZacKzf TDCCBX0wggRloAMCAQICEQCH7S4aKCZKxRmqOuu5DaLLMA0GCSqGSIb3DQEBCwUAMDkxCzAJBgNV BAYTAkZJMQ8wDQYDVQQKEwZTb25lcmExGTAXBgNVBAMTEFNvbmVyYSBDbGFzczIgQ0EwHhcNMTQx MjA1MDgxOTE1WhcNMjEwNDA1MTAyOTAwWjA3MRQwEgYDVQQKDAtUZWxpYVNvbmVyYTEfMB0GA1UE AwwWVGVsaWFTb25lcmEgUm9vdCBDQSB2MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB AMK+6yfwIaPzaSZVfp3FVRaRXP3vIb9TgHot0pGMYzHw7CTww6XScnwQbfQ3t+XmfHnqjLWCi65I tqwA3GV17CpNX8GH9SBlK4GoRz6JI5UwFpB/6FcHSOcZrr9FZ7E3GwYq/t75rH2D+1665I+XZ75L jo1kB1c4VWk0Nj0TSO9P4tNmHqTPGrdeNjPUtAa9GAH9d4RQAEX1jF3oI7x+/jXh7VB7qTCNGdMJ jmhnXb88lxhTuylixcpecsHHltTbLaC0H2kD7OriUPEMPPCs81Mt8Bz17Ww5OXOAFshSsCPN4D7c 3TxHoLs1iuKYaIu+5b9y7tL6pe0S7fyYGKkmdtwoSxAgHNN/Fnct7W+A90m7UwW7XWjH1Mh1Fj+J Wov3F0fUTPHSiXk+TT2YqGHeOh7S+F4D4MHJHIzTjU3TlTazN19jY5szFPAtJmtTfImMMsJu7D0h ADnJoWjiUIMusDor8zagrC/kb2HCUQk5PotTubtn2txTuXZZNp1D5SDgPTJghSJRt8czu90VL6R4 pgd7gUY2BIbdeTXHlSw7sKMXNeVzH7RcWe/a6hBle3rQf5+ztCo3O3CLm1u5K7fsslESl1MpWtTw EhDcTwK7EpIvYtQ/aUN8Ddb8WHUBiJ1YFkveupD/RwGJBmr2X7KQarMCpgKIv7NHfirZ1fpoeDVN AgMBAAGjggGAMIIBfDBOBggrBgEFBQcBAQRCMEAwPgYIKwYBBQUHMAKGMmh0dHA6Ly9jYS50cnVz dC50ZWxpYXNvbmVyYS5jb20vc29uZXJhY2xhc3MyY2EuY2VyMA8GA1UdEwEB/wQFMAMBAf8wGQYD VR0gBBIwEDAOBgwrBgEEAYIPAgMBAQIwDgYDVR0PAQH/BAQDAgEGMB0GA1UdDgQWBBTwj1k4ALP1 j5qWDNXr+nuqF+gTEjCBuQYDVR0fBIGxMIGuMG+gbaBrhmlsZGFwOi8vY3JsLTEudHJ1c3QudGVs aWFzb25lcmEuY29tL2NuPVNvbmVyYSUyMENsYXNzMiUyMENBLG89U29uZXJhLGM9Rkk/Y2VydGlm aWNhdGVyZXZvY2F0aW9ubGlzdDtiaW5hcnkwO6A5oDeGNWh0dHA6Ly9jcmwtMi50cnVzdC50ZWxp YXNvbmVyYS5jb20vc29uZXJhY2xhc3MyY2EuY3JsMBMGA1UdIwQMMAqACEqgqliE0148MA0GCSqG SIb3DQEBCwUAA4IBAQAQ1elFTM6fGkQ/aRKdkUZicO3Cb9uzBJOpOtFctw+1El0/17lsjoVvJkZB D3KnUobnrriFdAa+7FAN55KLmZeB/3Y2bG0bB4toSyaVHjOQnQY9M0dv8U852w0Q7GwchKfebLUI bh9TMt2hI3Xc6j4knFTBUo7C1WAfO51K4bn1irmX6/Ej2VTgiOFsvOAny28W6enFSEQpSHw60VhN fSttSqTOxyrRR/7kW7Y8yb/3DZDZ/dH6ZCfx/y+BNIv2NuSd85M9HXUzplXXohti4Ql/qeaMn6by Ius6XlMWZZfkdVRvTuk2PkeC7UmAJ2+/DUWOPpawaytMXVfF4Hvxk34NMIIGDTCCA/WgAwIBAgIQ Us79UtuT8yEU6XIq+TroQjANBgkqhkiG9w0BAQsFADBHMQswCQYDVQQGEwJTRTERMA8GA1UECgwI RXJpY3Nzb24xJTAjBgNVBAMMHEVyaWNzc29uIE5MIEluZGl2aWR1YWwgQ0EgdjMwHhcNMTcxMTI3 MTAxMzQ4WhcNMjAxMTI3MTAxMzQ3WjB0MREwDwYDVQQKDAhFcmljc3NvbjEcMBoGA1UEAwwTSW5n ZW1hciBKb2hhbnNzb24gUzEvMC0GCSqGSIb3DQEJARYgaW5nZW1hci5zLmpvaGFuc3NvbkBlcmlj c3Nvbi5jb20xEDAOBgNVBAUTB0VQTElKT0gwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB AQCvxK7X3xRTAzY+JaEVXSFcykcRDnb9xJCcMYXRb7WCh0HpVZ7PThGgp0ZUNQQJ9ypm7YcIcwRD kYdD4fgxGLU1bIUC4GcH7rKiuf93b+Ec3jsrkSEkCsXkO7ROjlS9+7y7H/qC8bB8D4CTQNs15o3J 0njOaYngdmLGb8bwYV47hv3sAiyqoY0aZeGZyk2a/D9Kc28b+towJxQrGnekMxynyQQesKF7mFaA gtlEa5bNbLEfIki5tZWceSZEd5xXxuib7vv0Rb/Gd+Q+jyvobAEyd5RDB6i9HAA/FFPZIJTgM+k9 7yIFPPhpLjjcjeowqH9PlEaody2drdP67tBDnNQTAgMBAAGjggHGMIIBwjBIBgNVHR8EQTA/MD2g O6A5hjdodHRwOi8vY3JsLnRydXN0LnRlbGlhLmNvbS9lcmljc3Nvbm5saW5kaXZpZHVhbGNhdjMu Y3JsMIGCBggrBgEFBQcBAQR2MHQwKAYIKwYBBQUHMAGGHGh0dHA6Ly9vY3NwMi50cnVzdC50ZWxp YS5jb20wSAYIKwYBBQUHMAKGPGh0dHA6Ly9jYS50cnVzdC50ZWxpYXNvbmVyYS5jb20vZXJpY3Nz b25ubGluZGl2aWR1YWxjYXYzLmNlcjArBgNVHREEJDAigSBpbmdlbWFyLnMuam9oYW5zc29uQGVy aWNzc29uLmNvbTBVBgNVHSAETjBMMEoGDCsGAQQBgg8CAwEBEjA6MDgGCCsGAQUFBwIBFixodHRw czovL3JlcG9zaXRvcnkudHJ1c3QudGVsaWFzb25lcmEuY29tL0NQUzAdBgNVHSUEFjAUBggrBgEF BQcDBAYIKwYBBQUHAwIwHQYDVR0OBBYEFH5/jYsQz4BrrLa7pwQSkqtzL4EOMB8GA1UdIwQYMBaA FBx7GZ6XnHasID3Y3OORauPbLaZTMA4GA1UdDwEB/wQEAwIFoDANBgkqhkiG9w0BAQsFAAOCAgEA uwe9UtkD6Q6/dW/uQOCxTOaGCX76lOy+a0Fqlyh+ZmlS8qWj8YTCtD866gJOXMzYUMVFrWoeggvP 9j/MBmkDDuuJMgCkN3TEuAu4AF9ekcbm/yjWOUFx3kje0SYL+GNwN50He1tuOlmHrsHo7S89kOL6 Pq8+QMRSESzEqva7nC6sJdYxXGyWAagM1ZWFnJTtFCzsW6Tm6P6rpit4yCPYvPSgNPhPRj7UbHon vU4uwGBiO15aKLwg0jfoJn6wF/CZqtGeza+ucmcuEKA+y/IRnQ2pA5+p3QnX41eypQ7OJfCG5U6k fXjxyu7PGMompQQutZk75tw76wvE8kWj5VpDNf0dRVhq/PQugzqiRjrF5tFSQbOnq1GGQ4X3PhJ8 TBaoC4otM95ZguVQrM8wdYMSxeinbJY4T91eVXYltNgtUd84zbmL/1JgVa7PN3wASXPhguc0WZu3 OWNn31yKlqL/QmgDEoOs8UsNqYU7ngsg620TE9tta4S96LXRhXhBiQCDKR16ZhEITbaJPT7KgoMO 30YCHQa0K7ah5UxC9S6+E6cPK+uKyHvzXzjNdILMh6DXQFMblld6ax+to7tQQEkv1Xr4tjCN2H/z elNX2LGB1lO2qra3MfgaENY5SMk6sIi4Y4QDrVEBnrZeKstS9Qd3TCvc+fHu6bt3wU9Tk3K0dOAw ggbCMIIEqqADAgECAhBTuH6D4ZyZKJOwm0kc7LjrMA0GCSqGSIb3DQEBCwUAMDcxFDASBgNVBAoM C1RlbGlhU29uZXJhMR8wHQYDVQQDDBZUZWxpYVNvbmVyYSBSb290IENBIHYxMB4XDTE1MTAyNzEy MTY0NloXDTI1MTAyNzEyMTY0NlowRzELMAkGA1UEBhMCU0UxETAPBgNVBAoMCEVyaWNzc29uMSUw IwYDVQQDDBxFcmljc3NvbiBOTCBJbmRpdmlkdWFsIENBIHYzMIICIjANBgkqhkiG9w0BAQEFAAOC Ag8AMIICCgKCAgEA7PLfAAC4UPKnu9hUt8aT9+PBqjvUw0Y0tLPOXkO2NC0y2XZks9nJfpWKrNM3 0k5vu5norG4ZKlF5C+3xc6HuIiGQof1bmFGluNOwmZQwl3rOJ+E6k0rqJJTerjj4WOxAvWVW1yC5 S4Ubppk3Q3cYVVuC3qNGsBIXy3/fDL1sc8Ah8zI/JumDpjY8fn/U3CRN6mgNKYrr0sZX6VXYgrpT 05ZrJldkUgUgMKgbIWWEXEASA36pnb5GqD/RMzSgIe8o7YQtIaYB2cmTCLNHjaOL9j1JhNK4bvmb NJ7o58IZYzwNv/G/L/bRosQ9c27U+86DNjrdZnpyaRaeMyVUn3SlYLaFqoObdh/xNF2NS8CXs/PV tO57HBKHMgZqQvsyQJisSocxFqiMj9VK2WhCBbvoTvrNDZvLDlDGuE5RuKwFIpHOVOU5lCBgUUBs bpWIXwM6kmH/KC1DC5MtQzmvXkbt7KdBXUAxM0JZxf4dS+ACtTDpF9b0vny4DrwaOS0VNXyz1GUO xSqw1wup5dpXbxLZYx1rLRgZqr9uWhLwAPsq66ZQof5GL0gY72Ym8/Tm28MeMqku+/zRzdYsmclT 9rOdgdgS3b6OMoc5Op0ZPEv/Mx2lFJAVK674ozw2hiuRTVUmoqBr5AuyCoqCEyn32C7U/V7oqyqx 5Yd1c5GsxuOqQFcCAwEAAaOCAbgwggG0MIGKBggrBgEFBQcBAQR+MHwwLQYIKwYBBQUHMAGGIWh0 dHA6Ly9vY3NwLnRydXN0LnRlbGlhc29uZXJhLmNvbTBLBggrBgEFBQcwAoY/aHR0cDovL3JlcG9z aXRvcnkudHJ1c3QudGVsaWFzb25lcmEuY29tL3RlbGlhc29uZXJhcm9vdGNhdjEuY2VyMBIGA1Ud EwEB/wQIMAYBAf8CAQAwVQYDVR0gBE4wTDBKBgwrBgEEAYIPAgMBAQIwOjA4BggrBgEFBQcCARYs aHR0cHM6Ly9yZXBvc2l0b3J5LnRydXN0LnRlbGlhc29uZXJhLmNvbS9DUFMwSwYDVR0fBEQwQjBA oD6gPIY6aHR0cDovL2NybC0zLnRydXN0LnRlbGlhc29uZXJhLmNvbS90ZWxpYXNvbmVyYXJvb3Rj YXYxLmNybDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwQwDgYDVR0PAQH/BAQDAgEGMB0G A1UdDgQWBBQcexmel5x2rCA92NzjkWrj2y2mUzAfBgNVHSMEGDAWgBTwj1k4ALP1j5qWDNXr+nuq F+gTEjANBgkqhkiG9w0BAQsFAAOCAgEAUFhr8dWMO7Quq1dDyIynw8sWmpyF/jWSxBjpHUCyhlto FS7Q1CUBD0bOULWmYjmzRwme5pkjTFXpOJZLf9Han1SBbrVcP0JMhRsAvfWZjcF0l/c/jqDMqBAR xr8OUWOr0ZWa49Lir3QEs2C+CjGge5tzcLqzQ5pjWxudrLkSGe+sAThDnXUWXGYk8udGZAamJ55d rdw96AV9jWQkMrLIVHKkXVG5Etdx0wiAoTLk1fVtLcz11DiaCZSZVPZ3fdSIpIRhDqz8H4sVprPg vLBdK/ajdbiRsehCzzohay3zbXDDTDGwKkR8KUi8Xt8HDZCRsb/U/C7MC4tVK0SEPOQCo6swZy0r I0RoGzICfsSrZ4JrxANeeSZqCn1A+w0Wz+iqdeP2PVxW0f1rg4/OG2DSl3uB3Q3NT/lDGJtepti+ i5CCKEZcdAOZoviu43sLhqsxSpGjzZidESwovuHeP+O2bNwwtz1DTsXThBB3+JJHVjmkiLo900GI Tb/i7IBdLoo4gZms9s1BQ2tm3CJCmpA2XwBTOB6B8/CtgWUWhyloXd3Wbmv7ZUoqqJFBV9g8Zh5m dZ+RzPTomgCFz/2aNsddI/2G9ZjN4tG6hmocZR2M5f0MhBv3bo6d5XsLlYwiNJjw5GRqYb8cqqeC aPKkveBJzqgb8ToH7WLoOzmPRCmPlpAxggMCMIIC/gIBATBbMEcxCzAJBgNVBAYTAlNFMREwDwYD VQQKDAhFcmljc3NvbjElMCMGA1UEAwwcRXJpY3Nzb24gTkwgSW5kaXZpZHVhbCBDQSB2MwIQUs79 UtuT8yEU6XIq+TroQjAJBgUrDgMCGgUAoIIBfDAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwG CSqGSIb3DQEJBTEPFw0yMDExMDIwOTUwNTNaMCMGCSqGSIb3DQEJBDEWBBSaSjhk2BSnze3y/YR3 2k+wV/927zBDBgkqhkiG9w0BCQ8xNjA0MAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggq hkiG9w0DAgIBQDAHBgUrDgMCGjBqBgkrBgEEAYI3EAQxXTBbMEcxCzAJBgNVBAYTAlNFMREwDwYD VQQKDAhFcmljc3NvbjElMCMGA1UEAwwcRXJpY3Nzb24gTkwgSW5kaXZpZHVhbCBDQSB2MwIQUs79 UtuT8yEU6XIq+TroQjBsBgsqhkiG9w0BCRACCzFdoFswRzELMAkGA1UEBhMCU0UxETAPBgNVBAoM CEVyaWNzc29uMSUwIwYDVQQDDBxFcmljc3NvbiBOTCBJbmRpdmlkdWFsIENBIHYzAhBSzv1S25Pz IRTpcir5OuhCMA0GCSqGSIb3DQEBAQUABIIBAEA6QCDOzSOhdmUAYLyzaXQbI6ftcXFpX2GkX5Wt Qmu2AjdfK6JiXP6KKXOJ7leAkh2jEKD/1VweU5WgWWImmlsmNUli0b18fyW05KrZDImpXG1XNaWw bumKQKnO4dVT5XUdJtiTFQqrCPkIbalLZDqRDYItTd+8NGCjsaqncb7arSAwXASW7HO3yCw80kQW YZFp+t3j+gH2uE1tBjwfqW1SizGVBCOMOFZoVysXfhjNH4YPLyvKS8IfZCaZunbn4tG25QYkSUvs STGycgk1muVT8aapXirEuZ0SBgJ7EmYo6K+7+IdMuNGa8cmR6GR+4oePsH9SnX8313goZvTb04oA AAAAAAA= ------=_NextPart_000_00DB_01D6B106.09411A70-- From nobody Mon Nov 2 02:28:00 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BF01E3A0DB0; Mon, 2 Nov 2020 02:27:57 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7L0fs7pxpCJN; Mon, 2 Nov 2020 02:27:55 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6C6453A0DAC; Mon, 2 Nov 2020 02:27:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1604312822; bh=zxhi8r9Q4YOYzQbis3rTJFyUFZiEMnOinvgwgFgOm2U=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=FTR6QQbCmT+bxyI5LkllO8w8Dd3b5OB2a+cpn/jAoGWU4isyS5CN381SDmZz8ZnmS f3c6s6jMNfHt7kHpgv4vXe8TuF0LwhtgJ/5TffTLm3/vuRvFDNIJy4OC3vYWNeVngf oGJlRB0rznAPwq+n3Bab3kwpCwSIMHomXg9oCm2c= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [134.76.241.253] ([134.76.241.253]) by web-mail.gmx.net (3c-app-gmx-bap61.server.lan [172.19.172.131]) (via HTTP); Mon, 2 Nov 2020 11:27:02 +0100 MIME-Version: 1.0 Message-ID: From: Sebastian Moeller To: Ingemar Johansson S Cc: "Rodney W. Grimes" , Bob Briscoe , tsvwg IETF list , iccrg IRTF list , Christian Huitema , TCP Prague List , "De Schepper, Koen (Koen)" , Ingemar Johansson S Content-Type: text/plain; charset=UTF-8 Date: Mon, 2 Nov 2020 11:27:02 +0100 Importance: normal Sensitivity: Normal In-Reply-To: References: <69a73308-6718-7304-be24-0eb84f77e50d@bobbriscoe.net> <202011012023.0A1KNkue011764@gndrsh.dnsmgr.net> Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K1:75w8E9R1+Xvu4V1tYGvta0s4W9cKhzb2z34zMQzZS6zbxw/5T3bOjgVjduiddB0PTTiDO h/I0dhZDqyj5tgfr5LRJtlo/R3aIcHM8sULBiK79Idki26mQS5aoQEYLpe6MtQfH/A7JFgJcMdJ+ 22lxCtFHFO228qiWF2lbTPPtJ9sy1uRpCHknu/46P/VoYgzZxxaWHl9fymCXbUUyGa5yp7StR8wa aZufITpVQyoZK0UHZcRpHbqqTbA3kxI9uphc3OHk472NMAcUyAwTSsBJb530SkzZagS5oXAMM+uO s0= X-UI-Out-Filterresults: notjunk:1;V03:K0:1shEMiC4Tzg=:NmIX8hZ47jxI5gMms7BfJ1 uwswc4PtKV92VvATXrKAVitSjvdAEl/zEu9KYFwkg7EfG+z8MGa0OEbBUVX8tLd+FkJ12I6RQ MmrtABAEZOa+z1QmW2Gb7BGSZiGF9nzQjUz1h6eSvoz11k3Nogc0E0dDJBqSwcSMX54o47wJ8 4YM4zen7m4qGA3ZZRQXNtPQ6HKBPm0NP1OtOxvL9RMfavQTotOpBcSRcEOWHVKdfX7vaUKNWG xHKernc8oubLgepFQn4DaIKZmcLqya8gMw9enjQvk3jglmP+DqCUu5pzriNO/WwFfct0gFp/Y B1NGOrK33dDQMD7lTLqcMKpgpncMbSs67FY7tSSAuBSZPrie+qJ0R6thinAiCLy4oqHoIclT1 eWaAjom6SmtkQtETCojzYEyW3oDWaqbkwbrUIiULC0DD0iPjW49WMTN7fs60JNFnkfg7OlNMo 0HI1wDJmGKbjz1lAUBucLFulKo5ateGfBzE5HMz4vubg7TbAaX2qskET8ugsML4xGXY6txusj ERi4717kP48Y8o6q7CWU18XYaD+gk0xRfTKAnkDs0+dNU+t8zZzt8YrOSu9NbWWHDQxNYrlpJ 6fOund+KZ8mTQ= Archived-At: Subject: Re: [tcpPrague] [tsvwg] [iccrg] Realistic Queue delay targets X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 10:27:58 -0000 Hi Ingemor, more below in-line, prefixed [SM2]=2E > Gesendet: Montag, 02=2E November 2020 um 10:50 Uhr > Von: "Ingemar Johansson S" > An: "Sebastian Moeller" > Cc: "Rodney W=2E Grimes" , "Bob Briscoe" , "tsvwg IETF list" , "iccrg IRTF lis= t" , "Christian Huitema" , "TCP Pr= ague List" , "De Schepper, Koen (Koen)" , "Ingemar Johansson S" > Betreff: RE: Re: [tsvwg] [iccrg] Realistic Queue delay targets > > Hi > I don't anymore know where all these discussions are heading=2E=20 [SM2] Ideally this process would lead to better designed and imple= mented L4S core methods and better written internet drafts=2E All that requ= ires team L4S to be a bit more open, to a) support their assertions with re= al data and b) at least occasionally accept criticism as valid and change m= ore than a few words in the draft as a consequence=2E As long as team L4S b= asically maintains L4S is perfect as it is, this is heading nowhere (at lea= st it should, in reality we are heading straight towards ratifying the exis= ting IMHO sub-optimal state of the L4S internet drafts as experimental trac= k RFCs, if only based on the argument that after working on this for a the = better part of a decade it is now time to show up something)=2E This view i= s supported by the fact that the chairs stopped soliciting actual data and = safety tests, and focus mainly on seeing proposals for text changes=2E Even= though it is still unclear if L4S is actually suitable for wider release i= nto the internet=2E The current drafts however are not written in a way to = conduct an "is L4S safe to deploy into the internet" experiment, as they of= fer neither criteria on how to asses safety, nor a plan for how to resolve = the experiment for either outcome=2E (On success it is simple, either do no= thing and keep experimental infrastructure in use, or try to move the RFC t= o standards track, but on failure, how to disable all the network nodes and= how to reclaim the ECT(1) codepoint for other experiments?) Seems like we=20 > always end up in infinite discussions that lead nowhere!=2E [SM2] Oh, no=2E we are making progress albeit at a worrisome slow = rate=2E For example, your points confirmed that relaying on protocol design= ers to meet requirements that only have subtle safety consequences (like th= e 15ms hack) seems overly optimistic and not a viable path to guaranteeing = safety=2E Yet, I am certain, that this assessment will a) not be shared uni= versally on this list, nor lead to changes in how team L4S plans to deal wi= th issue #28 (the strategy so far is to more or less ignore that issue)=2E > Network implementers will implement whatever queue management they find= =20 > useful, be it dualQ, fq-codel or whatever=2E [SM2] Yes, and that as a consequence means, that the core L4S meth= ods and components need to offer robust safety even when protocols might be= (accidentally) cutting corners, no? Because if even you, a strong proponen= t of L4S, did not understand that section 4=2E3 was not intended to be opti= onal, how can we expect other designers less involved in L4S get this right= ? >=20 > Yes, there are ways to deal with jitter, some are implemented, some are = in the=20 > pipe, some are on the drawing board and as I mentioned several times: It= is=20 > not very fruitful to take todays internet jitter as a litmus test on how= good=20 > L4S will behave in the future=2E > As examples, 5G has a shorter HARQ retransmission loop than 4G, this red= uces=20 > jitter, there is also work to reduce handover delay, shorter TTIs etc=2E= It is=20 > therefore quite meaningless to produce data on how well L4S will fare in= the=20 > precense of jitter=2E Leave that to the experiments=2E >=20 > /Ingemar >=20 >=20 > > -----Original Message----- > > From: Sebastian Moeller > > Sent: den 2 november 2020 09:47 > > To: Ingemar Johansson S > > Cc: Rodney W=2E Grimes ; Bob Briscoe > > ; tsvwg IETF list ; iccrg IRT= F list > > ; Christian Huitema ; TCP Pra= gue List > > ; De Schepper, Koen (Koen) > > > > Subject: Aw: Re: [tsvwg] [iccrg] Realistic Queue delay targets > > > > Hi Ingemar, > > > > more below in-line, prefixed [SM]=2E > > > > > Gesendet: Montag, 02=2E November 2020 um 09:30 Uhr > > > Von: "Ingemar Johansson S" > > > > > > An: "Rodney W=2E Grimes" , "Bob Briscoe" > > > > > > Cc: "tsvwg IETF list" , "iccrg IRTF list" > > > , "Christian Huitema" , "TC= P > > > Prague List" , "De Schepper, Koen (Koen)" > > > > > > Betreff: Re: [tsvwg] [iccrg] Realistic Queue delay targets > > > > > > Hi Rodney > > > > > > I believe part of the answer to your question is found in > > > https://tools=2Eietf=2Eorg/html/draft-ietf-tsvwg-l4s-arch-07#section= -6=2E3 =2E > > > The fact that we begin to see the scheduling jitter does not > > > invalidate L4S as a means to reduce the congestion related jitter=2E > > > > [SM] As so often that section does contain some theoretical=20 > > observations, > > but fails to empirically show which level of burstyness L4S can deal w= ith > > gracefully and what the consequences are (on the bursty flow itself, o= ther > > flows in the shallow-queue and and flows in the deep queue)=2E That wo= uld be > > meaningful responses to Rodney's concern=2E As is that section basical= ly just > > claims without supporting data, that even in the face of bursty flows,= L4S > > should still be tried, but that must be compared to a FiFO, because co= mpared > > to say fq_codel with a 5ms default target, traffic with > 5ms jitter w= ill=20 > > probably > > look much better after fq_codel than after L4S (L4S will conserve the > > burstyness much more, while fq_codel will smooth it out a bit)=2E > > I wonder why such verbose but essentially non-actionable (almo= st > > useless) sections keep getting added to already quite long and verbos= e > > internet drafts in the first place? > > > > Regards > > Sebastian > > > > > > > > Scheduling jitter exist also in cellular technology=2E Scheduling ji= tter > > > is however generally not evidence of congestion=2E L4S congestion > > > marking should therefore be designed to not trigger on this=2E The > > > master thesis** the L4S marking threshold was set sufficiently high = to > > > avoid this but it should be understood that L4S marking in cellular = can be > > implemented in smarter ways=2E > > > In the master thesis, the L4S marker only inspected the queue on the > > > RLC layer=2E > > > There are other causes to delay jitter in cellular, HARQ > > > retransmissions and handover delays are two examples=2E There is wor= k in > > > progress to reduce all kinds on delay jitter, L4S is a tool to reduc= e > > congestion related jitter=2E > > > In any case, non-congestion related jitter is not an argument agains= t > > > L4S > > > > > > /Ingemar > > > > > > ** > > > https://kth=2Ediva-portal=2Eorg/smash/record=2Ejsf?dswid=3D-6303&pid= =3Ddiva2%3A1 > > > 484466 > > > > > &c=3D1&searchType=3DSIMPLE&language=3Den&query=3Dbrunello&af=3D%5B%5D&= aq=3D > > %5B%5B% > > > 5D%5D& > > > > > aq2=3D%5B%5B%5D%5D&aqe=3D%5B%5D&noOfRows=3D50&sortOrder=3Dauthor_sor > > t_asc&sort > > > Order2 =3Dtitle_sort_asc&onlyFullText=3Dfalse&sf=3Dall > > > > > > > -----Original Message----- > > > > From: Rodney W=2E Grimes > > > > Sent: den 1 november 2020 21:24 > > > > To: Bob Briscoe > > > > Cc: Christian Huitema ; tsvwg IETF list > > > > ; iccrg IRTF list ; TCP Prague= List > > > > ; De Schepper, Koen (Koen) > > > > > > > > Subject: [iccrg] Realistic Queue delay targets > > > > > > > > Bob, > > > > > > > > > Christian, > > > > > > > > > > I've changed the subject line given it's no longer appropriate= =2E > > > > > See inline tagged [BB]=2E=2E=2E > > > > > > > > And I am changing it again=2E=2E=2E as only addressing a statement= that I > > > > have > > > great > > > > concerns about=2E > > > > > > > > See inline ragged [RWG] > > > > > > > > > On 01/11/2020 01:07, Christian Huitema wrote: > > > > =2E=2E=2E [ much text deleted ] =2E=2E=2E > > > > > > > > > [BB] Yes, once the IETF assigns the codepoint, BBRv2 will be 'al= lowed' > > > > > to send packets over the Internet as ECT(1)=2E Then, yes, an L4S > > > > > DualQ Coupled AQM will classify BBRv2 packets into the L4S queue= =2E > > > > > This should have a very shallow ECN marking threshold (500us - > > > > > 1ms), so even if the flow (whether QUIC or TCP) is flying just > > > > > under the available capacity, bunching of packets means it is > > > > > unlikely to completely avoid ECN marking between probes=2E If it > > > > > could avoid ECN marking, you are right that it would get a bump = of > > > > > ECN marks during the probe=2E I haven't studied the code, but wh= en > > > > > it experiences ECN marking I believe it switches into an L4S ECN > > > > > mode for a while, and uses ECN rather than delay probing to trac= k > > > > > available capacity=2E I assume it switches back to BBR's delay > > > > > probing mode if it gets no ECN for a while (e=2Eg=2E the bottlen= eck > > > > > might have moved)=2E But I haven't looked > > > at > > > > BBRv2's ECN behaviour in detail=2E > > > > > > > > [RWG] I have great concern about this 500uS to 1mS ECN marking > > > > threshold given that I have recently learned the latest WiFi > > > > standards actually run > > > with a > > > > packet aggregation time of 4mS thus making it probably impossible = to > > > > have such traffic work in such a targeted low latency queue=2E > > > > > > > > [RWG] Has any consideration been given to what is already deploye= d > > > > on the Internet as link layer technologies that would preclude the > > > > operation of > > > the > > > > L4S low latency queue? > > > > > > > > Regards, > > > > -- > > > > Rod Grimes > > > rgrimes@freebsd=2Eorg > > > > > > > > > > > From nobody Mon Nov 2 04:33:15 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5BD5B3A0E9F; Mon, 2 Nov 2020 04:33:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xIB-2pO8WeGp; Mon, 2 Nov 2020 04:33:05 -0800 (PST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2126.outbound.protection.outlook.com [40.107.22.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E4CA53A0EB7; Mon, 2 Nov 2020 04:33:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FZi7fhfnrZC7088iDot1FkPpdV7hyfzXd3IFesBcYY4NcMqjcgk1SpTvPUQdiGI6WiuVs8yZQzSBeLlqdP3pYDQGlWIc+WOjwepdOme+gXMEk049br6mligp/Ku3JdAhTCRZfrSYA1qcAoMWWrWatmGyfrOvi/3lULk8zg7r6CawI0FkmIxsTwm0L+pN+X0AfyU5NTPGICXdiqJZgh49X6GQLB6b6UavUCcVyS2rOTP2gRfMy9GrtE3+3KA1W5cQSx3ijmw+CB/PIv5cTJkFAPpUeJuUl8KAa/GfzxpKGbJoQGC98tOheRsI5cNGCI3LkY74X5L6ZVeLZBk5zDDCOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iYLZZHGYmhJBEXkmSEkZ0g6jr8AJevdcPBtPjYtzSjA=; b=VENqVWOyOlNteF1loOULTyO0gieVBhKPdgdTDLZaobjiLX1xvGemwFmnY9HRL6jyf7v0MMbEIel8vgFs/4ccRgUjW81ln/mlsudAsr50klaJWr0gUWi3wKsFohVwVVpEWKHqYcYKgMnudKuBGj2+bi5XFBDT8hzNkzukqLI1y12fzqFTHL7vtVHAk1sSiyP4UFCAYLZx5uZp+nNWwBOP7LsOEezoJTqNyftxnEpIvxDH+1n28Seg8EI6Dbn1TgCOnXTtAuAY1zsgmyt/h9LqpetwivsJ1d8WGtp1OjP60JuSTR5coEKmqnvjRcRb25IPGxuMt1JtMSPtEDXTbGF1Mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-bell-labs.com; dmarc=pass action=none header.from=nokia-bell-labs.com; dkim=pass header.d=nokia-bell-labs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iYLZZHGYmhJBEXkmSEkZ0g6jr8AJevdcPBtPjYtzSjA=; b=mxRmQ37PEy7nsSl6/C+2BXXzzNo1s1c9DRsZojTecdgIs+2jLDDlJOFakEO9rEPyQSfyaz7Sh6iVmFNBKLiFNMYEBa+/TTkT205KmeWHUESBxn8P040FOGYaa1VOrwVPhhQ1hSsCg69Zfdg6oUDp4R7WpeDWRlfVadFlr5T39v8= Received: from AM0PR07MB6114.eurprd07.prod.outlook.com (2603:10a6:208:113::33) by AM0PR07MB3940.eurprd07.prod.outlook.com (2603:10a6:208:46::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.10; Mon, 2 Nov 2020 12:33:01 +0000 Received: from AM0PR07MB6114.eurprd07.prod.outlook.com ([fe80::a9f4:9199:8226:d6c6]) by AM0PR07MB6114.eurprd07.prod.outlook.com ([fe80::a9f4:9199:8226:d6c6%6]) with mapi id 15.20.3541.010; Mon, 2 Nov 2020 12:33:01 +0000 From: "De Schepper, Koen (Nokia - BE/Antwerp)" To: Christian Huitema , Jonathan Morton CC: iccrg IRTF list , Bob Briscoe , tsvwg IETF list , TCP Prague List Thread-Topic: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text Thread-Index: AQHWsIfwC/FrblPY50a/ND1AVr6ZVKmzwIVwgAAzdwCAAAQwgIAAuLTQ Date: Mon, 2 Nov 2020 12:33:01 +0000 Message-ID: References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> In-Reply-To: Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: huitema.net; dkim=none (message not signed) header.d=none;huitema.net; dmarc=none action=none header.from=nokia-bell-labs.com; x-originating-ip: [2a02:1810:1e00:cb00:106c:a742:663:cd5d] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: b9f847ca-e70a-4bf8-f867-08d87f2b7056 x-ms-traffictypediagnostic: AM0PR07MB3940: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4uSY6tAcv+zjuApLt2VJGUV783/AsU+iCV4NJFgWjJd2A9SvK0qdB4CwMyk0PcMQK6UJRNIIP5mhqNM5Ivofd/O8vtUc03qu/NRDpQsJXbhUyk+1u2ZAp0cFqAQuhAeNZfi7wUGqi+x2i/EhLTzYHOixJS9XDBpAv2zMB3krjhU6j3R6vZJpHStIMKI1PPSYJiwIQq6iRUktV9N6L50QeadN9pviiuoGkHxx/BrZHcJt1YswbsK+0CwvGeWF94Mxl7p/CFpZUroofGdr0pEvgdZoPmiU9JF/kSXBcJgzc3/O/Dv0ZUGJBjTLPSM7k3DmmlIornM9IL7j9wm0Gxr3tiRc5k1YiUG99dAyNgj+PRUjJ5TcYLBc2hEjAE5ysHh9g6znBFy6NrLbm2XtN9OLWg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR07MB6114.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(346002)(376002)(39860400002)(396003)(66556008)(64756008)(66446008)(86362001)(52536014)(76116006)(4326008)(71200400001)(66946007)(8936002)(83380400001)(66476007)(186003)(7696005)(33656002)(110136005)(54906003)(316002)(8676002)(5660300002)(9686003)(166002)(2906002)(6506007)(966005)(53546011)(55016002)(9326002)(478600001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: ATAjPNFHijQ/D1JtYdU2+ZQs2YaikgZNqqg4XNemGHwjmWf/3u3ctz9ce3L5bUoWjZc+HDr5D3qrtMrbs1mcLGRDfsFlgjGRqNKdFt7+mTu1KHRvVh4H0zVRALmBwGIvGOYy79KVmQUJqM7g3YtnUAinaOAJaTgKFU4vwg/ACoGSdwlFGwk1pRzoXRPQ7P00QsbzZM1ak7GPic71bCdtghZB5pv/+rGkZ6DU2/Ya+5Rg6YtLGrImIoyuog3yWI2FCT3QuC0JVQq0vRbhPbkPdxU5eCUutJ14NZlvVrg/qfRraF/x2tsL9lcURYKrVPm/cT7hQ6WiL+kdLQ0ABWaj/A25yGW1Cql/v2ht3aBYSsyrFXod8rBLzCC3g+TShjeqRjf6qnwDhw9h0/arDPMcfauR8B33DN4tNrXxoj5FQ+p1hxSWe77QhxSFh6ls3lejQmxJ+5N+jIWVnKjIeyYprSPKzpEorCp8B8ZbYtLS0nD9awaoBfI31WFKvN+MQ7jQwlPlnDcsuUILFwvKt+W390Hi6XEJejPQdpTZUb6Q++/RAyUKrn114jFwIJcZOYN3vzdRREr4u55vxuIG5fHu91GNQyXe9w6A27p4zyudvymmlNrbI8dPZW3zXLaB9AGQdPjpCw18rkqygwYS3Jpep7iucOGJmzHZhELznzMU2ixQlQW7xPQsetG3H2BfyJFAbjVc/kVmAfR0UtnHKodqwg== x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR07MB6114D78B68E07B2F712E12E4B9100AM0PR07MB6114eurp_" MIME-Version: 1.0 X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR07MB6114.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b9f847ca-e70a-4bf8-f867-08d87f2b7056 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Nov 2020 12:33:01.6914 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: OG7un+pkTVfblW0dAWLq72e2d5zYeNOUUptV2G8rih5b8FVuel8RTOO9EZpMoGDNJvnALRKq7QQ5ZxachhprxWXcPhgfkCIV14Lvor4YVvjdjtWQPP65QVzxMFIANNof X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR07MB3940 Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 12:33:09 -0000 --_000_AM0PR07MB6114D78B68E07B2F712E12E4B9100AM0PR07MB6114eurp_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 SGkgQ2hyaXN0aWFuLA0KDQpJbmRlZWQgYSB2ZXJ5IGltcG9ydGFudCBkaXNjdXNzaW9uIHdlIG5l ZWQgdG8gaGF2ZSwgYW5kIGFncmVlIG9uLg0KDQo+PiBJZiB5b3UgZ2l2ZSB1cCB0aGUgZm9ybXVs YSwgeW91IGNvdWxkIGVuZCB1cCB3aXRoIGEgc2lnbmlmaWNhbnRseSBzaW1wbGVyIGV2ZW50IGJh c2VkIGNvbmdlc3Rpb24gY29udHJvbCwgaW4gd2hpY2ggYXBwbGljYXRpb25zIHNsb3cgZG93biBp ZiB0aGV5IHNlZSB0b28gbWFueSBDRSBtYXJrcywgYWNjZWxlcmF0ZSBpZiB0aGV5IG9ubHkgc2Vl IGEgdmVyeSBsb3cgbWFya2luZyByYXRlLCBhbmQgc3RheSBwdXQgaWYgdGhleSBzZWUgYW4gaW5m cmVxdWVudCBtYXJraW5nIHJhdGUuDQoNClRoaXMgaXMgZXhhY3RseSB3aGF0IEkgZXhwZWN0IG1v c3QgQ0NzIHdpbGwgZG86IHdvcmsgZXZlbnQgYmFzZWQuIFRoZSBmb3JtdWxhIGlzIG9ubHkgdG8g dHVuZSB0aGUgZXZlbnRzIHNvIGl0IGNvbnZlcmdlcyB0byB0aGUgZm9ybXVsYSAoZXZlbnR1YWxs eSkuIElmIHRoZXJlIGlzIGEgbG90IG9mIGR5bmFtaXNtLCBvYnZpb3VzbHkgc2hvcnRlciBSVFRz IGNhbiByZXNwb25kIGZhc3RlciBhbmQgd2lsbCBiZW5lZml0IGZyb20gdGhhdC4gVGhleSB3aWxs IGhhdmUgbWVtb3J5IG92ZXIgYSBzaG9ydGVyIHRpbWUgaW50ZXJ2YWwsIGFuZCBvbiBhdmVyYWdl IG1vcmUgdGhyb3VnaHB1dCB0aGFuIGEgbG9uZ2VyIFJUVCBmbG93LiBCdXQgaWYgdGhpbmdzIGFy ZSBzdGFibGUsIGl0IGlzIGdvb2QgdG8gY29udmVyZ2UgdG8gYSBrbm93biByYXRlLiBJIGRvbuKA mXQgc2VlIGFueSBiZXR0ZXIgc29sdXRpb24sIGRvIHlvdT8gSWYgd2UgZG9u4oCZdCBmb3Jlc2Vl IHN1Y2ggYSBtZWNoYW5pc20gaG93IGRvIHlvdSBleHBlY3QgYmFuZHdpZHRoIHRvIGJlIHNoYXJl ZCBvdmVyIHNoYXJlZCBpbmZyYXN0cnVjdHVyZT8gVGhlbiB0aGUgd2hvbGUgZGlzY3Vzc2lvbiBv ZiBmYWlybmVzcyBjYW4gYmUgZGlzbWlzc2VkLCBhbmQgQ0NzIHNob3VsZCBub3QgYmUgZXZhbHVh dGVkIGZvciB0aGlzIG5laXRoZXIuDQoNCj4+IFRoZSBtYXJraW5nIHJhdGUgaXMgYWN0dWFsbHkg MC4xJSwgYWJvdXQgMiBwYWNrZXRzIHBlciBSVFQNCg0KSW5kZWVkLCB0aGlzIGlzIHN0aWxsIDIg bWFya3MgcGVyIDE1bXMsIG11Y2ggYmV0dGVyIHRoYW4gMSBkcm9wL21hcmsgcGVyIDEwMDAgUlRU cyAoMTUgc2Vjb25kcykgZm9yIHlvdXIgZXhhbXBsZSB3aGVuIHVzaW5nIFJlbm8gKEN1YmljIHdp bGwgYWxzbyBiZSBzZWNvbmRzKS4gT25seSBpZiB0aGUgUlRUIGlzIG11Y2ggc21hbGxlciB0aGFu IDE1bXMsIGl0IHdpbGwgZ2V0IGxlc3MgbWFya3MgcGVyIFJUVCwgYnV0IGluIG91ciBMaW51eCBQ cmFndWUgd2UgaGF2ZSBhbiBvcHRpb24gdG8gZ3JhZHVhbGx5IHRha2UgdGhlIFJUVCBkZXBlbmRl bmNlIGludG8gYWNjb3VudCwgZGVwZW5kaW5nIG9uIGhvdyBsb25nIHRoZSBmbG93IGlzIHJ1bm5p bmcgYXBwbGljYXRpb24gdW5saW1pdGVkIChvciBieSBleHRlbnNpb24gaG93IGxvbmcgdGhpbmdz IGFyZSBzdGFibGUpLg0KDQpBbHNvIHN1Y2ggYSBmb3JtdWxhIGlzIGp1c3QgYSByZWZlcmVuY2Ug dG8gYWltIGF0LiBJdCBzaG91bGQgbm90IGJlIGEgY29udGludW91cyBoYXJkIHJlcXVpcmVtZW50 LiBUaGVyZSBhcmUgcmVhc29ucyB3aGVuIGEg4oCcZmxvd+KAnSBjYW4gZGV2aWF0ZSBmcm9tIGl0 IChMZXNzIHRoYW4gYmVzdCBlZmZvcnQgY2FuIHRha2UgbGVzcywgYSB0dW5uZWwgb2YgbXVsdGlw bGUgZmxvd3Mgd2lsbCB0YWtlIGFzIG1hbnkgdGltZXMgbW9yZSBhcyB0aGVyZSBhcmUgZmxvd3Mg aW5zaWRlLCBhbmQgYSByZWFsLXRpbWUgYXBwbGljYXRpb24gbWlnaHQgdGVtcG9yYXJ5IHRha2Ug bW9yZSB3aGVuIG5lZWRlZCkuDQoNCknigJltIGluIGZhdm9yIG9mIHNwZWNpZnlpbmcgZXhwbGlj aXRseSBzdWNoIGEgZm9ybXVsYSBhcyBhIHRhcmdldCBmb3IgTDRTIEND4oCZcyB0byBzaGFyZSBC VyBpbiBzdGVhZHkgc3RhdGUgc2l0dWF0aW9ucyAobG9uZyBmbG93cyBhbmQgc3RhYmxlIHRocm91 Z2hwdXQpLiBVcCB0byBub3cgSSBkb27igJl0IHRoaW5rIHdlIGhhdmUgc3VjaCBhbiBleHBsaWNp dCByZXF1aXJlbWVudCBpbiB0aGUgZHJhZnRzLiBBbnlvbmUgZWxzZSB0aGF0IHdvdWxkIHN1cHBv cnQgdGhpcz8gQXQgbGVhc3QgQ0MgY29udHJvbCBkZXZlbG9wZXJzIHRoZW4gaGF2ZSBhIHRhcmdl dCB0byBhaW0gZm9yIGFuZCBhIGNsZWFyIGV2YWx1YXRpb24gY3JpdGVyaXVtLg0KDQpSZWdhcmRz LA0KS29lbi4NCg0KDQpGcm9tOiBDaHJpc3RpYW4gSHVpdGVtYSA8aHVpdGVtYUBodWl0ZW1hLm5l dD4NClNlbnQ6IE1vbmRheSwgTm92ZW1iZXIgMiwgMjAyMCAxOjExIEFNDQpUbzogRGUgU2NoZXBw ZXIsIEtvZW4gKE5va2lhIC0gQkUvQW50d2VycCkgPGtvZW4uZGVfc2NoZXBwZXJAbm9raWEtYmVs bC1sYWJzLmNvbT47IEpvbmF0aGFuIE1vcnRvbiA8Y2hyb21hdGl4OTlAZ21haWwuY29tPg0KQ2M6 IGljY3JnIElSVEYgbGlzdCA8aWNjcmdAaXJ0Zi5vcmc+OyBCb2IgQnJpc2NvZSA8aWV0ZkBib2Ji cmlzY29lLm5ldD47IHRzdndnIElFVEYgbGlzdCA8dHN2d2dAaWV0Zi5vcmc+OyBUQ1AgUHJhZ3Vl IExpc3QgPHRjcFByYWd1ZUBpZXRmLm9yZz4NClN1YmplY3Q6IFJlOiBbdGNwUHJhZ3VlXSBbdHN2 d2ddIGVjbi1sNHMtaWQ6IFByb3Bvc2VkIENoYW5nZWQgdG8gTm9ybWF0aXZlIENsYXNzaWMgRUNO IGRldGVjdGlvbiBUZXh0DQoNCg0KDQpPbiAxMS8xLzIwMjAgMzo1NSBQTSwgQ2hyaXN0aWFuIEh1 aXRlbWEgd3JvdGU6DQoNCg0KT24gMTEvMS8yMDIwIDE6NTMgUE0sIERlIFNjaGVwcGVyLCBLb2Vu IChOb2tpYSAtIEJFL0FudHdlcnApIHdyb3RlOg0KDQo+PkJ5IHRoZSB3YXksIEkgZG8gdGhpbmsg dGhhdCBMNFMgbWFrZXMgdG9vIG1hbnkgaHlwb3RoZXNlcyBhYm91dCB0aGUgQ0MuIEkgd2lzaCB0 aGUgQVFNIHdhcyBiYXNlZCBvbiBpbnRyaW5zaWMgcHJvcGVydGllcywgc3VjaCBhcyAidGhpcyBm bG93IGlzIHVzaW5nIG1vcmUgdGhhbiBpdHMgZmFpciBzaGFyZSBvZiBiYW5kd2lkdGgiDQoNClRo aXMgbGFzdCBxdW90ZSBpcyBleGFjdGx5IHdoYXQgdGhlIEw0UyBtYXJraW5nIGlzIGludGVuZGVk IHRvIGRvOiB0aGUgc21vb3RoZWQgbWFya2luZyBwcm9iYWJpbGl0eSBvdmVyIGEgbG9uZ2VyIHRp bWUgaW50ZXJ2YWwgKGFib3V0IDEwIFJUVHMpIGluZGljYXRlcyB0aGUgZmFpciB0YXJnZXQgcmF0 ZS4gQXNzdW1pbmcgd2UgYWdyZWUgb24gMTVtcyBiZWluZyB0aGUgcmVmZXJlbmNlIFJUVCwgdGhl biB0aGUgYXZlcmFnZSByYXRlIGNhbiBiZSByPTIvKHAqMC4wMTUpPTEzMy9wIHBhY2tldHMgcGVy IHNlY29uZC4gU28geW91IGNhbiBldmFsdWF0ZSB5b3Vyc2VsZiBpZiB5b3UgdXNlIG1vcmUgb3Ig bGVzcyB0aGFuIHRoZSBmYWlyIHNoYXJlLiBIb3cgeW91IHJlc3BvbmQgdG8gdGhpcyBrbm93bGVk Z2UgaXMgdXAgdG8geW91ciBDQy4NCg0KSSB0aGluayB0aGF0IHBhcnQgb2YgdGhlIGNvbXBsZXhp dHkgaW4gTDRTIEFRTSBjb21lcyBmcm9tIHRyeWluZyB0byBhY2hpZXZlIGZhaXIgc2hhcmluZyBv ZiByZXNvdXJjZSB3aXRob3V0IGFjdHVhbGx5IGltcGxlbWVudGluZyBmYWlyIHF1ZXVpbmcuIFlv dSBlbmQgdXAgZXhwZWN0aW5nIHRoYXQgaW1wbGVtZW50YXRpb25zIGFwcGx5IGEgZm9ybXVsYSBh bmQgZGVyaXZlIHRoZWlyIHBhY2tldCBzZW5kaW5nIHJhdGUgZnJvbSBhIHNtb290aGVkIGF2ZXJh Z2Ugb2YgdGhlIHBhY2tldCBtYXJraW5nIHJhdGUuIEl0IG1heSB3b3JrIHdlbGwgaW4gY29udHJv bGxlZCBlbnZpcm9ubWVudHMsIGJ1dCBJIGFtIGFmcmFpZCB0aGF0IHZlcnkgZmV3IGltcGxlbWVu dGF0aW9ucyBhcmUgZXZlciBnb2luZyB0byBkbyB0aGF0IGluIHByYWN0aWNlLg0KDQpDb25nZXN0 aW9uIENvbnRyb2wgYWxnb3JpdGhtcyBhcmUgbXVjaCBtb3JlIGxpa2VseSB0byB0cmVhdCBwYWNr ZXQgbG9zc2VzLCBtYXJraW5ncywgb3IgZGVsYXkgaW5jcmVhc2VzIGFzIHNpZ25hbHMgdGhhdCB0 aGV5IGFyZSBzZW5kaW5nIHRvbyBmYXN0LCB3aGlsZSBjb25zaWRlcmluZyB0aGVpciBhYnNlbmNl IG9yIGxvd2VyIGZyZXF1ZW5jeSBhcyBzaWduYWwgdGhhdCB0aGV5IG1pZ2h0IGJlIGFibGUgdG8g c2VuZCBmYXN0ZXIuIEltcGxlbWVudGF0aW9ucyBhbHNvIGFyZSBsaWtlbHkgdG8gdGFrZSBpbnRv IGFjY291bnQgdGhlIHJlY2VudCBoaXN0b3J5LCBmb3IgZXhhbXBsZSBtb25pdG9yaW5nIHRoZSBt aW4gUlRUIChMRURCQVQsIEJCUiksIG9yIG1vbml0b3JpbmcgdGhlIHJlY2VudCBhdmFpbGFibGUg YmFuZHdpZHRoIChCQlIpLiBUaGV5IGFsc28gdHlwaWNhbGx5IGZpbHRlciBzaWduYWxzLCBlLmcu LCBjb25zaWRlciBvbmx5IG9uZSBwYWNrZXQgbG9zcyBvciBFQ04gbWFyayBwZXIgUlRUIChOZXcg UmVubywgQ3ViaWMsIGV0Yy4pLiBUaGlzIGlzIGV4cGVjdGVkLCBiZWNhdXNlIHdlIGRvIHNlZSBw YWNrZXQgbG9zc2VzIGFycml2aW5nIGluIGJhdGNoZXMsIGFuZCBjYW4gcmVhc29uYWJseSBleHBl Y3QgRUNOIG1hcmtzIHRvIGhhdmUgdGhlIHNhbWUgYmVoYXZpb3IuIEJ1dCBpbXBsZW1lbnRhdGlv bnMgYXJlIGluaGVyZW50bHkgc2VsZmlzaCwgZG8gbm90IHJlYWxseSB0cnVzdCB0aGUgc2lnbmFs cyBmcm9tIHRoZSBuZXR3b3JrLCBhbmQgYXJlIHZlcnkgdW5saWtlbHkgdG8gZW5kIHVwIGFwcGx5 aW5nIHRoZSBmb3JtdWxhIHRoYXQgeW91IHN1Z2dlc3QuDQoNCkluIGFueSBjYXNlLCB5b3UgaGF2 ZSBhIHNjYWxpbmcgaXNzdWUuIExldCdzIGNvbnNpZGVyIGEgMS41R2JwcyBsaW5rLCB3aXRoIDE1 IG1zIGRlbGF5IGFuZCAxNTAwIGJ5dGVzIHBhY2tldHMuIFRoZSBub21pbmFsIHNlbmRpbmcgcmF0 ZSBpcyAxMjUsMDAwIHBhY2tldHMgcGVyIHNlY29uZC4gVGhlIG1hcmtpbmcgcmF0ZSB3aXRoIHlv dXIgZm9ybXVsYSBzaGFsbCBiZSBwID0gMi8ociowLjAxNSksIHdoaWNoIGlzIGFib3V0IDAuMDAw OCUuIE92ZXIgdGhlIGxhc3QgMTAgUlRULCB0aGUgY29ubmVjdGlvbiB3aWxsIG9uIGF2ZXJhZ2Ug c2VlIDAuMTQgbWFya3MgLS0gdGhhdCBpcywgbm8gbWFyayBvdmVyIHRoZSBsYXN0IDEwIFJUVCA4 NiUgb2YgdGhlIHRpbWUuIFRoaXMgZmFsbHMgd2VsbCBzaG9ydCBvZiB0aGUgcmVxdWlyZW1lbnQg dG8gcHJvdmlkZSBmcmVxdWVudCBmZWVkYmFjayENCg0KU29ycnksIGJ1ZyBpbiBteSBzcHJlYWRz aGVldC4gVGhlIG1hcmtpbmcgcmF0ZSBpcyBhY3R1YWxseSAwLjElLCBhYm91dCAyIHBhY2tldHMg cGVyIFJUVC4gU3RpbGwgbm90IGZyZXF1ZW50IGVub3VnaCBmb3IgbXkgdGFzdGUsIGJ1dCBtdWNo IGJldHRlciB0aGFuIDAuMDAwOCUuIE5ldmVydGhlbGVzcywgdGhlIGludmVyc2UgcmVsYXRpb24g YmV0d2VlbiBtYXJraW5nIHJhdGUgYW5kIGRhdGEgcmF0ZSBpcyBub3QgZ3JlYXQuDQoNCklmIHlv dSBnaXZlIHVwIHRoZSBmb3JtdWxhLCB5b3UgY291bGQgZW5kIHVwIHdpdGggYSBzaWduaWZpY2Fu dGx5IHNpbXBsZXIgZXZlbnQgYmFzZWQgY29uZ2VzdGlvbiBjb250cm9sLCBpbiB3aGljaCBhcHBs aWNhdGlvbnMgc2xvdyBkb3duIGlmIHRoZXkgc2VlIHRvbyBtYW55IENFIG1hcmtzLCBhY2NlbGVy YXRlIGlmIHRoZXkgb25seSBzZWUgYSB2ZXJ5IGxvdyBtYXJraW5nIHJhdGUsIGFuZCBzdGF5IHB1 dCBpZiB0aGV5IHNlZSBhbiBpbmZyZXF1ZW50IG1hcmtpbmcgcmF0ZS4gSSBjYW4gc2VlIGFwcGxp Y2F0aW9ucyBkb2luZyB0aGF0LCByZXN1bHRpbmcgaW4gYSBmYWlybHkgc3RhYmxlIG5ldHdvcmsg YW5kIGxvdyBkZWxheXMuIE9mIGNvdXJzZSwgeW91IHdpbGwgaGF2ZSB0byBkbyBzb21lIGZvcm0g b2YgZmFpciBzaGFyZSBlbmZvcmNlbWVudCB0byBjYXRjaCB2aW9sYXRvcnMsIGJ1dCBJIGFtIHN1 cmUgeW91IGNhbiBjb21lIHVwIHdpdGggc29tZXRoaW5nLg0KDQotLSBDaHJpc3RpYW4gSHVpdGVt YQ0KDQoNCg0KDQoNCg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXw0KDQp0Y3BQcmFndWUgbWFpbGluZyBsaXN0DQoNCnRjcFByYWd1ZUBpZXRmLm9yZzxt YWlsdG86dGNwUHJhZ3VlQGlldGYub3JnPg0KDQpodHRwczovL3d3dy5pZXRmLm9yZy9tYWlsbWFu L2xpc3RpbmZvL3RjcHByYWd1ZQ0K --_000_AM0PR07MB6114D78B68E07B2F712E12E4B9100AM0PR07MB6114eurp_ Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxl PjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6 IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1m YWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJcGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAy IDQ7fQ0KQGZvbnQtZmFjZQ0KCXtmb250LWZhbWlseTpDb25zb2xhczsNCglwYW5vc2UtMToyIDEx IDYgOSAyIDIgNCAzIDIgNDt9DQovKiBTdHlsZSBEZWZpbml0aW9ucyAqLw0KcC5Nc29Ob3JtYWws IGxpLk1zb05vcm1hbCwgZGl2Lk1zb05vcm1hbA0KCXttYXJnaW46MGNtOw0KCW1hcmdpbi1ib3R0 b206LjAwMDFwdDsNCglmb250LXNpemU6MTEuMHB0Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJpIixz YW5zLXNlcmlmO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNvLXN0eWxlLXByaW9y aXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQpwcmUN Cgl7bXNvLXN0eWxlLXByaW9yaXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOiJIVE1MIFByZWZvcm1h dHRlZCBDaGFyIjsNCgltYXJnaW46MGNtOw0KCW1hcmdpbi1ib3R0b206LjAwMDFwdDsNCglmb250 LXNpemU6MTAuMHB0Ow0KCWZvbnQtZmFtaWx5OiJDb3VyaWVyIE5ldyI7fQ0Kc3Bhbi5IVE1MUHJl Zm9ybWF0dGVkQ2hhcg0KCXttc28tc3R5bGUtbmFtZToiSFRNTCBQcmVmb3JtYXR0ZWQgQ2hhciI7 DQoJbXNvLXN0eWxlLXByaW9yaXR5Ojk5Ow0KCW1zby1zdHlsZS1saW5rOiJIVE1MIFByZWZvcm1h dHRlZCI7DQoJZm9udC1mYW1pbHk6Q29uc29sYXM7fQ0Kc3Bhbi5FbWFpbFN0eWxlMjANCgl7bXNv LXN0eWxlLXR5cGU6cGVyc29uYWwtcmVwbHk7DQoJZm9udC1mYW1pbHk6IkNhbGlicmkiLHNhbnMt c2VyaWY7DQoJY29sb3I6d2luZG93dGV4dDt9DQouTXNvQ2hwRGVmYXVsdA0KCXttc28tc3R5bGUt dHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAuMHB0O30NCkBwYWdlIFdvcmRTZWN0aW9u MQ0KCXtzaXplOjYxMi4wcHQgNzkyLjBwdDsNCgltYXJnaW46NzIuMHB0IDcyLjBwdCA3Mi4wcHQg NzIuMHB0O30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0KLS0+PC9z dHlsZT48IS0tW2lmIGd0ZSBtc28gOV0+PHhtbD4NCjxvOnNoYXBlZGVmYXVsdHMgdjpleHQ9ImVk aXQiIHNwaWRtYXg9IjEwMjYiIC8+DQo8L3htbD48IVtlbmRpZl0tLT48IS0tW2lmIGd0ZSBtc28g OV0+PHhtbD4NCjxvOnNoYXBlbGF5b3V0IHY6ZXh0PSJlZGl0Ij4NCjxvOmlkbWFwIHY6ZXh0PSJl ZGl0IiBkYXRhPSIxIiAvPg0KPC9vOnNoYXBlbGF5b3V0PjwveG1sPjwhW2VuZGlmXS0tPg0KPC9o ZWFkPg0KPGJvZHkgbGFuZz0iRU4tVVMiIGxpbms9ImJsdWUiIHZsaW5rPSJwdXJwbGUiPg0KPGRp diBjbGFzcz0iV29yZFNlY3Rpb24xIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkhpIENocmlzdGlh biw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+ PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+SW5kZWVkIGEgdmVyeSBpbXBvcnRhbnQgZGlzY3Vz c2lvbiB3ZSBuZWVkIHRvIGhhdmUsIGFuZCBhZ3JlZSBvbi48bzpwPjwvbzpwPjwvcD4NCjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1h bCI+Jmd0OyZndDsgSWYgeW91IGdpdmUgdXAgdGhlIGZvcm11bGEsIHlvdSBjb3VsZCBlbmQgdXAg d2l0aCBhIHNpZ25pZmljYW50bHkgc2ltcGxlciBldmVudCBiYXNlZCBjb25nZXN0aW9uIGNvbnRy b2wsIGluIHdoaWNoIGFwcGxpY2F0aW9ucyBzbG93IGRvd24gaWYgdGhleSBzZWUgdG9vIG1hbnkg Q0UgbWFya3MsIGFjY2VsZXJhdGUgaWYgdGhleSBvbmx5IHNlZSBhIHZlcnkgbG93IG1hcmtpbmcg cmF0ZSwgYW5kIHN0YXkgcHV0DQogaWYgdGhleSBzZWUgYW4gaW5mcmVxdWVudCBtYXJraW5nIHJh dGUuPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpw PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPlRoaXMgaXMgZXhhY3RseSB3aGF0IEkgZXhwZWN0 IG1vc3QgQ0NzIHdpbGwgZG86IHdvcmsgZXZlbnQgYmFzZWQuIFRoZSBmb3JtdWxhIGlzIG9ubHkg dG8gdHVuZSB0aGUgZXZlbnRzIHNvIGl0IGNvbnZlcmdlcyB0byB0aGUgZm9ybXVsYSAoZXZlbnR1 YWxseSkuIElmIHRoZXJlIGlzIGEgbG90IG9mIGR5bmFtaXNtLCBvYnZpb3VzbHkgc2hvcnRlciBS VFRzIGNhbiByZXNwb25kIGZhc3RlciBhbmQgd2lsbCBiZW5lZml0DQogZnJvbSB0aGF0LiBUaGV5 IHdpbGwgaGF2ZSBtZW1vcnkgb3ZlciBhIHNob3J0ZXIgdGltZSBpbnRlcnZhbCwgYW5kIG9uIGF2 ZXJhZ2UgbW9yZSB0aHJvdWdocHV0IHRoYW4gYSBsb25nZXIgUlRUIGZsb3cuIEJ1dCBpZiB0aGlu Z3MgYXJlIHN0YWJsZSwgaXQgaXMgZ29vZCB0byBjb252ZXJnZSB0byBhIGtub3duIHJhdGUuIEkg ZG9u4oCZdCBzZWUgYW55IGJldHRlciBzb2x1dGlvbiwgZG8geW91PyBJZiB3ZSBkb27igJl0IGZv cmVzZWUgc3VjaCBhIG1lY2hhbmlzbQ0KIGhvdyBkbyB5b3UgZXhwZWN0IGJhbmR3aWR0aCB0byBi ZSBzaGFyZWQgb3ZlciBzaGFyZWQgaW5mcmFzdHJ1Y3R1cmU/IFRoZW4gdGhlIHdob2xlIGRpc2N1 c3Npb24gb2YgZmFpcm5lc3MgY2FuIGJlIGRpc21pc3NlZCwgYW5kIENDcyBzaG91bGQgbm90IGJl IGV2YWx1YXRlZCBmb3IgdGhpcyBuZWl0aGVyLjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj4mZ3Q7 Jmd0OyBUaGUgbWFya2luZyByYXRlIGlzIGFjdHVhbGx5IDAuMSUsIGFib3V0IDIgcGFja2V0cyBw ZXIgUlRUPG86cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwv bzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkluZGVlZCwgdGhpcyBpcyBzdGlsbCAyIG1h cmtzIHBlciAxNW1zLCBtdWNoIGJldHRlciB0aGFuIDEgZHJvcC9tYXJrIHBlciAxMDAwIFJUVHMg KDE1IHNlY29uZHMpIGZvciB5b3VyIGV4YW1wbGUgd2hlbiB1c2luZyBSZW5vIChDdWJpYyB3aWxs IGFsc28gYmUgc2Vjb25kcykuIE9ubHkgaWYgdGhlIFJUVCBpcyBtdWNoIHNtYWxsZXIgdGhhbiAx NW1zLCBpdCB3aWxsIGdldCBsZXNzIG1hcmtzIHBlciBSVFQsIGJ1dA0KIGluIG91ciBMaW51eCBQ cmFndWUgd2UgaGF2ZSBhbiBvcHRpb24gdG8gZ3JhZHVhbGx5IHRha2UgdGhlIFJUVCBkZXBlbmRl bmNlIGludG8gYWNjb3VudCwgZGVwZW5kaW5nIG9uIGhvdyBsb25nIHRoZSBmbG93IGlzIHJ1bm5p bmcgYXBwbGljYXRpb24gdW5saW1pdGVkIChvciBieSBleHRlbnNpb24gaG93IGxvbmcgdGhpbmdz IGFyZSBzdGFibGUpLjxvOnA+PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4m bmJzcDs8L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5BbHNvIHN1Y2ggYSBmb3JtdWxh IGlzIGp1c3QgYSByZWZlcmVuY2UgdG8gYWltIGF0LiBJdCBzaG91bGQgbm90IGJlIGEgY29udGlu dW91cyBoYXJkIHJlcXVpcmVtZW50LiBUaGVyZSBhcmUgcmVhc29ucyB3aGVuIGEg4oCcZmxvd+KA nSBjYW4gZGV2aWF0ZSBmcm9tIGl0IChMZXNzIHRoYW4gYmVzdCBlZmZvcnQgY2FuIHRha2UgbGVz cywgYSB0dW5uZWwgb2YgbXVsdGlwbGUgZmxvd3Mgd2lsbCB0YWtlIGFzIG1hbnkgdGltZXMNCiBt b3JlIGFzIHRoZXJlIGFyZSBmbG93cyBpbnNpZGUsIGFuZCBhIHJlYWwtdGltZSBhcHBsaWNhdGlv biBtaWdodCB0ZW1wb3JhcnkgdGFrZSBtb3JlIHdoZW4gbmVlZGVkKS48bzpwPjwvbzpwPjwvcD4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1z b05vcm1hbCI+SeKAmW0gaW4gZmF2b3Igb2Ygc3BlY2lmeWluZyBleHBsaWNpdGx5IHN1Y2ggYSBm b3JtdWxhIGFzIGEgdGFyZ2V0IGZvciBMNFMgQ0PigJlzIHRvIHNoYXJlIEJXIGluIHN0ZWFkeSBz dGF0ZSBzaXR1YXRpb25zIChsb25nIGZsb3dzIGFuZCBzdGFibGUgdGhyb3VnaHB1dCkuIFVwIHRv IG5vdyBJIGRvbuKAmXQgdGhpbmsgd2UgaGF2ZSBzdWNoIGFuIGV4cGxpY2l0IHJlcXVpcmVtZW50 IGluIHRoZSBkcmFmdHMuIEFueW9uZQ0KIGVsc2UgdGhhdCB3b3VsZCBzdXBwb3J0IHRoaXM/IEF0 IGxlYXN0IENDIGNvbnRyb2wgZGV2ZWxvcGVycyB0aGVuIGhhdmUgYSB0YXJnZXQgdG8gYWltIGZv ciBhbmQgYSBjbGVhciBldmFsdWF0aW9uIGNyaXRlcml1bS48bzpwPjwvbzpwPjwvcD4NCjxwIGNs YXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1h bCI+UmVnYXJkcyw8bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPktvZW4uPG86 cD48L286cD48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4N CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPGRpdj4NCjxkaXYg c3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci10b3A6c29saWQgI0UxRTFFMSAxLjBwdDtwYWRkaW5n OjMuMHB0IDBjbSAwY20gMGNtIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxiPkZyb206PC9iPiBD aHJpc3RpYW4gSHVpdGVtYSAmbHQ7aHVpdGVtYUBodWl0ZW1hLm5ldCZndDsgPGJyPg0KPGI+U2Vu dDo8L2I+IE1vbmRheSwgTm92ZW1iZXIgMiwgMjAyMCAxOjExIEFNPGJyPg0KPGI+VG86PC9iPiBE ZSBTY2hlcHBlciwgS29lbiAoTm9raWEgLSBCRS9BbnR3ZXJwKSAmbHQ7a29lbi5kZV9zY2hlcHBl ckBub2tpYS1iZWxsLWxhYnMuY29tJmd0OzsgSm9uYXRoYW4gTW9ydG9uICZsdDtjaHJvbWF0aXg5 OUBnbWFpbC5jb20mZ3Q7PGJyPg0KPGI+Q2M6PC9iPiBpY2NyZyBJUlRGIGxpc3QgJmx0O2ljY3Jn QGlydGYub3JnJmd0OzsgQm9iIEJyaXNjb2UgJmx0O2lldGZAYm9iYnJpc2NvZS5uZXQmZ3Q7OyB0 c3Z3ZyBJRVRGIGxpc3QgJmx0O3RzdndnQGlldGYub3JnJmd0OzsgVENQIFByYWd1ZSBMaXN0ICZs dDt0Y3BQcmFndWVAaWV0Zi5vcmcmZ3Q7PGJyPg0KPGI+U3ViamVjdDo8L2I+IFJlOiBbdGNwUHJh Z3VlXSBbdHN2d2ddIGVjbi1sNHMtaWQ6IFByb3Bvc2VkIENoYW5nZWQgdG8gTm9ybWF0aXZlIENs YXNzaWMgRUNOIGRldGVjdGlvbiBUZXh0PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0K PHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8cD48bzpwPiZuYnNw OzwvbzpwPjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5PbiAxMS8xLzIwMjAgMzo1 NSBQTSwgQ2hyaXN0aWFuIEh1aXRlbWEgd3JvdGU6PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxi bG9ja3F1b3RlIHN0eWxlPSJtYXJnaW4tdG9wOjUuMHB0O21hcmdpbi1ib3R0b206NS4wcHQiPg0K PHA+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+T24g MTEvMS8yMDIwIDE6NTMgUE0sIERlIFNjaGVwcGVyLCBLb2VuIChOb2tpYSAtIEJFL0FudHdlcnAp IHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0ibWFyZ2lu LXRvcDo1LjBwdDttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxwPiZndDsmZ3Q7QnkgdGhlIHdheSwg SSBkbyB0aGluayB0aGF0IEw0UyBtYWtlcyB0b28gbWFueSBoeXBvdGhlc2VzIGFib3V0IHRoZSBD Qy4gSSB3aXNoIHRoZSBBUU0gd2FzIGJhc2VkIG9uIGludHJpbnNpYyBwcm9wZXJ0aWVzLCBzdWNo IGFzICZxdW90O3RoaXMgZmxvdyBpcyB1c2luZyBtb3JlIHRoYW4gaXRzIGZhaXIgc2hhcmUgb2Yg YmFuZHdpZHRoJnF1b3Q7DQo8bzpwPjwvbzpwPjwvcD4NCjxwPlRoaXMgbGFzdCBxdW90ZSBpcyBl eGFjdGx5IHdoYXQgdGhlIEw0UyBtYXJraW5nIGlzIGludGVuZGVkIHRvIGRvOiB0aGUgc21vb3Ro ZWQgbWFya2luZyBwcm9iYWJpbGl0eSBvdmVyIGEgbG9uZ2VyIHRpbWUgaW50ZXJ2YWwgKGFib3V0 IDEwIFJUVHMpIGluZGljYXRlcyB0aGUgZmFpciB0YXJnZXQgcmF0ZS4gQXNzdW1pbmcgd2UgYWdy ZWUgb24gMTVtcyBiZWluZyB0aGUgcmVmZXJlbmNlIFJUVCwgdGhlbiB0aGUgYXZlcmFnZSByYXRl IGNhbiBiZQ0KIHI9Mi8ocCowLjAxNSk9MTMzL3AgcGFja2V0cyBwZXIgc2Vjb25kLiBTbyB5b3Ug Y2FuIGV2YWx1YXRlIHlvdXJzZWxmIGlmIHlvdSB1c2UgbW9yZSBvciBsZXNzIHRoYW4gdGhlIGZh aXIgc2hhcmUuIEhvdyB5b3UgcmVzcG9uZCB0byB0aGlzIGtub3dsZWRnZSBpcyB1cCB0byB5b3Vy IENDLjxvOnA+PC9vOnA+PC9wPg0KPC9ibG9ja3F1b3RlPg0KPHA+SSB0aGluayB0aGF0IHBhcnQg b2YgdGhlIGNvbXBsZXhpdHkgaW4gTDRTIEFRTSBjb21lcyBmcm9tIHRyeWluZyB0byBhY2hpZXZl IGZhaXIgc2hhcmluZyBvZiByZXNvdXJjZSB3aXRob3V0IGFjdHVhbGx5IGltcGxlbWVudGluZyBm YWlyIHF1ZXVpbmcuIFlvdSBlbmQgdXAgZXhwZWN0aW5nIHRoYXQgaW1wbGVtZW50YXRpb25zIGFw cGx5IGEgZm9ybXVsYSBhbmQgZGVyaXZlIHRoZWlyIHBhY2tldCBzZW5kaW5nIHJhdGUgZnJvbSBh IHNtb290aGVkDQogYXZlcmFnZSBvZiB0aGUgcGFja2V0IG1hcmtpbmcgcmF0ZS4gSXQgbWF5IHdv cmsgd2VsbCBpbiBjb250cm9sbGVkIGVudmlyb25tZW50cywgYnV0IEkgYW0gYWZyYWlkIHRoYXQg dmVyeSBmZXcgaW1wbGVtZW50YXRpb25zIGFyZSBldmVyIGdvaW5nIHRvIGRvIHRoYXQgaW4gcHJh Y3RpY2UuDQo8bzpwPjwvbzpwPjwvcD4NCjxwPkNvbmdlc3Rpb24gQ29udHJvbCBhbGdvcml0aG1z IGFyZSBtdWNoIG1vcmUgbGlrZWx5IHRvIHRyZWF0IHBhY2tldCBsb3NzZXMsIG1hcmtpbmdzLCBv ciBkZWxheSBpbmNyZWFzZXMgYXMgc2lnbmFscyB0aGF0IHRoZXkgYXJlIHNlbmRpbmcgdG9vIGZh c3QsIHdoaWxlIGNvbnNpZGVyaW5nIHRoZWlyIGFic2VuY2Ugb3IgbG93ZXIgZnJlcXVlbmN5IGFz IHNpZ25hbCB0aGF0IHRoZXkgbWlnaHQgYmUgYWJsZSB0byBzZW5kIGZhc3Rlci4gSW1wbGVtZW50 YXRpb25zDQogYWxzbyBhcmUgbGlrZWx5IHRvIHRha2UgaW50byBhY2NvdW50IHRoZSByZWNlbnQg aGlzdG9yeSwgZm9yIGV4YW1wbGUgbW9uaXRvcmluZyB0aGUgbWluIFJUVCAoTEVEQkFULCBCQlIp LCBvciBtb25pdG9yaW5nIHRoZSByZWNlbnQgYXZhaWxhYmxlIGJhbmR3aWR0aCAoQkJSKS4gVGhl eSBhbHNvIHR5cGljYWxseSBmaWx0ZXIgc2lnbmFscywgZS5nLiwgY29uc2lkZXIgb25seSBvbmUg cGFja2V0IGxvc3Mgb3IgRUNOIG1hcmsgcGVyIFJUVCAoTmV3DQogUmVubywgQ3ViaWMsIGV0Yy4p LiBUaGlzIGlzIGV4cGVjdGVkLCBiZWNhdXNlIHdlIGRvIHNlZSBwYWNrZXQgbG9zc2VzIGFycml2 aW5nIGluIGJhdGNoZXMsIGFuZCBjYW4gcmVhc29uYWJseSBleHBlY3QgRUNOIG1hcmtzIHRvIGhh dmUgdGhlIHNhbWUgYmVoYXZpb3IuIEJ1dCBpbXBsZW1lbnRhdGlvbnMgYXJlIGluaGVyZW50bHkg c2VsZmlzaCwgZG8gbm90IHJlYWxseSB0cnVzdCB0aGUgc2lnbmFscyBmcm9tIHRoZSBuZXR3b3Jr LCBhbmQgYXJlDQogdmVyeSB1bmxpa2VseSB0byBlbmQgdXAgYXBwbHlpbmcgdGhlIGZvcm11bGEg dGhhdCB5b3Ugc3VnZ2VzdC48bzpwPjwvbzpwPjwvcD4NCjxwPkluIGFueSBjYXNlLCB5b3UgaGF2 ZSBhIHNjYWxpbmcgaXNzdWUuIExldCdzIGNvbnNpZGVyIGEgMS41R2JwcyBsaW5rLCB3aXRoIDE1 IG1zIGRlbGF5IGFuZCAxNTAwIGJ5dGVzIHBhY2tldHMuIFRoZSBub21pbmFsIHNlbmRpbmcgcmF0 ZSBpcyAxMjUsMDAwIHBhY2tldHMgcGVyIHNlY29uZC4gVGhlIG1hcmtpbmcgcmF0ZSB3aXRoIHlv dXIgZm9ybXVsYSBzaGFsbCBiZSBwID0gMi8ociowLjAxNSksIHdoaWNoIGlzIGFib3V0IDAuMDAw OCUuIE92ZXINCiB0aGUgbGFzdCAxMCBSVFQsIHRoZSBjb25uZWN0aW9uIHdpbGwgb24gYXZlcmFn ZSBzZWUgMC4xNCBtYXJrcyAtLSB0aGF0IGlzLCBubyBtYXJrIG92ZXIgdGhlIGxhc3QgMTAgUlRU IDg2JSBvZiB0aGUgdGltZS4gVGhpcyBmYWxscyB3ZWxsIHNob3J0IG9mIHRoZSByZXF1aXJlbWVu dCB0byBwcm92aWRlIGZyZXF1ZW50IGZlZWRiYWNrITxvOnA+PC9vOnA+PC9wPg0KPC9ibG9ja3F1 b3RlPg0KPHA+U29ycnksIGJ1ZyBpbiBteSBzcHJlYWRzaGVldC4gVGhlIG1hcmtpbmcgcmF0ZSBp cyBhY3R1YWxseSAwLjElLCBhYm91dCAyIHBhY2tldHMgcGVyIFJUVC4gU3RpbGwgbm90IGZyZXF1 ZW50IGVub3VnaCBmb3IgbXkgdGFzdGUsIGJ1dCBtdWNoIGJldHRlciB0aGFuIDAuMDAwOCUuIE5l dmVydGhlbGVzcywgdGhlIGludmVyc2UgcmVsYXRpb24gYmV0d2VlbiBtYXJraW5nIHJhdGUgYW5k IGRhdGEgcmF0ZSBpcyBub3QgZ3JlYXQuPG86cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHls ZT0ibWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxwPklmIHlvdSBnaXZl IHVwIHRoZSBmb3JtdWxhLCB5b3UgY291bGQgZW5kIHVwIHdpdGggYSBzaWduaWZpY2FudGx5IHNp bXBsZXIgZXZlbnQgYmFzZWQgY29uZ2VzdGlvbiBjb250cm9sLCBpbiB3aGljaCBhcHBsaWNhdGlv bnMgc2xvdyBkb3duIGlmIHRoZXkgc2VlIHRvbyBtYW55IENFIG1hcmtzLCBhY2NlbGVyYXRlIGlm IHRoZXkgb25seSBzZWUgYSB2ZXJ5IGxvdyBtYXJraW5nIHJhdGUsIGFuZCBzdGF5IHB1dCBpZiB0 aGV5IHNlZSBhbiBpbmZyZXF1ZW50DQogbWFya2luZyByYXRlLiBJIGNhbiBzZWUgYXBwbGljYXRp b25zIGRvaW5nIHRoYXQsIHJlc3VsdGluZyBpbiBhIGZhaXJseSBzdGFibGUgbmV0d29yayBhbmQg bG93IGRlbGF5cy4gT2YgY291cnNlLCB5b3Ugd2lsbCBoYXZlIHRvIGRvIHNvbWUgZm9ybSBvZiBm YWlyIHNoYXJlIGVuZm9yY2VtZW50IHRvIGNhdGNoIHZpb2xhdG9ycywgYnV0IEkgYW0gc3VyZSB5 b3UgY2FuIGNvbWUgdXAgd2l0aCBzb21ldGhpbmcuPG86cD48L286cD48L3A+DQo8cD4tLSBDaHJp c3RpYW4gSHVpdGVtYTxvOnA+PC9vOnA+PC9wPg0KPHA+PG86cD4mbmJzcDs8L286cD48L3A+DQo8 cD48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxicj4NCjxicj4N CjxvOnA+PC9vOnA+PC9wPg0KPHByZT5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fXzxvOnA+PC9vOnA+PC9wcmU+DQo8cHJlPnRjcFByYWd1ZSBtYWlsaW5nIGxp c3Q8bzpwPjwvbzpwPjwvcHJlPg0KPHByZT48YSBocmVmPSJtYWlsdG86dGNwUHJhZ3VlQGlldGYu b3JnIj50Y3BQcmFndWVAaWV0Zi5vcmc8L2E+PG86cD48L286cD48L3ByZT4NCjxwcmU+PGEgaHJl Zj0iaHR0cHM6Ly93d3cuaWV0Zi5vcmcvbWFpbG1hbi9saXN0aW5mby90Y3BwcmFndWUiPmh0dHBz Oi8vd3d3LmlldGYub3JnL21haWxtYW4vbGlzdGluZm8vdGNwcHJhZ3VlPC9hPjxvOnA+PC9vOnA+ PC9wcmU+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg== --_000_AM0PR07MB6114D78B68E07B2F712E12E4B9100AM0PR07MB6114eurp_-- From nobody Mon Nov 2 04:42:53 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2A53E3A0EFA; Mon, 2 Nov 2020 04:42:45 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.902 X-Spam-Level: X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rv5QmNqpZ800; Mon, 2 Nov 2020 04:42:44 -0800 (PST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80104.outbound.protection.outlook.com [40.107.8.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6522F3A0EF6; Mon, 2 Nov 2020 04:42:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gNoS/IHJDX/fnt+VizSWYPs0cQzPcRDT6UQ3EqoVRdl583PbnwhBZE3D2iJscupn2w5m/TsC1AkKlQaDnE3N+7UY4Lxn4pths/vJIkfamV1MsG3hmL7IW5UtmCoBXkkA2RvGvY10D/3DqnX4Y97pJayzflG/sC5xVG2/w8IcfM7t7Ed8LTDtge7XDO/UtZ74y3hVPl7eERFkjg9lVctbLHxZhu9OBxZeKjR1vruwKWDLkl7yaCZJ3GJuSmEUU+txE5CczpY/C8oe/cbstqkHJgEWbcrIUNrFjKydjJ9a8avsp+1MfduCHvkWgstj2undXt7Frejy9pZ05XRsPvpQvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=di6f9QS0yHUUgPERN7AAh11pz3hvtqIQ5SLfTaVCDwg=; b=kcRYZ4oe9YSNDboE2t82bfgYkLhnAIUUNNHDxe0UYGq7e5sPx8TZZcQ4cWnRJycBh2Qf5W/nNMlgffKn4F4s+uVtnbzq4UYozzSlbzE2DXKFBQBitXKY/mNdvQQXq7PVrE/K+E3RSIO4IauAgtf1rUYDSVQjPjeyzm/Ov4gSMETtEuB4DuL0/pGj3V8rlfBotA0hySyL9foIJncZB2/FCOtPwIWvvdETeOlL9qOiH17djRJBcskALf6PqrnYLGmn0lFynU1sEvEZehzbJs/P5ryhvycp4LMa/0efE0cLTZ/xL/IsQrxN3pL12qaJLqb5ueV77ngteng9Uyirr+E+4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-bell-labs.com; dmarc=pass action=none header.from=nokia-bell-labs.com; dkim=pass header.d=nokia-bell-labs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=di6f9QS0yHUUgPERN7AAh11pz3hvtqIQ5SLfTaVCDwg=; b=wdlwBXU+UoGke0gHNYea5Tg8R5VGPU8OwgdCeBwv02zdD1RRLTD4sPtmHOXJ9zKLOWAmLaRXAX6NgvlR59WsaKOxgPi2SA3FF76fUnBKe6NKuCiWezifo4OmDFWFot9+1O0K8fUlxaCrNmSXLwLT9A2eAyO1JEQDbAArrjIjDWs= Received: from AM0PR07MB6114.eurprd07.prod.outlook.com (2603:10a6:208:113::33) by AM0PR0702MB3556.eurprd07.prod.outlook.com (2603:10a6:208:24::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.10; Mon, 2 Nov 2020 12:42:40 +0000 Received: from AM0PR07MB6114.eurprd07.prod.outlook.com ([fe80::a9f4:9199:8226:d6c6]) by AM0PR07MB6114.eurprd07.prod.outlook.com ([fe80::a9f4:9199:8226:d6c6%6]) with mapi id 15.20.3541.010; Mon, 2 Nov 2020 12:42:40 +0000 From: "De Schepper, Koen (Nokia - BE/Antwerp)" To: Jonathan Morton , Christian Huitema CC: iccrg IRTF list , Bob Briscoe , tsvwg IETF list , TCP Prague List Thread-Topic: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text Thread-Index: AQHWsIfwC/FrblPY50a/ND1AVr6ZVKmzwIVwgAAzdwCAAAQwgIAAC4kAgADEkjA= Date: Mon, 2 Nov 2020 12:42:40 +0000 Message-ID: References: <1b71a610-75ea-e1d4-e3ce-f0ae6a4c12f7@bobbriscoe.net> <28247e5f-5df3-1f75-50e6-b4a7e80d5ab0@huitema.net> <61B6E9DC-8786-4CC0-9AC0-7F74E3529A73@gmail.com> <40751956-daa0-5693-d880-c45e8ca262cc@huitema.net> <5ba51c98-9600-a55f-5d9f-bff012eb39e4@huitema.net> <84E29EA3-67DF-4130-8E92-940D5A4B19DA@gmail.com> In-Reply-To: <84E29EA3-67DF-4130-8E92-940D5A4B19DA@gmail.com> Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=nokia-bell-labs.com; x-originating-ip: [2a02:1810:1e00:cb00:106c:a742:663:cd5d] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 9c17f3a0-4a92-4d23-ba01-08d87f2cc973 x-ms-traffictypediagnostic: AM0PR0702MB3556: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3513; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: JkvdiZdExfTkc6RWsrPsi0zMRFIAp6Ii67AWxA/n9hAe5T2NaKngJLrQ0/AXwNZGAj/1jWlbuXOTBLr8vt+IuSxvsWgRQ+aIq4ESgZ8jsYvns3d7V2Ychdll/PcfLXzd9Aym3v+Tm+ntaVlYQoj04wh/Oqc4aiwznsuC+h4CJFDXziYRXosP8F43w/AWXonxi/4Q6OSTo+OtMYUw2eJCZkFd16NginLbI0kjwJxGkRl4b1sGs4EfQeJtFg6nkHSqZ7ER/mFXvDFqYhR+fiL/3NWCXoLysAC9D4kULH5lVk22MXujvSV4gLtdaiS9e+KY x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR07MB6114.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(346002)(376002)(396003)(136003)(366004)(8676002)(6506007)(7696005)(53546011)(186003)(2906002)(4326008)(8936002)(52536014)(5660300002)(71200400001)(66446008)(64756008)(86362001)(66556008)(66476007)(66946007)(76116006)(33656002)(83380400001)(54906003)(316002)(478600001)(110136005)(9686003)(55016002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: e9hfeVlv255X4qwxaRG73VRVCxTZtaJv3hrAQHmbHCoyVGSXBf433Yf/Rrz6g2NaChRG5AiYY8PZODGNf4+TDU3UD1duxSN2JYNN61uCCeLYvpBtj3jZIwAHtm5V/SFET1Im6Mv+E3Y0gxA9bYrTn0hO1GS4hssgOPcEcv3l48ukBsCLaiCpBliVEgVUhToFzY+PtFzkDQI4237xZl4iEY2Fv/9Xs4W8Pt28oJRiVsFKSr0zQHm8ToXuc1dBdW3TTMZljiMqaW0FdI6nUs7b5+xjJTXbGKvpcRh6EwKwII+jcueubXSr/5MGU+leCpyey1vKljqR4LHr9FYia1jFWKhjg5CQJWnz8DmsENn7uBdMOAhgd3B2vIH/41U2rNX4mCJkne1kTIMrMrbII/a7gn3JAJSHrzV8F5WFNScpvONvsl8JKvPqLEV3FHjvj+6Xuy3jEN4n9botd57D7Ko/LToqsVTnpgQ3aUpnyhsMl20HbDKklNtexNId6ratIe4/g4a7LdfiU+eqrrbrsJ1m0TGgjdleS8q3e3W1SfhNUcv/h+BW76pdXJ8vNDAQAxJI1U12VHgWRcOjDvkN8BPPNNBgYjx6c5WTYVGeuQHk966ARBnENXCnAeCaEKt4JXtaTownDb0M5gQMZfBKvoQlE5WCrFxywF1NlsLWQtZj0jjzmkzZZMGprVmB7p3jDOxDX0ETDGvqlvUpsveMtaM9Nw== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR07MB6114.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9c17f3a0-4a92-4d23-ba01-08d87f2cc973 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Nov 2020 12:42:40.6509 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ftImQVxoadVPsmjOK4hdDSvdbvn4y/uDTYAHbX6YAAN70vy9uZ/o25M7ybA/y5vIAZA1orBHEnqqLL/QwVRNlCCwWl7uBrr+qryx+lgvhU4U1Iwouc4qGuC3mKtR2ekY X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0702MB3556 Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2020 12:42:45 -0000 Hi Jonathan, Indeed, FQ is ensuring (actually enforcing) per 5 tuple equal BW. Do you su= ggest we need to have FQ per 5 tuple in every bottleneck? This would indeed= be a simple solution for CC developers. But not sure if every NW box would= like to implement this functionality. From the moment flows are treated as= an aggregate, the rates that CC's will get is exactly based on their respe= ctive behavior (say their formula), and if they are vastly different, so wi= ll their rates be... As long as there is no consensus about universal FQ, I think we need to dis= cuss what the formula should be. Regards, Koen. -----Original Message----- From: Jonathan Morton =20 Sent: Monday, November 2, 2020 1:52 AM To: Christian Huitema Cc: De Schepper, Koen (Nokia - BE/Antwerp) ; iccrg IRTF list ; Bob Briscoe = ; tsvwg IETF list ; TCP Prague List Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative = Classic ECN detection Text > On 2 Nov, 2020, at 2:10 am, Christian Huitema wrote= : >=20 >> In any case, you have a scaling issue. Let's consider a 1.5Gbps link, wi= th 15 ms delay and 1500 bytes packets. The nominal sending rate is 125,000 = packets per second. The marking rate with your formula shall be p =3D 2/(r*= 0.015), which is about 0.0008%. Over the last 10 RTT, the connection will o= n average see 0.14 marks -- that is, no mark over the last 10 RTT 86% of th= e time. This falls well short of the requirement to provide frequent feedba= ck! >=20 > Sorry, bug in my spreadsheet. The marking rate is actually 0.1%, about 2 = packets per RTT. Still not frequent enough for my taste, but much better th= an 0.0008%. Nevertheless, the inverse relation between marking rate and dat= a rate is not great. A property of both Reno and DCTCP is that a particular CE marking probabili= ty results in a particular average cwnd, over some reasonably long period. = The relationship formula is of course very different for each one, but tha= t qualitative rule happens to be the same. It is a property that DualQ's c= oupled AQM design relies on. CUBIC is a little different in its high-BDP r= egime, but acts like Reno otherwise. What this means is that competing flows experiencing the same marking proba= bility will converge to the same cwnd, and their relative throughputs will = be inversely proportional to their effective RTTs. This convergence is not= necessarily very fast, but given a single queue and a single AQM, it's abo= ut as good as you can expect. When you have an AQM per flow, you can do a bit better by applying a differ= ent marking probability to each flow. This makes convergence faster, and y= ou can make them converge to the same throughput, not merely the same cwnd.= When you have a separate queue per flow, you can additionally prevent one= "fat" flow from affecting the latency of sparser flows, and *enforce* the = throughput equality on short timescales, even with flows that are unrespons= ive to congestion signals. This is all established practice. And that is the key to achieving RTT independence with the minimum of added= complexity. - Jonathan Morton From nobody Tue Nov 3 09:30:25 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D6D5D3A0E06 for ; Tue, 3 Nov 2020 09:30:24 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.101 X-Spam-Level: X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gWHeRbcv3U7X for ; Tue, 3 Nov 2020 09:30:23 -0800 (PST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50058.outbound.protection.outlook.com [40.107.5.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F409D3A0E02 for ; Tue, 3 Nov 2020 09:30:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KyffZZF8B1NcQ6Q8JhjDZJaChW24rmprA4DKS7j/NItbWZ8bz9zj9/Iug6AmCu4cWxVRZswYZKJLFzCAGKXND6GNRDvnDt3Jdxw9aTYmH62vF8EaEo2Hl9S3VyHwboUUrqMFDtHUE7SGJm655CaBIVPDczaT/Me5ZaKs2vmUcDBVeFiJNcav4C426hwbNS9GALrNHjl8jn1zZez2ejIImTrMtMHjLT3ReabeBKXz68ZDIDR3Ioqf+WeKXoKb+sC0kXNs138lxd+dqKBsq3Bq5863HFaWMWxMtFDeBtVVGPzxk0Erhivh/cQ0P/edT5ycYkkrREJkVVRA5idD7rRizg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7EpnpiME3Oi56hPMLRXVvVwPfAk2XtkqQsK7S2VZA5w=; b=FWCcNcfIcL2r4fX49o3agDg9mgL7d+VRMtZJXESKnz6TwocMNgWAzULjjjShUCVlsCQfMF7Ke8G/5tNCfaYUcIfLwLGvbKXeII7wDemhTkjBnVadHIKTwoKM66XZJY6w32uzcr6FdEPBsmlxaYm7a7Yb+ZHaIf6b+gHBNnku96lOR/NiKQEOGjeO7ehrseEBy3WV8/SofrS2ZkapTlLO+AtfH31h3+E0arfP3NVfZBj09oA1OboO98Sfz8cYuECR+0p66HdjqAKkfCpM0VmMIVl7ehfm0/I6+RQs/WPL+j1Uo5bLBlgwPwdDZksejGOvLB3kxew1ggnSILMOxJOKTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7EpnpiME3Oi56hPMLRXVvVwPfAk2XtkqQsK7S2VZA5w=; b=QFKlLlmDbVZ6HaRcijF/vuzRIumTK/EQS8ItDN7qMJZP9MROpVATWCXe3zMFLNxYrmk0HrwgsDI7lkHuk29wuY3D8I1aIG8NUEZ8OC7igZOinC1fKhrRoUy57gCL7pNhe52NnQhEDGNWJ1J1w428cNWitMtq0QmN1RTfXfFWVqY= Received: from AM0PR07MB3953.eurprd07.prod.outlook.com (2603:10a6:208:42::26) by AM0PR07MB6356.eurprd07.prod.outlook.com (2603:10a6:20b:157::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3541.10; Tue, 3 Nov 2020 17:30:20 +0000 Received: from AM0PR07MB3953.eurprd07.prod.outlook.com ([fe80::54c8:b912:84f7:bc38]) by AM0PR07MB3953.eurprd07.prod.outlook.com ([fe80::54c8:b912:84f7:bc38%7]) with mapi id 15.20.3541.015; Tue, 3 Nov 2020 17:30:20 +0000 From: Szilveszter Nadas To: "tcpprague@ietf.org" CC: Ferenc Fejes , Gombos Gergo , LAKI Sandor Thread-Topic: A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results Thread-Index: AdayBknBddpo1c1ATr20xSsm+1eksg== Date: Tue, 3 Nov 2020 17:30:20 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [178.164.252.23] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 81db2b08-5976-41d0-aeb5-08d8801e236d x-ms-traffictypediagnostic: AM0PR07MB6356: x-ld-processed: 92e84ceb-fbfd-47ab-be52-080c6b87953f,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6zF6kWw1Da4DyGdspnDwZBDIGHs/EnymjNQjnRCwvueIYC7uhJtvZVRugHXJ7gLrF0V38hEE7LdgIV6XbzhyiK+Rqphn4cYo2OoKiWncJSmkVvAXmLbc696AAoPLkxwjEYSkF8nkO+oMIxXz9zuPj75PQtedZq5oBYoWWeeOoJdfkY39njUEglenyXzpkFTnR2wYL+nn6W8/ukb5LoYwu1jEYKJ4w9bNfNnSfy4hjeyg6YLx0gAwOZ7teC9R7rjBJQjnpikL2BYh8ZGnLHzA+2lKWmqGeLtjyFYQx04a0WOucmFy/PoTVhSn5kcYBQ2CjbraxfyUJnyozyeTVWqPhGU7429qcpIm61HOy9RaweuRn3RLfAoIGjZbfFSuRTRbnli2gl1mhRL79JRttaT5zKuJ0QIL7AcMsvdBq5Vm15C4OatcIhRc+5nh8BJ4reFAQW5mPkzRPLlkmdTFm7c1qw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR07MB3953.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(396003)(39860400002)(136003)(366004)(66946007)(66446008)(54906003)(86362001)(2906002)(55016002)(4744005)(52536014)(66476007)(66556008)(4743002)(64756008)(26005)(83080400002)(8676002)(71200400001)(8936002)(186003)(76116006)(6916009)(4326008)(9686003)(966005)(6506007)(316002)(7696005)(33656002)(478600001)(5660300002)(166002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: MwtdxV7SBlqb1nfc4uBxhJRVBuV47CESwizake8dffFv2Gu3lb7mulCKfZPpfOGeO5wNhDzvrTVTLRlzyONGmMyHfzd7p0zktaItmhxgTKh3JLDckY5G3YUuR6pZhDN2tMrY2Wed6/jFUKSwtkgHUD9oMF0GDB24+exhXki1frIJWf71lw95oxdwtB8SNbpHckMgr5/Rv0IrlXRSk7w4x/nie2VXY/nhe/N1JChKacN2RvmXsqbHm26OljwJhQj/fZ33XDTH5ByQS8QQpECxZwKeS4ZSGcpzm3bnXhKIitqX6C7VtrQjfbtiB+cESLjKaNnIGPVsyURbtuwowuU4A3PQrSTFlA6WeIhiO9oD6hxjEv7HE4TkTiKyiYTM2dmnto86XhoJth17IEkbx4J8Gn34em4Z/RsC6SLxPyUNVeQ7g0+QA1t4M0fcg+xDWKKwp324NKRFvTGi7t7d1pYyXCgc1rkMmnsjYTdt/dGOg/pH7m8bVcWQTdgJXy0Tz5z8JGcKaIW9dP6CfxM8Yh1xKR+Dr4GfeBgT4zcKMsNNAJcDXELuJvWY7faNy8P81dgZm6EdIjtYVIuBTK3AJnlMZKddPKiY5NpnUoimR/nBLVSmcGnoiXnpC3e5fAc9jUW4IJv4RNjVhG9bWpA/oxrCcg== x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM0PR07MB39533753D400A496BD46CB588B110AM0PR07MB3953eurp_" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR07MB3953.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81db2b08-5976-41d0-aeb5-08d8801e236d X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Nov 2020 17:30:20.3930 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: WKROgGvrPhEAuNuHqp+pqJ1ASEpUiaFgMz06DIz1s9/kVyMBJMiu/36R2b+fTq7pDUOti8aM2wX0QAww2LnPnAY6QoQu9QOhXddAH/GP7IQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR07MB6356 Archived-At: Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Nov 2020 17:30:25 -0000 --_000_AM0PR07MB39533753D400A496BD46CB588B110AM0PR07MB3953eurp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi all, During the review of our article "A Congestion Control Independent L4S Sche= duler" we received comments from one of the reviewers, on why we did not al= so evaluate TCP Prague. So now we rerun the test cases with replacing DCTCP= to TCP Prague. The results are at the below link. We are somewhat surprised by much increa= sed aggressiveness of TCP Prague. Can you comment on it? The settings and s= etup we used are described in the pdf (and in the original article). Results: http://ppv.elte.hu/tcp-prague/ Original article (including YouTube presentation): http://ppv.elte.hu/cc-in= dependent-l4s/ Can you comment on the results and the settings we used? Cheers, Szilveszter --_000_AM0PR07MB39533753D400A496BD46CB588B110AM0PR07MB3953eurp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi all,

 

During the review of our article “A Congestion= Control Independent L4S Scheduler” we received comments from one of = the reviewers, on why we did not also evaluate TCP Prague. So now we rerun = the test cases with replacing DCTCP to TCP Prague.

 

The results are at the below link. We are somewhat s= urprised by much increased aggressiveness of TCP Prague. Can you comment on= it? The settings and setup we used are described in the pdf (and in the or= iginal article).

Results: = http://ppv.elte.hu/tcp-prague/

Original article (including YouTube presentation): <= a href=3D"http://ppv.elte.hu/cc-independent-l4s/"> http://ppv.elte.hu/cc-independent-l4s/

 

Can you comment on the results and the settings we u= sed?

 

Cheers,

Szilveszter

--_000_AM0PR07MB39533753D400A496BD46CB588B110AM0PR07MB3953eurp_-- From nobody Thu Nov 5 03:10:00 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0E9C53A1781 for ; Thu, 5 Nov 2020 03:09:54 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.903 X-Spam-Level: X-Spam-Status: No, score=-1.903 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E0uQJ0NygBmF for ; Thu, 5 Nov 2020 03:09:52 -0800 (PST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60099.outbound.protection.outlook.com [40.107.6.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 120CE3A1769 for ; Thu, 5 Nov 2020 03:09:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qgwo/h1E4QI9JDo7k7C2F6JEomKSh4NvX/yHEbRwQm6NFFaR9vShFvuMNEG2ZHAoxAGE2lTBjSIpf7zugH17/5ra/SEV5tFcNVQpT7LaEzETg5BCb7Y4dOEKXST3K3B+tmsyZVEUmBsbjXq/V98Ft8Z58h2h1I4usqHJFkp32asFEmTFYDsnqgcSjkct/8RNUY6DyZisArR1uW/vnzUuxIRLp6ClpU/3zZA1c8ReL/xEkbkpfeqTj2EQcBma4J5M6QhYRZir3n2DBJJwQkmQ6gvzhL3PaSUVKW5kV8AqjdLbqEPn4kT7GCgeOzv6gcHFcCGRsGhTPcysOSfoWyZWHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/3BrNMXNxl/lG2xFsqTMS2mVB/wHIdyxvuMwR3PYjx4=; b=NcWT5wOcqExjwda+Gp7UA4qO2+vTsMX9IUPL/wgBXKWd6vNYW/j0HJ6rgVxQ+jrhYSmcBrgvd7n9OkcqdFvNNJb5ET69Ut/CRWcXMySwdBkLa4vtcFSaAGrwZdE3Z+MHGT0DkRUNCDsVXDe9z76+zwpeS95NgFaHLmb7XxRrDUJ0vfQhm4DAYVyCpKV+sbdwDl6qebJB2aA169aapMts/RDv7VfsTcsHGdSk7XFV7fzPxzbB/NLU/dxnjSZ64w9c2Q2zOruD20rLh2rEUesTq53sSl/i1Ii3HbpXe4hsN8F5jyUvaqGLmSdfKCHaVh6MFWtscsBpPg+wKvrfbuSW+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-bell-labs.com; dmarc=pass action=none header.from=nokia-bell-labs.com; dkim=pass header.d=nokia-bell-labs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/3BrNMXNxl/lG2xFsqTMS2mVB/wHIdyxvuMwR3PYjx4=; b=zXWXNi0lEhXX3GgSEjaYxtf6fnLnbh9YLdg1jOo1L9idFoHm1Yql/cKqw7Wfe1Q6zQQgaKh3o4HMI8jD3HAEQJR5Ts37Fxf4F3jmk5+pk59v8hnT1WdtCFJULZc3yxsK+W5v90gCdxn0BO+V0gpVwATf/R7Qhbf68HqC7lv+MZI= Received: from VI1PR0701MB2464.eurprd07.prod.outlook.com (2603:10a6:800:64::12) by VI1PR0702MB3565.eurprd07.prod.outlook.com (2603:10a6:803:d::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3564.13; Thu, 5 Nov 2020 11:09:49 +0000 Received: from VI1PR0701MB2464.eurprd07.prod.outlook.com ([fe80::147a:b335:dc08:a938]) by VI1PR0701MB2464.eurprd07.prod.outlook.com ([fe80::147a:b335:dc08:a938%5]) with mapi id 15.20.3564.013; Thu, 5 Nov 2020 11:09:49 +0000 From: "Tilmans, Olivier (Nokia - BE/Antwerp)" To: Szilveszter Nadas , "tcpprague@ietf.org" CC: Ferenc Fejes , Gombos Gergo , LAKI Sandor Thread-Topic: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results Thread-Index: AdayBknBddpo1c1ATr20xSsm+1eksgBWJt2g Date: Thu, 5 Nov 2020 11:09:48 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dmarc.ietf.org; dkim=none (message not signed) header.d=none;dmarc.ietf.org; dmarc=none action=none header.from=nokia-bell-labs.com; x-originating-ip: [2a02:a03f:636b:f600:3986:7cc4:5893:2315] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 7576d6b2-2a0b-4bcb-8621-08d8817b4faa x-ms-traffictypediagnostic: VI1PR0702MB3565: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 2YnzI7utmEf/ZTro26T8HZo8IPxIUVR6FAVB0lk4YgaBpg/0Nx8oAVIGzUzaJNcnJl5j2ks5f+SQrz69hBH4/1jA53/mZGn7UbsySa2dwVKy01gUXf0urdnEM4+u9WAJoL/pIEvPey6Z3AhcACvSUPpmny4Ir065ccScLiRna0worPzr2kZcLYtVNd0Bf7rQ0daB7gx7GBU3Rq/SCwPhP1TlLLeb9J7pXXEZUe6u+rcAVZW5MaQB7bdDZQnM9Ff3sssJTcfJWJfzj+vRV82a8Za/sQdpFvdB0yruTnzBhXTRh20jhW5pPkh0qyh8rCw1czPxhg2J9gMY0NfVsGqj3Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0701MB2464.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(396003)(346002)(136003)(366004)(9686003)(86362001)(55016002)(71200400001)(186003)(66556008)(66476007)(110136005)(478600001)(54906003)(7696005)(8936002)(33656002)(66446008)(66946007)(52536014)(64756008)(76116006)(2906002)(6506007)(8676002)(4326008)(5660300002)(83380400001)(316002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: JdKEfLV9uH5kKZjl2F6hjS55gEZPIDlzNtT0GivKYHDwwfDhgFCrfsbNfprxno7dJaM9MGIpidTkpUtJril+a4vApTVmXkcKzveZ1vF2uDmYmB1TTxLzs0DVEvR/SBDCVGBxIWpPuZsPFNlq6RPKacA6nEFuFFPrc8LMvN+7KFAE0mRAS+MG8NqG4PBJtfJtp9wjE7O36oi70CRrnXdofMUAiUnYZF9TL1CPsLkEweQtlu+GgOXoiqRV4p04IGQCuNRIVmez/j0m3wAJHoaIpUP9VrJII2i2dLCAC57OgFH84lzqPgxL6lzlD0WPDROiS41YTa4A9n5+mGNuq9jcYp0tBATy2MwS1QLiryOYZyAvtY7slgy/AJ2jnX5pizK1fHNHznf7Eh7vEMbQV8hdWeCT291rEPGR4o1MJYbQvPqV4n+mcNLoQvf7vPg62IpV7wzUqiw2F9Wol1IWjnN5YvPa6KoRroEXINMFQwuPhkOLxPn6xY/s9e4rYEBxaLnxoztu8Qjm7UoUhyFYPXZUMkPdmK9oJz2XlFtXA4d4lvlBHiFnZjG2jpxQ6PyQYIg0aJ5P6d9uZDB7ZkQfwYuHPJp50M7KWMGBgFQWnCIUKeiPfNS/GC1Llr+tdwThYnBxKkNlbocDPxuOsYgf6Lu3iofJzHkh34mKLGPzzaE+u1IawFa+B/+Ht6KAXDfMqrRDXPD76bO8rPUCBEpTzfF2Bg== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR0701MB2464.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7576d6b2-2a0b-4bcb-8621-08d8817b4faa X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Nov 2020 11:09:48.8731 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: gcv7dFOTxgJOuxf6ldOJz9o/TyN9dh06G9Kui5b+bXkYi67buVrFREaPlJL/qUcRpAr8YjMD7wACi2Ks4XBYVyep0xPCYgaoViDhS4wU+xXZUMKodAy11qvhLdnLg0ne X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0702MB3565 Archived-At: Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2020 11:10:00 -0000 Hi Szilveszter, =20 > The results are at the below link. We are somewhat surprised by much > increased aggressiveness of TCP Prague. Can you comment on it? The setti= ngs > and setup we used are described in the pdf (and in the original article)= . Thanks for sharing these. * Short answer: DCTCP actively hurts itself, tripping the step AQM unnecessarily. This is mostly benign in DC environments due to the almost non existent RTT, and small cwnd, but causes observable performance hits elsewh= ere. Prague implements fixes for these issues. * Long answer: Please find below the main changes from dctcp to prague. The intuition behind these is to avoid triggering the step AQM part of the dual queue when there is classic traffic, or limit the extent of the received marks. The first goal ensure that, when there is classic traffic, prague only receives marks from the coupled PI2 component, which are strong enough on their own to keep the L4S queue empty most of the time. Additionally getting marks from the step AQM means that prague is hurting itself, as it will reduce more than necessary. The second goal ensures that prague does not unnecessarily drive the AQM (either PI2 or the step) to high-marking rate; again because it will hurts its throughput and also because it will create a standing queue. Note that depending on your experimental setups, not all the below factors will contribute. 1. Prague is always paced, regardless of the egress qdisc on the data sende= r, and paces itself at 100% of the estimated BDP. DCTCP is only paced when used in combination with fq, and paces at 120% of the BDP when that is the case. 2. Prague actively limits its gso bursts (defaults to 250us) DCTCP uses Linux defaults, 1ms 3. Prague implements a more accurate internal marking estimate (alpha) DCTCP, especially when operating with low marking probabilities, will tend to push down alpha to 0, delaying its response to marks (and thus increasing queue pressure/causing larger reduction down the line) 4. Prague carry over sub-cwnd reduction, such that multiple marks in a row occurring at low overall probability will eventually cause a cwnd reduct= ion DCTCP's cwnd reduction code has no memory, i.e., as long as alpha is low enough compared to cwnd, no cwnd reduction will ever occur. This again drives the aqm to larger mark probabilities. All of these points contribute to prague being more gentle towards the queu= e, and more reactive to the received marks (i.e., achieving a lower overall marking level)--the direct effect being smaller cwnd reduction (thus sawtoo= th) than DCTP, which is critical when operating at higher e2e RTTs with a shall= ow queue since it lowers the time to recovery. Additionally, when operating over a long RTT path and controlling by PI2 (i= .e., random marking, such that it receives marks almost every RTT--as opposed to= a step where it receives nothing for several RTTs and then a RTT with most of= the packets being marked), DCTCP suffers from the inaccurate/memoryless computa= tions in 3./4. and its interactions with Linux's PRR code (which prague does not = use), which can prevent it from ever reaching its expected rate. I hope this helps to understand a bit better why prague appears to be more aggressive--it is actually the opposite: dctcp systematically hurts itself at longer base RTT, hence progressively give way as the RTT increases. You can validate this by observing that the expected rate ratios between classic flows and prague over the dualQ match the theoretical equations (compounding the queue delay difference) even that higher base RTTs, which is not the case for DCTCP. Best, Olivier From nobody Thu Nov 12 01:35:03 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 995603A1531 for ; Thu, 12 Nov 2020 01:35:01 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rvFIW6HIJHJ3 for ; Thu, 12 Nov 2020 01:35:00 -0800 (PST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80107.outbound.protection.outlook.com [40.107.8.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6EE283A152F for ; Thu, 12 Nov 2020 01:34:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=i++oZs7p3pUeh7sneDlC5GuFlokEUzZ6OZBkINHdYnYGjwZvRGW/jWtxq/ri8IRJEOxcCfXjbbtNVBLIN6Uc1Uudbq5L2IYpl8oPBJ3qDWM2ntXdYDuooXj4ZTLO3HtyNbquz4QbMitTIbj6zjqGNMDqRNeU1BTGshnMRqDxF497VamN3oXNiWAe1WHRIDb6yIbAninHLpeV6v+szPcrEMtka6NMjBYLAwkdhGFqvqz7shx65yEZFNiL0xpSjAII+SliPLcCjaHMnc0qP9MRJnfb8CT64TjbHIHVVBCoo7L8+01JwMlfAXCQYBTRRLbLG7Lw7ua5qoYv+vvOYNfefQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yotiCwPl9ozylo2VHiv9Z2eGDWfbcfEwDJBNmyebOjQ=; b=kZfsol4cAUXLqACrxtwNJSrVr1rRLEmQj1bCR0OcX+ukg3I2DHo4TDhYi9sT3NBwFTvax95TicXClt51qOuAFrEDlW8IcQ7XINeluLRU9MfxypXMokaXcWaoSz/9UNKTNgRcDoq9+ECZfAUrdLx5/SOUblNTVl6W19BAu7pGnZxYruiz7RaNL9nDLrQXVNF4VRYDBCO2dsLaBSdbiI1cnxID986LXOYdoZ1x6M8NolDnXETSu3DRnxwyPltAfjfFvMURkFWn7ENjwdjQDf67XlHQbLtbhirITDab5D4DX7S8mDAa60ftEb4Lh7nFi8FG5EboiUQ5r+VAWAAGP3OtBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-bell-labs.com; dmarc=pass action=none header.from=nokia-bell-labs.com; dkim=pass header.d=nokia-bell-labs.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yotiCwPl9ozylo2VHiv9Z2eGDWfbcfEwDJBNmyebOjQ=; b=C9JTF7I3FAVvES5LMoEkgxKRgxrOYygKZpYI3HF9aD5xD68h8HzniAVV77BMIdtpRs2xXgnkxdDf+t4C1/Rghl3uq8cprRR8TMvTdDuX0wBxYlZwBf9K5CYiJJ80VY4mdtnHKSJV3Sx4PTsN7IBlLD0Zljuy0U+JQovtNuPh+2g= Received: from AM8PR07MB7476.eurprd07.prod.outlook.com (2603:10a6:20b:24e::12) by AM0PR07MB4227.eurprd07.prod.outlook.com (2603:10a6:208:b8::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3564.15; Thu, 12 Nov 2020 09:34:56 +0000 Received: from AM8PR07MB7476.eurprd07.prod.outlook.com ([fe80::e966:7a41:b22a:1560]) by AM8PR07MB7476.eurprd07.prod.outlook.com ([fe80::e966:7a41:b22a:1560%4]) with mapi id 15.20.3564.024; Thu, 12 Nov 2020 09:34:56 +0000 From: "De Schepper, Koen (Nokia - BE/Antwerp)" To: Szilveszter Nadas , "tcpprague@ietf.org" CC: Ferenc Fejes , Gombos Gergo , LAKI Sandor Thread-Topic: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results Thread-Index: AdayBknBddpo1c1ATr20xSsm+1eksgGzRlkA Date: Thu, 12 Nov 2020 09:34:56 +0000 Message-ID: References: In-Reply-To: Accept-Language: nl-BE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dmarc.ietf.org; dkim=none (message not signed) header.d=none;dmarc.ietf.org; dmarc=none action=none header.from=nokia-bell-labs.com; x-originating-ip: [2a02:1810:1e00:cb00:210b:63c2:20dd:546d] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 14d15210-7f4e-4471-74bd-08d886ee3753 x-ms-traffictypediagnostic: AM0PR07MB4227: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: umlXaIcrYmMCxXXB15N3NhQicioWcyFTl4PKEJc7/iQGWrlnucP8h9i+hWgmQ2tRt8WnMaFjilrx+Kngh48hSaTd2UyWkkYt8MeFW/RHqnKaMnn9rMqdTJhKQGVlRPvZgNN0ihZLIzlGVLQ8Ws0JA1rcE8aUtAXaV9RFQMixhUHs/Fa/6AIeSeYJRSXb2HsrMMIriRvqSxh0am7NhDeu8vzXsHxadDsSDGvhwc4TY9ehex6GOm7Xly3L3aAcCzHjqOTPVmeEonBAAvyPIrzthoJmkh0Pg1fLsth6dh6/0dNA/uIFM2SzdW1rpNI/291h4Y1pC0OFSWsnEm8im1ati9BSL2N+f04W2BMIwyH0Aqob5dbN5hfZXR9kFSTUReYIq5/3b92SmLyiq/8fBE3McSElDK+VSCPMI2reYwgf5lLbjctpzVbgZO7/9wMpddEm/X/a3usQ4aPXaW6NXRdOMw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR07MB7476.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(136003)(346002)(396003)(376002)(366004)(33656002)(8676002)(66556008)(316002)(83080400002)(9686003)(66476007)(55016002)(7696005)(5660300002)(83380400001)(9326002)(4326008)(66946007)(166002)(76116006)(966005)(66446008)(6506007)(8936002)(186003)(52536014)(71200400001)(53546011)(2906002)(86362001)(478600001)(110136005)(54906003)(64756008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?uPgHJV7RENxJYJtbLLWUMEvs/BcBkgeLyKgnpJoZ3EpMraWecddCTaOkUy?= =?iso-8859-1?Q?BZrioSmX/6xOGplT/ykbhygJc/upUBHb77nIha4eWQ5IKelealiFV0SVVF?= =?iso-8859-1?Q?xvL1ZeFNsraAFNNoHs64mHwZmtESgHOVO5boTAlyIw8vr63ch5djlThOG5?= =?iso-8859-1?Q?veROEObtQhLangzDMj7bz3++NiT2VyDbmDzYoMINeDADMqJ4M4tC0peTbE?= =?iso-8859-1?Q?EZjw0PKw96uhULY56bgEMncZ2pqM0qKlkM+TaAMVXd2lILZ93nZ35eeSbT?= =?iso-8859-1?Q?hY+OkDsiGNIrnDw9LZn2vyn9lp7x5A2hdSsHoJyOopDXGXChkQrcb7/XWY?= =?iso-8859-1?Q?VdJqXZd7LOWgcQ0/Nj7S001iURd3YnAECNZz4YDjhXtsg1lPp7JBkTNQwU?= =?iso-8859-1?Q?mtCcOEcucvpVv5afTslX525SEAHD59JXRtjqzT0vjP5cV8Fir+9MafznZi?= =?iso-8859-1?Q?3SnlJOzdVoliRw2qUfVt/Wk010GOGu5IULUwRODPmmumsFeoubjTrtKQKd?= =?iso-8859-1?Q?NU8vsWYY8HSebwcHMB+elE3tncmW1IGMsXa3lyDM8DQA1vmYvMATEVUOsU?= =?iso-8859-1?Q?9atze9qp+M5qqWbG1DlUfgy3SHtHSnBt4aqEqJfy/VoHnTJ+ryJAdnJAAk?= =?iso-8859-1?Q?IyEdjaI17/TaFztQGs1AM+u7JfKsBNIUaOopJ5UBxRU/3g6Ow3PjIUmYmL?= =?iso-8859-1?Q?jUaDRlr8dCwnGnTQ/MbBkCIZTSwkgzMHRTAcz5nHvT1B4WtkaTuKaUzBeI?= =?iso-8859-1?Q?9ExV77taQm56OGKSFSsz3ezE3yJXw48tF0bYeCm59PAU5X0dJFfSHVSR5U?= =?iso-8859-1?Q?dduoNliKMhDKlTjtIkybrFFUYwzD1FDY2mZn6CHbXO7rRWm4cEwy4Xpwrk?= =?iso-8859-1?Q?6gLX6JdH6z5j+Vstb0lA5iUiUEdF+k7nQwWQ7VVXSmp7q1Nf/oJwXCKU4N?= =?iso-8859-1?Q?WY498c/xn/WkF3KavDeAOGsMvmN7qvoMB2iNyQzb1UKVf03bWdQqiA=3D?= =?iso-8859-1?Q?=3D?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_AM8PR07MB7476F8110C025D0F8855B960B9E70AM8PR07MB7476eurp_" MIME-Version: 1.0 X-OriginatorOrg: nokia-bell-labs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM8PR07MB7476.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14d15210-7f4e-4471-74bd-08d886ee3753 X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Nov 2020 09:34:56.0324 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 2Z0DFKnrd+ALeOf0RXJFbzqVQorpNxY9xkpyZFVtly9FPB7VX9ydughSigZRy5CGFcHWogT4MY4IuCcJo0Sy9Spb+3HlNWSSpwLxjrxBnkRPhCxzIpLbFb2TxuX0FItP X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR07MB4227 Archived-At: Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2020 09:35:02 -0000 --_000_AM8PR07MB7476F8110C025D0F8855B960B9E70AM8PR07MB7476eurp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Szilveszter, Interesting work, thanks for sharing. There is indeed a very big difference= between how DCTCP behaves today in recent kernels, than what we used in th= e original L4S work testing (Linux kernel 3.19). That original version was = a "clean" DCTCP version that matched very well the theoretical equations: r= =3D2/(p.RTT) for uniform stable random and r=3D2/(p=B2.RTT) on/off marking.= Due to many interactions, integer range constraints, pacing/GSO interactio= ns and bugs, the performance really degraded in the recent Linux versions. = These issues were fixed within the Linux Prague version, so that is why Pra= gue should perform "better" in respect to matching the equations. It would = be interesting to get an idea of the deviation from the r=3D2/(p.RTT) equat= ion in your tests (relevant in DualPI2 as the coupling gives a smooth proba= bility). Collecting the average RTT (base + queuing latency), marking proba= bility, and rate would make it possible to evaluate this. Main reasons why we saw deviations is when the probability varies a lot in = time. As you might know DCTCP's (and Prague's) equation becomes r=3D2/(p=B2= .RTT) when it receives on/off marking episodes (on episodes in the order of= 1 RTT, off in the order of multiple RTTs). Any not so stable marking proba= bility results in a rate between those 2 boundaries. This is the theory, lo= oking forward on how your practice matches this. >From a first quick look at the results it might not deviate that much, I gu= ess. The more aggressive part is probably due to the RTT unfairness: 1 to 5= rate ratio (with a 5ms base RTT, Classic gets a queue of 5ms+20ms target, = so about 25ms and 5 times less throughput). Did you try to use the RTT inde= pendent version? If you set it to the f=3Dmax(15, RTT) mode, the minimum ef= fective RTT becomes 15ms, so the rate ratio is 3 to 5 in that case. Thanks and Regards, Koen. From: tcpPrague On Behalf Of Szilveszter Nadas Sent: Tuesday, November 3, 2020 6:30 PM To: tcpprague@ietf.org Cc: Ferenc Fejes ; Gombos Gergo ; L= AKI Sandor Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP P= rague preliminary results Hi all, During the review of our article "A Congestion Control Independent L4S Sche= duler" we received comments from one of the reviewers, on why we did not al= so evaluate TCP Prague. So now we rerun the test cases with replacing DCTCP= to TCP Prague. The results are at the below link. We are somewhat surprised by much increa= sed aggressiveness of TCP Prague. Can you comment on it? The settings and s= etup we used are described in the pdf (and in the original article). Results: http://ppv.elte.hu/tcp-prague/ Original article (including YouTube presentation): http://ppv.elte.hu/cc-in= dependent-l4s/ Can you comment on the results and the settings we used? Cheers, Szilveszter --_000_AM8PR07MB7476F8110C025D0F8855B960B9E70AM8PR07MB7476eurp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi Szilveszter,

 

Interesting work, thanks for sharing. There is indee= d a very big difference between how DCTCP behaves today in recent kernels, = than what we used in the original L4S work testing (Linux kernel 3.19). Tha= t original version was a “clean” DCTCP version that matched very well the theoretical equations: r=3D2/(p.RTT) fo= r uniform stable random and r=3D2/(p=B2.RTT) on/off marking. Due to many in= teractions, integer range constraints, pacing/GSO interactions and bugs, th= e performance really degraded in the recent Linux versions. These issues were fixed within the Linux Prague version, s= o that is why Prague should perform “better” in respect to matc= hing the equations. It would be interesting to get an idea of the deviation= from the r=3D2/(p.RTT) equation in your tests (relevant in DualPI2 as the coupling gives a smooth probability). Collecti= ng the average RTT (base + queuing latency), marking probability, and rate = would make it possible to evaluate this.

 

Main reasons why we saw deviations is when the proba= bility varies a lot in time. As you might know DCTCP’s (and Prague= 217;s) equation becomes r=3D2/(p=B2.RTT) when it receives on/off marking ep= isodes (on episodes in the order of 1 RTT, off in the order of multiple RTTs). Any not so stable marking probability results= in a rate between those 2 boundaries. This is the theory, looking forward = on how your practice matches this.

 

From a first quick look at the results it might not = deviate that much, I guess. The more aggressive part is probably due to the= RTT unfairness: 1 to 5 rate ratio (with a 5ms base RTT, Classic gets a que= ue of 5ms+20ms target, so about 25ms and 5 times less throughput). Did you try to use the RTT independent versi= on? If you set it to the f=3Dmax(15, RTT) mode, the minimum effective RTT b= ecomes 15ms, so the rate ratio is 3 to 5 in that case.

 

Thanks and Regards,

Koen.

 

From: tcpPrague <tcpprague-bounces@ietf.or= g> On Behalf Of Szilveszter Nadas
Sent: Tuesday, November 3, 2020 6:30 PM
To: tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; Gombos Gergo <ggombos= @inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler = - TCP Prague preliminary results

 

Hi all,

 

During the review of our article “A Congestion= Control Independent L4S Scheduler” we received comments from one of = the reviewers, on why we did not also evaluate TCP Prague. So now we rerun = the test cases with replacing DCTCP to TCP Prague.

 

The results are at the below link. We are somewhat s= urprised by much increased aggressiveness of TCP Prague. Can you comment on= it? The settings and setup we used are described in the pdf (and in the or= iginal article).

Results: = http://ppv.elte.hu/tcp-prague/

Original article (including YouTube presentation): <= a href=3D"http://ppv.elte.hu/cc-independent-l4s/"> http://ppv.elte.hu/cc-independent-l4s/

 

Can you comment on the results and the settings we u= sed?

 

Cheers,

Szilveszter

--_000_AM8PR07MB7476F8110C025D0F8855B960B9E70AM8PR07MB7476eurp_-- From nobody Thu Nov 12 23:27:22 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C90C83A15EE for ; Thu, 12 Nov 2020 23:27:19 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.101 X-Spam-Level: X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZbWFbQdDQTjS for ; Thu, 12 Nov 2020 23:27:17 -0800 (PST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80051.outbound.protection.outlook.com [40.107.8.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 58DBE3A148E for ; Thu, 12 Nov 2020 23:27:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VkdWZVKRaWghD0Rey3AHOWRtAtZDxXE+V9P38QsQ1eK4a/e4GO6Hcyuf/uwo0mrLa4jPU9TsaZbHNz68yoc220eHPHrVw07yuahguen6Jlle4gU6t/XGsb4tXm1TpUW7WdIPpvm2WZkDJ8neWySpAehDv41d1xBFNzfS1vrpBmb1QIlVdHjPP0EqJHMQye/jtFY/vJ3PVyj47tXAsAGcmCRRJZl6wmWo5kfjQejyGl/yvV0+I0r8TL77Nm9wX/R80UOddrf+R5ocGyBn+E5OM6qv3gcAXHJ7j6qUpC4/7zLn8BOr8rnNF6sdytzW3XwzrALSNI/nDjs0SLJc2gT7+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yu9mDd07NjE1+3rJ5gRNJD2VTX5GgXmUhRwkvshrSvw=; b=KEy2r4WiKbyGwgSEX2+oVNq7xeT83aqXoOjR/pPuWGgwNBUxjDbBWzv+qf4wFuz64b2O1WS6n1N1QYkBEXKW7zuPBKu1pUlB5SW67VPEOdgClC3UcycRg1tn3RfpuBn6XAfLTqjRwme3kSF8X5UtJwARocydp0Gaa41/nKtHDNuSziKYe233aaoMOSySySW711jbDVlQoZ6N5DWaqIC29PTGSyVCXRyVM+RW0B/yPPq/vh6FQRjegcCKz3FiWU54bqXKLPuM1e05covIsiwy+g7rx0E/Z/yk+Xb6PwsJlsXt0OcpQbPMpGJECVsIfjqxkxbY1sGGmWhSQcdyeaudHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yu9mDd07NjE1+3rJ5gRNJD2VTX5GgXmUhRwkvshrSvw=; b=h1Fh516tYbmE72L258JulokE1rLzaPla85db7rB2S7mSKnNLnQOoAqQ0CHPgIwWlpwL3G5p2Px7DOwB/iamHJnS2Ld2DqVmWZcURCcfyHDCzyp1P4YMaqIKojJCSZQlFd99LNPdk8JP1BK+ZaabAucPGpXjy9PLEEZGAIA8l3+M= Received: from HE1PR0701MB2876.eurprd07.prod.outlook.com (2603:10a6:3:56::8) by HE1PR07MB4426.eurprd07.prod.outlook.com (2603:10a6:7:a1::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3499.7; Fri, 13 Nov 2020 07:27:09 +0000 Received: from HE1PR0701MB2876.eurprd07.prod.outlook.com ([fe80::50c8:a7da:1a:48a3]) by HE1PR0701MB2876.eurprd07.prod.outlook.com ([fe80::50c8:a7da:1a:48a3%11]) with mapi id 15.20.3564.021; Fri, 13 Nov 2020 07:27:09 +0000 From: Ingemar Johansson S To: "De Schepper, Koen (Nokia - BE/Antwerp)" , Szilveszter Nadas , "tcpprague@ietf.org" CC: Ferenc Fejes , Gombos Gergo , LAKI Sandor , Ingemar Johansson S Thread-Topic: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results Thread-Index: AQHWuS50oyuR9tTdP0WFyRXkVqRckKnFpaCA Date: Fri, 13 Nov 2020 07:27:09 +0000 Message-ID: References: In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: authentication-results: nokia-bell-labs.com; dkim=none (message not signed) header.d=none;nokia-bell-labs.com; dmarc=none action=none header.from=ericsson.com; x-originating-ip: [83.227.122.88] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4a16de89-0659-40fd-8b09-08d887a587f4 x-ms-traffictypediagnostic: HE1PR07MB4426: x-ld-processed: 92e84ceb-fbfd-47ab-be52-080c6b87953f,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: LBIP+Um75MYh62csqZi3094huMyrLy+u4/kvtuzSa4bY5OoXpmev7FYXJhOwAz1XsW4vU6za8rAQ30pdW0cYpQAzIQdr/v7CpqA38Eva65eqRDgW+D/hOFcVJC6o8Kslyxud37VLW/Hq3Rwjdknq5zV/5X8K7yvpAabr2mG9Y3Id4zMUHFfiWwRnpVFgk6qOQ+GmFQc79JEctdEjdOaB60N2gbwelTggp2Aw7/3gFTwbdAeCi+pD4DPekQ93DGHOV2hbbss+u7Z172hRA+dUX1vg636nUZu61hRivoskBvbb2n8yRGHWV5j55iaArRrsqm0Iu+dTHd4UdCSWqa0m8KQ6wlxAXdhPnFvn2sDB5k32nd7DflBS3ePSzzJOHewKvr9squcKafYqrTuPAFrWyjybigdVdIWlc/wDjaFp6CJR5gtWpAxMC7GDA+oCti8b/Dm9o9T8+6SLmeyzmmhxUw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:HE1PR0701MB2876.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(376002)(396003)(366004)(136003)(39860400002)(66556008)(71200400001)(107886003)(316002)(99936003)(66476007)(64756008)(5660300002)(66616009)(7696005)(66446008)(33656002)(166002)(83380400001)(110136005)(54906003)(9686003)(86362001)(66946007)(6506007)(52536014)(8936002)(8676002)(76116006)(4326008)(45080400002)(26005)(2906002)(55016002)(83080400002)(478600001)(9326002)(186003)(53546011)(966005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: RAFkEJE6u5yyfGD1eiJk4ouErrDTzXA+T0r8p7biRuaQiPFJ9QG6RPPTN3a/SjvRMHSb0HaKcjuLOZTYDB+BUtn4dW2fJWOELwD/OAQdRipQXhpYTjl2nhKbmfLJ7qffjtPSL+524zuuDhv19H2WaX6cE1LdywCG0bUDitJoDsYITWa/KxXVEcYto1CXPISRFEFbUqb9vIs6edfwqqcHo5Z1NnNgpnb2UsjVJFdP3mDcrHnoe2SKIMKa0Fg1et1BcaeOXj4Zg+B+/hf8Prd2XfeF+UOH7pHcDUlmJYari63p5fq3X4s3KeyQ0QGagbQ2EnyuMlZDj1MAsfVEXguEzXCyADc7zz7ZhhxddNbAQLb9eSnsZGMgWI9lzOjZPMLE7UKBvlW052xABRXZkK9PzH+pFAK8FJjxOwuEwGcLWu++jxsA7Tz9vSyf/7UGqqfbaR2DdkhbH3BSTLnCfWiLpXx3hL1kVNctZHacKqsplPFjha+QnHz4Hmh639S2oNI7WklJQM1/XMvzI2+kcQX+VlQQG+acY7w8VRd8DOqRX9om5qJSXQK+dSIAhVs6oB8BymqXzGUfibY3RDAq/ej/U1Xel2RpkDE5rzEh7IpJuwkt/79vASgiC+nVhFFqg0a/MMt2gWnuqKA51mj11zf8iQ== Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_000B_01D6B996.C58BBA30" MIME-Version: 1.0 X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: HE1PR0701MB2876.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4a16de89-0659-40fd-8b09-08d887a587f4 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Nov 2020 07:27:09.2029 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4BzuUJHaGoNIc+1wp3Rs3iJNmWJwd6EPoEnXmupzrObPxfzg5X6Q80Ydj8fhlBBsX0+Bl+LlHP2iGwgb3ha6kkdkrgEICEu+hKJiglHusGNNM9i6+GSn6yfWLKRaOA9V X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR07MB4426 Archived-At: Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 07:27:20 -0000 ------=_NextPart_000_000B_01D6B996.C58BBA30 Content-Type: multipart/alternative; boundary="----=_NextPart_001_000C_01D6B996.C58BBA30" ------=_NextPart_001_000C_01D6B996.C58BBA30 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi =20 Just some food for thought.=20 Hybrid Slowstart (HyStart) in Cubic is known to have the problem that it = can exit to congestion avoidance prematurely. We have in fact seen issues = with HyStart in cellular and the outcome is that it over-reacts to scheduling jitter. This issue was mentioned in the long since expired https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-02 (section 4) = and a presentation at ICCRG https://datatracker.ietf.org/meeting/96/materials/slides-96-iccrg-1=20 The Linux TCP stack enables HyStart with both the delay and ack train algorithms on by default. It could be a good idea to at some stage rerun = the tests with HyStart completely disabled. I am not sure in this is the = reason but believe that it is worth to rule out the problems with HyStart . =20 /Ingemar =20 PS HyStart++ developed by Praveen and others at Microsoft instead exits = to a limited slowstart https://tools.ietf.org/html/draft-balasubramanian-tcpm-hystartplusplus-03= .=20 =20 =20 =20 From: De Schepper, Koen (Nokia - BE/Antwerp) =20 Sent: den 12 november 2020 10:35 To: Szilveszter Nadas = ; tcpprague@ietf.org Cc: Ferenc Fejes ; Gombos Gergo = ; LAKI Sandor Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler = - TCP Prague preliminary results =20 Hi Szilveszter, =20 Interesting work, thanks for sharing. There is indeed a very big = difference between how DCTCP behaves today in recent kernels, than what we used in = the original L4S work testing (Linux kernel 3.19). That original version was = a =93clean=94 DCTCP version that matched very well the theoretical = equations: r=3D2/(p.RTT) for uniform stable random and r=3D2/(p=B2.RTT) on/off = marking. Due to many interactions, integer range constraints, pacing/GSO interactions = and bugs, the performance really degraded in the recent Linux versions. = These issues were fixed within the Linux Prague version, so that is why Prague should perform =93better=94 in respect to matching the equations. It = would be interesting to get an idea of the deviation from the r=3D2/(p.RTT) = equation in your tests (relevant in DualPI2 as the coupling gives a smooth = probability). Collecting the average RTT (base + queuing latency), marking = probability, and rate would make it possible to evaluate this. =20 Main reasons why we saw deviations is when the probability varies a lot = in time. As you might know DCTCP=92s (and Prague=92s) equation becomes = r=3D2/(p=B2.RTT) when it receives on/off marking episodes (on episodes in the order of 1 = RTT, off in the order of multiple RTTs). Any not so stable marking = probability results in a rate between those 2 boundaries. This is the theory, = looking forward on how your practice matches this. =20 >From a first quick look at the results it might not deviate that much, I guess. The more aggressive part is probably due to the RTT unfairness: 1 = to 5 rate ratio (with a 5ms base RTT, Classic gets a queue of 5ms+20ms = target, so about 25ms and 5 times less throughput). Did you try to use the RTT independent version? If you set it to the f=3Dmax(15, RTT) mode, the = minimum effective RTT becomes 15ms, so the rate ratio is 3 to 5 in that case. =20 Thanks and Regards, Koen. =20 From: tcpPrague > On Behalf Of Szilveszter Nadas Sent: Tuesday, November 3, 2020 6:30 PM To: tcpprague@ietf.org =20 Cc: Ferenc Fejes >; Gombos Gergo >; LAKI Sandor > Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler - = TCP Prague preliminary results =20 Hi all, =20 During the review of our article =93A Congestion Control Independent L4S Scheduler=94 we received comments from one of the reviewers, on why we = did not also evaluate TCP Prague. So now we rerun the test cases with replacing DCTCP to TCP Prague.=20 =20 The results are at the below link. We are somewhat surprised by much increased aggressiveness of TCP Prague. Can you comment on it? The = settings and setup we used are described in the pdf (and in the original = article). Results: http://ppv.elte.hu/tcp-prague/=20 Original article (including YouTube presentation): http://ppv.elte.hu/cc-independent-l4s/=20 =20 Can you comment on the results and the settings we used? =20 Cheers, Szilveszter ------=_NextPart_001_000C_01D6B996.C58BBA30 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi

 

Just some = food for thought.

Hybrid Slowstart = (HyStart) in Cubic is known to have the problem that it can exit to = congestion avoidance prematurely. We have in fact seen issues with = HyStart in cellular and the outcome is that it over-reacts to scheduling = jitter. This issue was mentioned in the long since expired http= s://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-02 (section 4) = and a presentation at ICCRG https://datatracker.ietf.org/meeting/96/materials/slides-96-iccrg-1

The Linux TCP stack enables = HyStart with both the delay and ack train algorithms on by default. It = could be a good idea to at some stage rerun the tests with HyStart = completely disabled. I am not sure in this is the reason but believe = that it is worth to rule out the problems with HyStart = .

 

/Ingemar

 

PS HyStart++ = developed by Praveen and others at Microsoft =A0instead exits to a = limited slowstart https://tools.ietf.org/html/draft-balasubramanian-tcpm-hystartp= lusplus-03.

 

 

 

From: De = Schepper, Koen (Nokia - BE/Antwerp) = <koen.de_schepper@nokia-bell-labs.com>
Sent: den 12 = november 2020 10:35
To: Szilveszter Nadas = <Szilveszter.Nadas=3D40ericsson.com@dmarc.ietf.org>; = tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; = Gombos Gergo <ggombos@inf.elte.hu>; LAKI Sandor = <lakis@inf.elte.hu>
Subject: Re: [tcpPrague] A = Congestion Control Independent L4S Scheduler - TCP Prague preliminary = results

 

Hi = Szilveszter,

 

Interesting work, thanks for sharing. There is indeed = a very big difference between how DCTCP behaves today in recent kernels, = than what we used in the original L4S work testing (Linux kernel 3.19). = That original version was a “clean” DCTCP version that = matched very well the theoretical equations: r=3D2/(p.RTT) for uniform = stable random and r=3D2/(p=B2.RTT) on/off marking. Due to many = interactions, integer range constraints, pacing/GSO interactions and = bugs, the performance really degraded in the recent Linux versions. = These issues were fixed within the Linux Prague version, so that is why = Prague should perform “better” in respect to matching the = equations. It would be interesting to get an idea of the deviation from = the r=3D2/(p.RTT) equation in your tests (relevant in DualPI2 as the = coupling gives a smooth probability). Collecting the average RTT (base + = queuing latency), marking probability, and rate would make it possible = to evaluate this.

 

Main reasons = why we saw deviations is when the probability varies a lot in time. As = you might know DCTCP’s (and Prague’s) equation becomes = r=3D2/(p=B2.RTT) when it receives on/off marking episodes (on episodes = in the order of 1 RTT, off in the order of multiple RTTs). Any not so = stable marking probability results in a rate between those 2 boundaries. = This is the theory, looking forward on how your practice matches = this.

 

From a first quick look at the results it might not = deviate that much, I guess. The more aggressive part is probably due to = the RTT unfairness: 1 to 5 rate ratio (with a 5ms base RTT, Classic gets = a queue of 5ms+20ms target, so about 25ms and 5 times less throughput). = Did you try to use the RTT independent version? If you set it to the = f=3Dmax(15, RTT) mode, the minimum effective RTT becomes 15ms, so the = rate ratio is 3 to 5 in that case.

 

Thanks and = Regards,

Koen.

 

From: tcpPrague <tcpprague-bounces@ietf.org= > On Behalf Of Szilveszter Nadas
Sent: Tuesday, = November 3, 2020 6:30 PM
To: tcpprague@ietf.org
Cc: = Ferenc Fejes <fejes@inf.elte.hu>; Gombos = Gergo <ggombos@inf.elte.hu>; LAKI = Sandor <lakis@inf.elte.hu>
Subject= : [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP = Prague preliminary results

 

Hi = all,

 

During the review of our article “A Congestion = Control Independent L4S Scheduler” we received comments from one = of the reviewers, on why we did not also evaluate TCP Prague. So now we = rerun the test cases with replacing DCTCP to TCP Prague. =

 

The results are at the below link. We are somewhat = surprised by much increased aggressiveness of TCP Prague. Can you = comment on it? The settings and setup we used are described in the pdf = (and in the original article).

Results: http://ppv.elte.hu/tcp-prague/

Original article (including = YouTube presentation): http://ppv.elte.hu/cc-ind= ependent-l4s/

 

Can you = comment on the results and the settings we used?

 

Cheers,

Szilveszter

------=_NextPart_001_000C_01D6B996.C58BBA30-- ------=_NextPart_000_000B_01D6B996.C58BBA30 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIVaTCCAyAw ggIIoAMCAQICAR0wDQYJKoZIhvcNAQEFBQAwOTELMAkGA1UEBhMCRkkxDzANBgNVBAoTBlNvbmVy YTEZMBcGA1UEAxMQU29uZXJhIENsYXNzMiBDQTAeFw0wMTA0MDYwNzI5NDBaFw0yMTA0MDYwNzI5 NDBaMDkxCzAJBgNVBAYTAkZJMQ8wDQYDVQQKEwZTb25lcmExGTAXBgNVBAMTEFNvbmVyYSBDbGFz czIgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCQF0o1ncrwDZbHRPoWN/xIvb1/ gC01O+FvqGepvwMcTYxvMkfVQWikEwTBNQyahEP8XB3/ibPoFxjNkV/7iePqv05dfBsm03V57eaE 41flrSnE9Doo56V7hDZps/1edr2jLZnTkE4jKH0YY/FUOyaddluXQrL/rvBO7N05lU6DBn/nSUDI xQGyVFpmHT38+ek8Cp6BuHDwAYvkI1R8yK74kB4AlnLUVM9hI7zq+50CldG2uXE6aQg/D7ThQseI 9T+YqKe6HOBxce9YV4FQelxrdEYOgwOYw46obvJ2Mm4ng8Jz89wY6LST6nVEawRgIHFXh53zvqCQ Iz2KJOHaIdvDAgMBAAGjMzAxMA8GA1UdEwEB/wQFMAMBAf8wEQYDVR0OBAoECEqgqliE0148MAsG A1UdDwQEAwIBBjANBgkqhkiG9w0BAQUFAAOCAQEAWs6H+RZyFVdLHdmb56ImMOyTZ9/WLdI0r/c4 pc6rFrmrL3w1y6zQD7RMK/yA72uMkV82dvfbsxsZ6vSyEf1hcUS/KLM6Hb+zQ+ifv9wxCHGwnY3W NEcykMZlJPegSnwEc485bxeMcrW9S8h6+HuDwyhOnAnqZz+yZwQbwxTa+OdJJJHQHWr6YTnva+ch dQYH2BK0ISBwQnGB2jyaNr6mWw1qbJofkXv5+e9Cuk5OnswMjZTc2UWcXuxCUGOu9F3EsRLcyjuo Lp0UWgV1t+zXY+K6NbYECJHo2p2c9ma1GKwKplQmNDPSG8HUfxo6jguqMm7b/E8ln9kyx5ZacKzf TDCCBX0wggRloAMCAQICEQCH7S4aKCZKxRmqOuu5DaLLMA0GCSqGSIb3DQEBCwUAMDkxCzAJBgNV BAYTAkZJMQ8wDQYDVQQKEwZTb25lcmExGTAXBgNVBAMTEFNvbmVyYSBDbGFzczIgQ0EwHhcNMTQx MjA1MDgxOTE1WhcNMjEwNDA1MTAyOTAwWjA3MRQwEgYDVQQKDAtUZWxpYVNvbmVyYTEfMB0GA1UE AwwWVGVsaWFTb25lcmEgUm9vdCBDQSB2MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB AMK+6yfwIaPzaSZVfp3FVRaRXP3vIb9TgHot0pGMYzHw7CTww6XScnwQbfQ3t+XmfHnqjLWCi65I tqwA3GV17CpNX8GH9SBlK4GoRz6JI5UwFpB/6FcHSOcZrr9FZ7E3GwYq/t75rH2D+1665I+XZ75L jo1kB1c4VWk0Nj0TSO9P4tNmHqTPGrdeNjPUtAa9GAH9d4RQAEX1jF3oI7x+/jXh7VB7qTCNGdMJ jmhnXb88lxhTuylixcpecsHHltTbLaC0H2kD7OriUPEMPPCs81Mt8Bz17Ww5OXOAFshSsCPN4D7c 3TxHoLs1iuKYaIu+5b9y7tL6pe0S7fyYGKkmdtwoSxAgHNN/Fnct7W+A90m7UwW7XWjH1Mh1Fj+J Wov3F0fUTPHSiXk+TT2YqGHeOh7S+F4D4MHJHIzTjU3TlTazN19jY5szFPAtJmtTfImMMsJu7D0h ADnJoWjiUIMusDor8zagrC/kb2HCUQk5PotTubtn2txTuXZZNp1D5SDgPTJghSJRt8czu90VL6R4 pgd7gUY2BIbdeTXHlSw7sKMXNeVzH7RcWe/a6hBle3rQf5+ztCo3O3CLm1u5K7fsslESl1MpWtTw EhDcTwK7EpIvYtQ/aUN8Ddb8WHUBiJ1YFkveupD/RwGJBmr2X7KQarMCpgKIv7NHfirZ1fpoeDVN AgMBAAGjggGAMIIBfDBOBggrBgEFBQcBAQRCMEAwPgYIKwYBBQUHMAKGMmh0dHA6Ly9jYS50cnVz dC50ZWxpYXNvbmVyYS5jb20vc29uZXJhY2xhc3MyY2EuY2VyMA8GA1UdEwEB/wQFMAMBAf8wGQYD VR0gBBIwEDAOBgwrBgEEAYIPAgMBAQIwDgYDVR0PAQH/BAQDAgEGMB0GA1UdDgQWBBTwj1k4ALP1 j5qWDNXr+nuqF+gTEjCBuQYDVR0fBIGxMIGuMG+gbaBrhmlsZGFwOi8vY3JsLTEudHJ1c3QudGVs aWFzb25lcmEuY29tL2NuPVNvbmVyYSUyMENsYXNzMiUyMENBLG89U29uZXJhLGM9Rkk/Y2VydGlm aWNhdGVyZXZvY2F0aW9ubGlzdDtiaW5hcnkwO6A5oDeGNWh0dHA6Ly9jcmwtMi50cnVzdC50ZWxp YXNvbmVyYS5jb20vc29uZXJhY2xhc3MyY2EuY3JsMBMGA1UdIwQMMAqACEqgqliE0148MA0GCSqG SIb3DQEBCwUAA4IBAQAQ1elFTM6fGkQ/aRKdkUZicO3Cb9uzBJOpOtFctw+1El0/17lsjoVvJkZB D3KnUobnrriFdAa+7FAN55KLmZeB/3Y2bG0bB4toSyaVHjOQnQY9M0dv8U852w0Q7GwchKfebLUI bh9TMt2hI3Xc6j4knFTBUo7C1WAfO51K4bn1irmX6/Ej2VTgiOFsvOAny28W6enFSEQpSHw60VhN fSttSqTOxyrRR/7kW7Y8yb/3DZDZ/dH6ZCfx/y+BNIv2NuSd85M9HXUzplXXohti4Ql/qeaMn6by Ius6XlMWZZfkdVRvTuk2PkeC7UmAJ2+/DUWOPpawaytMXVfF4Hvxk34NMIIF+jCCA+KgAwIBAgIP AXWsTwOGurO54gJ+lN9MMA0GCSqGSIb3DQEBCwUAMEcxCzAJBgNVBAYTAlNFMREwDwYDVQQKDAhF cmljc3NvbjElMCMGA1UEAwwcRXJpY3Nzb24gTkwgSW5kaXZpZHVhbCBDQSB2MzAeFw0yMDExMDkw OTIwNTlaFw0yMzExMTAwOTIwNTlaMGIxETAPBgNVBAoMCEVyaWNzc29uMRwwGgYDVQQDDBNJbmdl bWFyIEpvaGFuc3NvbiBTMS8wLQYJKoZIhvcNAQkBFiBpbmdlbWFyLnMuam9oYW5zc29uQGVyaWNz c29uLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKM8NLGONT21wL/w2E/7OdQi eLiQdF9r2oH7aowAVCWcHrdaIg+RidSZj9MVEjymz2GNC+/NuJRpssy9ueaT3SeJYqw7rBBEDeIs bwqR4dB3HnbH4nJEJ+/RWt/M6qvkGWRIaKXWulA2Gpi4ogD8w7qcp+g2SOXod7Zk4ieqOvi3Bfqf a5TG8BLWxb8iOo/mtf6zRE8XDpe522ZcPbl3cl0JH9Ec57wS7Q4wtsTxqJKoFBbFrTwxV1M+MZq2 j4qYlJYy0R9dsXPOlFYN+iDoDNi0lHLLLg+eiSgFXf3mQ4wUTuZpc6WJ4lpkOooRjFcR7lTaIsCF 4viQbxnEe2ihL4sCAwEAAaOCAcYwggHCMB8GA1UdIwQYMBaAFBx7GZ6XnHasID3Y3OORauPbLaZT MB0GA1UdDgQWBBQWsmSmqV2RO4zberN0nQ1XJ2q3fjAOBgNVHQ8BAf8EBAMCBaAwVQYDVR0gBE4w TDBKBgwrBgEEAYIPAgMBARIwOjA4BggrBgEFBQcCARYsaHR0cHM6Ly9yZXBvc2l0b3J5LnRydXN0 LnRlbGlhc29uZXJhLmNvbS9DUFMwKwYDVR0RBCQwIoEgaW5nZW1hci5zLmpvaGFuc3NvbkBlcmlj c3Nvbi5jb20wSAYDVR0fBEEwPzA9oDugOYY3aHR0cDovL2NybC50cnVzdC50ZWxpYS5jb20vZXJp Y3Nzb25ubGluZGl2aWR1YWxjYXYzLmNybDAdBgNVHSUEFjAUBggrBgEFBQcDBAYIKwYBBQUHAwIw gYIGCCsGAQUFBwEBBHYwdDAoBggrBgEFBQcwAYYcaHR0cDovL29jc3AyLnRydXN0LnRlbGlhLmNv bTBIBggrBgEFBQcwAoY8aHR0cDovL2NhLnRydXN0LnRlbGlhc29uZXJhLmNvbS9lcmljc3Nvbm5s aW5kaXZpZHVhbGNhdjMuY2VyMA0GCSqGSIb3DQEBCwUAA4ICAQCkqYizUblUVAO2Hd8qacFV0dUO cJ4M/ZOMfrm/6KByKPObAVjz2j8eQ9O6oiaE/Wwhqv5XAVeE+F1dxhajzbuav+NjRHqMfOGLYsgM BLZkz/F9yHVNAo6YUtfABs2maFgOrfezn31tGupRCLb12H2MjGcMTHGNEqyXUTysrjMIyz87h5l5 5Q7eXelpaXYGKY+2W0JXSzNwek6jhipnXcjaLp+4cA5Oj4RwWxORl7HTCOStsbXIv91Hl/05+8qR XIAESI1emcrd9QjJ0r1b9MSlGMB4ORfMXfRutKn6/+m5vnyeq9JT4qGSJWZi3PN1Z4D6QMY8xDs+ 6tebCfPYzXzvNYvxPbyTxbNPi1FNszXbQpo4y3LvP7wuQJat7wUmD2CmxIv9Qu7OKkmKzT6sndY0 4VRJdQb2S+q+onf/g+WE4JPNS2dQ9oTPu6JwdYaWCAE80D2Sz7NSGncJkywh/e78EhyTfJGrm7o8 dpkuT6+IxWV/kiIVQchx9vTOZGvWhN4XcFaB+EuKzgNHlyDSmYP3E9UQ5MtSemsaUgRI6xk2adt4 tf2qN/CF24K93BIM6YPj+eKB7Y662IGwOye/OIL1p5XprdQV4uKfazQ0ZiawYErTK/qiCdQG0+n1 ORDMeVSUbznRwL5sklhsHeNGrJd/G7pU0UpZtDan4Or+8FERdTCCBsIwggSqoAMCAQICEFO4foPh nJkok7CbSRzsuOswDQYJKoZIhvcNAQELBQAwNzEUMBIGA1UECgwLVGVsaWFTb25lcmExHzAdBgNV BAMMFlRlbGlhU29uZXJhIFJvb3QgQ0EgdjEwHhcNMTUxMDI3MTIxNjQ2WhcNMjUxMDI3MTIxNjQ2 WjBHMQswCQYDVQQGEwJTRTERMA8GA1UECgwIRXJpY3Nzb24xJTAjBgNVBAMMHEVyaWNzc29uIE5M IEluZGl2aWR1YWwgQ0EgdjMwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDs8t8AALhQ 8qe72FS3xpP348GqO9TDRjS0s85eQ7Y0LTLZdmSz2cl+lYqs0zfSTm+7meisbhkqUXkL7fFzoe4i IZCh/VuYUaW407CZlDCXes4n4TqTSuoklN6uOPhY7EC9ZVbXILlLhRummTdDdxhVW4Leo0awEhfL f98MvWxzwCHzMj8m6YOmNjx+f9TcJE3qaA0piuvSxlfpVdiCulPTlmsmV2RSBSAwqBshZYRcQBID fqmdvkaoP9EzNKAh7yjthC0hpgHZyZMIs0eNo4v2PUmE0rhu+Zs0nujnwhljPA2/8b8v9tGixD1z btT7zoM2Ot1menJpFp4zJVSfdKVgtoWqg5t2H/E0XY1LwJez89W07nscEocyBmpC+zJAmKxKhzEW qIyP1UrZaEIFu+hO+s0Nm8sOUMa4TlG4rAUikc5U5TmUIGBRQGxulYhfAzqSYf8oLUMLky1DOa9e Ru3sp0FdQDEzQlnF/h1L4AK1MOkX1vS+fLgOvBo5LRU1fLPUZQ7FKrDXC6nl2ldvEtljHWstGBmq v25aEvAA+yrrplCh/kYvSBjvZibz9Obbwx4yqS77/NHN1iyZyVP2s52B2BLdvo4yhzk6nRk8S/8z HaUUkBUrrvijPDaGK5FNVSaioGvkC7IKioITKffYLtT9XuirKrHlh3VzkazG46pAVwIDAQABo4IB uDCCAbQwgYoGCCsGAQUFBwEBBH4wfDAtBggrBgEFBQcwAYYhaHR0cDovL29jc3AudHJ1c3QudGVs aWFzb25lcmEuY29tMEsGCCsGAQUFBzAChj9odHRwOi8vcmVwb3NpdG9yeS50cnVzdC50ZWxpYXNv bmVyYS5jb20vdGVsaWFzb25lcmFyb290Y2F2MS5jZXIwEgYDVR0TAQH/BAgwBgEB/wIBADBVBgNV HSAETjBMMEoGDCsGAQQBgg8CAwEBAjA6MDgGCCsGAQUFBwIBFixodHRwczovL3JlcG9zaXRvcnku dHJ1c3QudGVsaWFzb25lcmEuY29tL0NQUzBLBgNVHR8ERDBCMECgPqA8hjpodHRwOi8vY3JsLTMu dHJ1c3QudGVsaWFzb25lcmEuY29tL3RlbGlhc29uZXJhcm9vdGNhdjEuY3JsMB0GA1UdJQQWMBQG CCsGAQUFBwMCBggrBgEFBQcDBDAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYEFBx7GZ6XnHasID3Y 3OORauPbLaZTMB8GA1UdIwQYMBaAFPCPWTgAs/WPmpYM1ev6e6oX6BMSMA0GCSqGSIb3DQEBCwUA A4ICAQBQWGvx1Yw7tC6rV0PIjKfDyxaanIX+NZLEGOkdQLKGW2gVLtDUJQEPRs5QtaZiObNHCZ7m mSNMVek4lkt/0dqfVIFutVw/QkyFGwC99ZmNwXSX9z+OoMyoEBHGvw5RY6vRlZrj0uKvdASzYL4K MaB7m3NwurNDmmNbG52suRIZ76wBOEOddRZcZiTy50ZkBqYnnl2t3D3oBX2NZCQysshUcqRdUbkS 13HTCIChMuTV9W0tzPXUOJoJlJlU9nd91IikhGEOrPwfixWms+C8sF0r9qN1uJGx6ELPOiFrLfNt cMNMMbAqRHwpSLxe3wcNkJGxv9T8LswLi1UrRIQ85AKjqzBnLSsjRGgbMgJ+xKtngmvEA155JmoK fUD7DRbP6Kp14/Y9XFbR/WuDj84bYNKXe4HdDc1P+UMYm16m2L6LkIIoRlx0A5mi+K7jewuGqzFK kaPNmJ0RLCi+4d4/47Zs3DC3PUNOxdOEEHf4kkdWOaSIuj3TQYhNv+LsgF0uijiBmaz2zUFDa2bc IkKakDZfAFM4HoHz8K2BZRaHKWhd3dZua/tlSiqokUFX2DxmHmZ1n5HM9OiaAIXP/Zo2x10j/Yb1 mM3i0bqGahxlHYzl/QyEG/dujp3lewuVjCI0mPDkZGphvxyqp4Jo8qS94EnOqBvxOgftYug7OY9E KY+WkDGCAv8wggL7AgEBMFowRzELMAkGA1UEBhMCU0UxETAPBgNVBAoMCEVyaWNzc29uMSUwIwYD VQQDDBxFcmljc3NvbiBOTCBJbmRpdmlkdWFsIENBIHYzAg8BdaxPA4a6s7niAn6U30wwCQYFKw4D AhoFAKCCAXowGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAxMTEz MDcyNzA2WjAjBgkqhkiG9w0BCQQxFgQUQnfN5VuC3VOVNzBn0S0NlZ9LsQEwQwYJKoZIhvcNAQkP MTYwNDAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAhow aQYJKwYBBAGCNxAEMVwwWjBHMQswCQYDVQQGEwJTRTERMA8GA1UECgwIRXJpY3Nzb24xJTAjBgNV BAMMHEVyaWNzc29uIE5MIEluZGl2aWR1YWwgQ0EgdjMCDwF1rE8DhrqzueICfpTfTDBrBgsqhkiG 9w0BCRACCzFcoFowRzELMAkGA1UEBhMCU0UxETAPBgNVBAoMCEVyaWNzc29uMSUwIwYDVQQDDBxF cmljc3NvbiBOTCBJbmRpdmlkdWFsIENBIHYzAg8BdaxPA4a6s7niAn6U30wwDQYJKoZIhvcNAQEB BQAEggEAHMDzm/LrxWsYBEK/P0Dx/T5PjO/B5t/+5bp53/QuM0DgkaDBHahTgAtKWiSjxyIlBHDn dFS5cr37+/5+suai8RJxIC4+IeCxZntH+KH74CktxC1jIYUPaecs96bQv+aF18C8WzAUMjmpZjBo 5IvVQqzVLLo266Xln/mC1JqD0lA5L5GbHmSal6roHG8aQNOfDeZoOLxlbtTpxj9cLey3BhzUwewg P1FoO/ygkcc3oR3I2lBnxoctg5OrjaS/GMdj5NUKhow9/oriDNyuINGz2P0j6wxdBqOH8PTRi7uS L9G92slxZ5ygY/14tkfsIPEYiOxFqY8EkRHYWbW+QU2OGgAAAAAAAA== ------=_NextPart_000_000B_01D6B996.C58BBA30-- From nobody Fri Nov 13 00:14:10 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9A0083A14C3; Fri, 13 Nov 2020 00:14:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.649 X-Spam-Level: X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mT-8yEf8uq1t; Fri, 13 Nov 2020 00:14:07 -0800 (PST) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B47D53A14ED; Fri, 13 Nov 2020 00:14:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1605255199; bh=CIucrS9zJ3TjfuB+2T6E0+0YTwtThaN5iH9NFjj4Hqg=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=bae0jKt/ICgtf2A1OhVxa43/rirgHEo4fg7syf0cDSZ8bqwBVeJgtcawT848ARL4Z R9IhPKzjbHIcV7j30egg3r111cJakKo9rpcSJhhgauNCiTQvJNlz4ZAIsQLnrfteja D0a8KdN/BcVhk10XBf27QQ+ANX9v4Gbs6WSf83NM= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.250.102] ([134.76.241.253]) by mail.gmx.com (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1McH9i-1k18iJ2PgC-00ci75; Fri, 13 Nov 2020 09:13:19 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) From: Sebastian Moeller In-Reply-To: Date: Fri, 13 Nov 2020 09:13:17 +0100 Cc: tsvwg IETF list , iccrg IRTF list , TCP Prague List , "De Schepper, Koen (Koen)" Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Bob Briscoe X-Mailer: Apple Mail (2.3445.104.17) X-Provags-ID: V03:K1:SUyON4oqldymy1bE2uFJzSZvt9SaKBc3JR7sOgKUJ903gBe5ylI dJ1I4Ds/COZ52l0YQUCh8QqVOGLQdeTHJlhHbudDXHoa6w7pLh3eJK9l7eUE/+5B/njzcue 4jP12d3A2tpbmTkxGJpkVoS15AvqLOO8vGeKVbgPKSPtqv8cL3L6telPayZd93fSuLqBLaf gBNU9cg2OfFENAWykImZQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:bsuzGwrlYzo=:jOuX4AD7dIzw4GLI1+wbLa yDh14NfPAVDXgtsAKCNR1ObRuLsp+3i1SzlRUMHgy4jWDzgv8rLQ9Kww63uHbSe/LOKXVcPzw sZbAYaTTULxwAsdBKnonDWCf8iYSoJznjrpCnsWquQR+LFhlXFEs/TXpSASTBV7Kd36otcQRy h/m4nvS4Ln+BBgPEPXX0R7UxjffEeADnsDTlPa442Ot0XY69N4zd4+d6F77ZirRJTwQLKUMU1 H60kFG9VNbCB52wAjqlTHuT5+Q8PMs0oDuv2I2l+k3Y+puV9Abzv+TdOJ7vNb7SF0iJMn04Dp QLhqCP5ZKhO5QXET+V3/cKPWZcvM6rCKzXmD/7CNZcY+qZ/d2yUSOBeX38ornwr5GoEh+sKN0 x1Viu3wEsamHv927K/+1VvHCXHZ7A55M6dZ5eacDn8g0abrALzjyq5AryRtDWsEP429Af1KNe ByUkULoTOzxtYjIwy/IJ2yCBPf0Y6KBsng4zeLYlvkNlBs6pZXuS0n1F72j+3PNIjHnWrhnwC T8brSRbg1njVEIqY2jsZjsxNFpvSEPXxjcY1ly0x9cdg7RxXFCThbki84DydNvIMIRLqxWqkW kcLCV4DHp+WXGsOn9AfjC+I8pjwHyay3nI9RoP6aBIpVSfXXSVPX8tmRXs2HmfevKHFtt9V8r sLROaY94G+JbAeCR6MRlAoqOgwj8ldvWMTmti9IDWivgBaMFf+4jD12GrsYnPiv+sUIejPm4m wG6l0Za8ZszYOOaxhukAEeD+EOF/Wmp5q5MXUcaXqhxUqJygsQDGVogxA3RK7QHGWWJ+7IFtM e1MBPlJ9PsYpS7o3VIfRAcCWhFQ+d95Y8tltCjR/Zv95U1+ty3NFTjE2L/b/ptir8sbjAauj1 I+F/UetZziQ7uLTpNnqQ== Archived-At: Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative RTT Bias Text (request to reject) X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 08:14:09 -0000 Dear list, given the recent data by Pete Heist and Jonathan Morton, it seems that = the combination of TCP Prague and DualQ introduce a level of RTT bias = not seen before in the internet, including noticeably stronger = suppression of long-RTT Prague throughput by short RTT Prague. Under = that perspective it is clear that L4S does not at all "reduce or = eliminate RTT bias" but makes the situation considerably worse. Allowing = that behavior to stand, by changing the requirement to Bob's proposal = below is IMHO not acceptable, unless our goal in the WG is to built a = fast priority lane for short-RTT flows at the expense of everything = else, like long-RTT flows or god-forbid non TCP Prague flows. The proposed rationale " there is no need to mandate that an L4S = implementer does no harm to themselves" is problematic, if L4S and = L4S-compliant transport protocols are intended as replacement for = traditional CCs, as we clearly would turn the internet into a mostly = short range affair. I have claimed in the past that RTT bias is an unavoidable consequence = of differential "reaction times" for control loops at different delays, = so I always was critical of the "reduce or eliminate RTT bias" mandate, = but I believe firmly that a "do not make RTT (considerably) bias worse" = than it is today should be a non-negotiable MUST requirement for any = L4S-compliant transport. Best Regards Sebastian > On Oct 30, 2020, at 19:42, Bob Briscoe = wrote: >=20 > Folks, >=20 > The co-authors of ECN L4S ID have been reviewing the correctness of = the normative 'Prague' requirements.=20 > See = https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-10#section-4.3 > This is the first of 2 emails, about 2 of the requirements that we = think ought to be reworded a little. >=20 > If you agree with the rationale, but think the new wording still = doesn't capture the requirement well, pls suggest sthg better. > If you disagree with the rationale, pls discuss. >=20 > 4.3. Prerequisite Congestion Response > ... > CURRENT: > o A scalable congestion control MUST reduce or eliminate RTT bias > over as wide a range of RTTs as possible, or at least over the > typical range of RTTs that will interact in the intended > deployment scenario (see=20 > Appendix A.1.5 > for rationale). >=20 >=20 > PROPOSED: > A scalable congestion control MUST eliminate RTT bias as much as = possible in the range between the minimum likely RTT and typical RTTs = expected=20 > in the intended deployment scenario (see Appendix A.1.5 for = rationale). >=20 > RATIONALE: > 1/ "eliminate as much as possible" is stronger than "reduce or = eliminate". > 2/ This requirement was motivated by 'do no harm to others' relative = to existing standard (RFC5681 Reno) congestion control. So there is no = need to mandate that an L4S implementer does no harm to themselves, = which window-based congestion controls tend to do at higher RTT. Of = course, this doesn't preclude implementers reducing or eliminating RTT = bias for larger than typical RTTs, but it removes any requirement to do = so.=20 >=20 > Cheers >=20 >=20 > Bob >=20 > --=20 > ________________________________________________________________ > Bob Briscoe =20 > http://bobbriscoe.net/ From nobody Fri Nov 13 02:16:27 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7DFA23A0769 for ; Fri, 13 Nov 2020 02:16:26 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.896 X-Spam-Level: X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1oLXm8vnjXrx for ; Fri, 13 Nov 2020 02:16:24 -0800 (PST) Received: from pegasus.erg.abdn.ac.uk (pegasus.erg.abdn.ac.uk [IPv6:2001:630:42:150::2]) by ietfa.amsl.com (Postfix) with ESMTP id 37BDB3A0656 for ; Fri, 13 Nov 2020 02:16:21 -0800 (PST) Received: from GF-MacBook-Pro.lan (fgrpf.plus.com [212.159.18.54]) by pegasus.erg.abdn.ac.uk (Postfix) with ESMTPSA id 3BB231B00193; Fri, 13 Nov 2020 10:16:07 +0000 (GMT) To: Ingemar Johansson S , "De Schepper, Koen (Nokia - BE/Antwerp)" , Szilveszter Nadas , "tcpprague@ietf.org" Cc: Ferenc Fejes , Ingemar Johansson S , Gombos Gergo , LAKI Sandor References: From: Gorry Fairhurst Message-ID: Date: Fri, 13 Nov 2020 10:16:06 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:78.0) Gecko/20100101 Thunderbird/78.4.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------ABB35AFED009C96944A6F77F" Content-Language: en-GB Archived-At: Subject: [tcpPrague] TCP Prague preliminary results & Hystart with larger RTT X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 10:16:27 -0000 This is a multi-part message in MIME format. --------------ABB35AFED009C96944A6F77F Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Changing the title, to focus on this obserevation... On 13/11/2020 07:27, Ingemar Johansson S wrote: > > Hi > > Just some food for thought. > > Hybrid Slowstart (HyStart) in Cubic is known to have the problem that > it can exit to congestion avoidance prematurely. We have in fact seen > issues with HyStart in cellular and the outcome is that it over-reacts > to scheduling jitter. > > This issue was mentioned in the long since expired > https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-02 > (section > 4) and a presentation at ICCRG > https://datatracker.ietf.org/meeting/96/materials/slides-96-iccrg-1 > > We've also been recently testing QUIC over broadband satellite systems, and we also observe the same sort of reaction to the path characteristics (likely also due to increased jitter in the timeslot-based transmission of ACKs when used on a shared broadband satellite return link). The result was a forward path that was under-utilised. These experiments used google's (latest) QUIC - and we saw a resulting reduction in the throughput when the RTT was larger. > The Linux TCP stack enables HyStart with both the delay and ack train > algorithms on by default. It could be a good idea to at some stage > rerun the tests with HyStart completely disabled. I am not sure in > this is the reason but believe that it is worth to rule out the > problems with HyStart . > > /Ingemar > > PS HyStart++ developed by Praveen and others at Microsoft  instead > exits to a limited slowstart > https://tools.ietf.org/html/draft-balasubramanian-tcpm-hystartplusplus-03 > . > > This topic isn't new, but I think might need some attention wrt to new transport methods, and might be something to explore for a range of protocols. If it is still a concern, maybe we should open this up perhaps initially in TCPM ? Gorry > > *From:* De Schepper, Koen (Nokia - BE/Antwerp) > > *Sent:* den 12 november 2020 10:35 > *To:* Szilveszter Nadas > ; tcpprague@ietf.org > *Cc:* Ferenc Fejes ; Gombos Gergo > ; LAKI Sandor > *Subject:* Re: [tcpPrague] A Congestion Control Independent L4S > Scheduler - TCP Prague preliminary results > > Hi Szilveszter, > > Interesting work, thanks for sharing. There is indeed a very big > difference between how DCTCP behaves today in recent kernels, than > what we used in the original L4S work testing (Linux kernel 3.19). > That original version was a “clean” DCTCP version that matched very > well the theoretical equations: r=2/(p.RTT) for uniform stable random > and r=2/(p˛.RTT) on/off marking. Due to many interactions, integer > range constraints, pacing/GSO interactions and bugs, the performance > really degraded in the recent Linux versions. These issues were fixed > within the Linux Prague version, so that is why Prague should perform > “better” in respect to matching the equations. It would be interesting > to get an idea of the deviation from the r=2/(p.RTT) equation in your > tests (relevant in DualPI2 as the coupling gives a smooth > probability). Collecting the average RTT (base + queuing latency), > marking probability, and rate would make it possible to evaluate this. > > Main reasons why we saw deviations is when the probability varies a > lot in time. As you might know DCTCP’s (and Prague’s) equation becomes > r=2/(p˛.RTT) when it receives on/off marking episodes (on episodes in > the order of 1 RTT, off in the order of multiple RTTs). Any not so > stable marking probability results in a rate between those 2 > boundaries. This is the theory, looking forward on how your practice > matches this. > > From a first quick look at the results it might not deviate that much, > I guess. The more aggressive part is probably due to the RTT > unfairness: 1 to 5 rate ratio (with a 5ms base RTT, Classic gets a > queue of 5ms+20ms target, so about 25ms and 5 times less throughput). > Did you try to use the RTT independent version? If you set it to the > f=max(15, RTT) mode, the minimum effective RTT becomes 15ms, so the > rate ratio is 3 to 5 in that case. > > Thanks and Regards, > > Koen. > > *From:* tcpPrague > *On Behalf Of *Szilveszter Nadas > *Sent:* Tuesday, November 3, 2020 6:30 PM > *To:* tcpprague@ietf.org > *Cc:* Ferenc Fejes >; > Gombos Gergo >; LAKI > Sandor > > *Subject:* [tcpPrague] A Congestion Control Independent L4S Scheduler > - TCP Prague preliminary results > > Hi all, > > During the review of our article “A Congestion Control Independent L4S > Scheduler” we received comments from one of the reviewers, on why we > did not also evaluate TCP Prague. So now we rerun the test cases with > replacing DCTCP to TCP Prague. > > The results are at the below link. We are somewhat surprised by much > increased aggressiveness of TCP Prague. Can you comment on it? The > settings and setup we used are described in the pdf (and in the > original article). > > Results: http://ppv.elte.hu/tcp-prague/ > > Original article (including YouTube presentation): > http://ppv.elte.hu/cc-independent-l4s/ > > > Can you comment on the results and the settings we used? > > Cheers, > > Szilveszter > > > _______________________________________________ > tcpPrague mailing list > tcpPrague@ietf.org > https://www.ietf.org/mailman/listinfo/tcpprague --------------ABB35AFED009C96944A6F77F Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit
Changing the title, to focus on this obserevation...

On 13/11/2020 07:27, Ingemar Johansson S wrote:

Hi

 

Just some food for thought.

Hybrid Slowstart (HyStart) in Cubic is known to have the problem that it can exit to congestion avoidance prematurely. We have in fact seen issues with HyStart in cellular and the outcome is that it over-reacts to scheduling jitter.

This issue was mentioned in the long since expired https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-02 (section 4) and a presentation at ICCRG https://datatracker.ietf.org/meeting/96/materials/slides-96-iccrg-1

We've also been recently testing QUIC over broadband satellite systems, and we also observe the same sort of reaction to the path characteristics (likely also due to increased jitter in the timeslot-based transmission of ACKs when used on a shared broadband satellite return link). The result was a forward path that was under-utilised. These experiments used google's (latest) QUIC - and we saw a resulting reduction in the throughput when the RTT was larger.

The Linux TCP stack enables HyStart with both the delay and ack train algorithms on by default. It could be a good idea to at some stage rerun the tests with HyStart completely disabled. I am not sure in this is the reason but believe that it is worth to rule out the problems with HyStart .

 

/Ingemar

 

PS HyStart++ developed by Praveen and others at Microsoft  instead exits to a limited slowstart https://tools.ietf.org/html/draft-balasubramanian-tcpm-hystartplusplus-03.

 

 

This topic isn't new, but I think might need some attention wrt to new transport methods, and might be something to explore for a range of protocols. If it is still a concern, maybe we should open this up perhaps initially in TCPM ?

Gorry

 

From: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-bell-labs.com>
Sent: den 12 november 2020 10:35
To: Szilveszter Nadas <Szilveszter.Nadas=40ericsson.com@dmarc.ietf.org>; tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; Gombos Gergo <ggombos@inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results

 

Hi Szilveszter,

 

Interesting work, thanks for sharing. There is indeed a very big difference between how DCTCP behaves today in recent kernels, than what we used in the original L4S work testing (Linux kernel 3.19). That original version was a “clean” DCTCP version that matched very well the theoretical equations: r=2/(p.RTT) for uniform stable random and r=2/(p˛.RTT) on/off marking. Due to many interactions, integer range constraints, pacing/GSO interactions and bugs, the performance really degraded in the recent Linux versions. These issues were fixed within the Linux Prague version, so that is why Prague should perform “better” in respect to matching the equations. It would be interesting to get an idea of the deviation from the r=2/(p.RTT) equation in your tests (relevant in DualPI2 as the coupling gives a smooth probability). Collecting the average RTT (base + queuing latency), marking probability, and rate would make it possible to evaluate this.

 

Main reasons why we saw deviations is when the probability varies a lot in time. As you might know DCTCP’s (and Prague’s) equation becomes r=2/(p˛.RTT) when it receives on/off marking episodes (on episodes in the order of 1 RTT, off in the order of multiple RTTs). Any not so stable marking probability results in a rate between those 2 boundaries. This is the theory, looking forward on how your practice matches this.

 

From a first quick look at the results it might not deviate that much, I guess. The more aggressive part is probably due to the RTT unfairness: 1 to 5 rate ratio (with a 5ms base RTT, Classic gets a queue of 5ms+20ms target, so about 25ms and 5 times less throughput). Did you try to use the RTT independent version? If you set it to the f=max(15, RTT) mode, the minimum effective RTT becomes 15ms, so the rate ratio is 3 to 5 in that case.

 

Thanks and Regards,

Koen.

 

From: tcpPrague <tcpprague-bounces@ietf.org> On Behalf Of Szilveszter Nadas
Sent: Tuesday, November 3, 2020 6:30 PM
To: tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; Gombos Gergo <ggombos@inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results

 

Hi all,

 

During the review of our article “A Congestion Control Independent L4S Scheduler” we received comments from one of the reviewers, on why we did not also evaluate TCP Prague. So now we rerun the test cases with replacing DCTCP to TCP Prague.

 

The results are at the below link. We are somewhat surprised by much increased aggressiveness of TCP Prague. Can you comment on it? The settings and setup we used are described in the pdf (and in the original article).

Results: http://ppv.elte.hu/tcp-prague/

Original article (including YouTube presentation): http://ppv.elte.hu/cc-independent-l4s/

 

Can you comment on the results and the settings we used?

 

Cheers,

Szilveszter


_______________________________________________
tcpPrague mailing list
tcpPrague@ietf.org
https://www.ietf.org/mailman/listinfo/tcpprague


--------------ABB35AFED009C96944A6F77F-- From nobody Fri Nov 13 08:37:40 2020 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 34D943A0EE5 for ; Fri, 13 Nov 2020 08:37:39 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7dMxL18ZWitw for ; Fri, 13 Nov 2020 08:37:36 -0800 (PST) Received: from NAM06-DM3-obe.outbound.protection.outlook.com (mail-eopbgr640120.outbound.protection.outlook.com [40.107.64.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BFEB53A0EDF for ; Fri, 13 Nov 2020 08:37:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JTWH+4JGMFh0iX8jlb81rAxmS5OrnxHDukpM0XnxsKZ+HsSOCES9wbNVqE9dBPiEemnkVTBoxeqmpcp/yKait2UTz6zcvMvjFWkJOeZ8aRg7l/DN6BoXnJHjxejqwgdk5RIteL8Y5TpcsIhr64q4UnIisJPMCb7nNN40+/vK1FTdV8GKN/FGrLHEKB7n3w+0EZqxJ6YmPsDNzFDxlS/agcrMM6dyJf/X3aecDAoii8TtSY8Or9fFZJ4j44Neb2TaRLAl9upmCgJIKenjMFYz4SLXMNQikOH2/mzyJXTrZ88x9pW4BARmo5V1vJRE6EcsrDzY+RtsrzmaCMKR8BjBgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R/QMMRnmNMn6da1hNo772UdRMBPJ01jrYRy+cio/NzY=; b=kmkSsFulzsURZP0evD7NecCY15/trbfo6i2kDiUxY90dJ1+l3Zt5TMpzXHgGjl1nv3uHSGtuqtYyBmwIFX5zsBNL4IoKzeZV9/O50T+TpLipWRhajr/Lue9bcMx70w1y4ZmaMdkXI1U85ahixKK4PvgN04k8BLFAWji6SzLTmg+7EjY9qJLvk83JvF6bCV2bs2hY33lSsV9RlZPR4qddEImrzMW5N677THGvzj5UpcplqIoabZmatizcnxzj2BSSg39bUZVFKEzUNRBk2eWx+zsuQwf44437UN7Mx+LGsCuasFZzoL+CjTSuh4x0GNHU5Sli1fLSofeTaLLMh26mBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=R/QMMRnmNMn6da1hNo772UdRMBPJ01jrYRy+cio/NzY=; b=LYmH6XrgI5c1x5mjhLAyp6uOYNVrBxEW/02X0gfwVHaJP3nW1TTgYnBq5522G3imdlfKr3xCWZK2PnSgHaaHtZ35m76keKJMmFRYKGGH4bDXq0LTYztYB3K9dyEBUBWTZKJJp55sfVCbrbPmnn0TE7hcSXEUvv9tWxKvG9VRRYA= Received: from (2603:10b6:5:166::18) by DM6PR00MB0783.namprd00.prod.outlook.com (2603:10b6:5:1bd::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3610.0; Fri, 13 Nov 2020 16:37:30 +0000 Received: from DM6PR00MB0633.namprd00.prod.outlook.com ([fe80::c102:dbde:b6e0:48ff]) by DM6PR00MB0633.namprd00.prod.outlook.com ([fe80::c102:dbde:b6e0:48ff%4]) with mapi id 15.20.3614.000; Fri, 13 Nov 2020 16:37:30 +0000 From: Praveen Balasubramanian To: "ingemar.s.johansson=40ericsson.com@dmarc.ietf.org" , "koen.de_schepper@nokia-bell-labs.com" , "Szilveszter.Nadas=40ericsson.com@dmarc.ietf.org" , "tcpprague@ietf.org" CC: "fejes@inf.elte.hu" , "ingemar.s.johansson@ericsson.com" , "ggombos@inf.elte.hu" , "lakis@inf.elte.hu" Thread-Topic: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results Thread-Index: AQHWuNc6oA6xqubGIUqvKlTLcTC9ranFqxyAgACZLyA= Date: Fri, 13 Nov 2020 16:37:29 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=86d2fd11-2a75-4a03-92a7-952609490252; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2020-11-13T16:35:25Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; authentication-results: dmarc.ietf.org; dkim=none (message not signed) header.d=none;dmarc.ietf.org; dmarc=none action=none header.from=microsoft.com; x-originating-ip: [2001:4898:80e8:f:95c9:3f73:f682:d302] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: de6ff06b-6016-4f91-15ab-08d887f269d4 x-ms-traffictypediagnostic: DM6PR00MB0783: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: npeu2gwFEYGDJG4+EaIQ5A26VjQbG5tpn2NTE2sOmNRryyfoOU31/whLv6bvfKywvF8kR6R9Zwc6Rx+NK2vc7ixyHVWtvtjDPx5SQB3EpZInJm2azm0NJx26CC+4ezwXnPAymhXR8nbdUJA+BF/UICNHSyTN/aqTgmAv6sFKYwX4rIbI/FAEc+uIekm1VwyQTKxuVE7I0/Xb1nD+VBGvhlqfB03KQrs06E+5GRUMXsC8sDkgwniQxKZQe3tOfn0MI2vd6/xZFWLYoTVO7+GXMcgssMZNgY4O/p2156B5dKx17p0hIP+3+Q9Z3htqWbYZNgMoKFYTTD/rUqxIX6ObBEn1svtagMjbjNTNlGS01iB1u7f9ovVEHJKDVb4Z8l0u8qw7z02qHzzY8UXuH4z+dp5ohg/T98P/te580+LnZwMxnh2kH8imrRNKhpy+s2u/ADjHRtQbVn/PD17apoVBDQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR00MB0633.namprd00.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(396003)(376002)(346002)(366004)(966005)(76116006)(71200400001)(54906003)(166002)(6506007)(316002)(9686003)(66476007)(83380400001)(86362001)(64756008)(55016002)(110136005)(66556008)(66946007)(66446008)(33656002)(83080400002)(478600001)(8676002)(4326008)(2906002)(53546011)(7696005)(5660300002)(82950400001)(10290500003)(82960400001)(186003)(8990500004)(8936002)(52536014); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?rsC4yJeLs/tU49c4qPKfJKlvifWl0MtRuc2XLOwPVwKd64ngkXcOYglY8o?= =?iso-8859-1?Q?ebd/oK6cU7WUca4JNf1UELDj0dDxE7EsneE7dKmihTnimCU0oOijnzTniv?= =?iso-8859-1?Q?Zpwj83nWN95lbvoclZ3hUREMxGzFwXZZLRjnEWu8LfT9SGC73M3bhw3iuQ?= =?iso-8859-1?Q?N37aQQIqcZks8C75xif6gaV11/AEOoTc5uuDumzJDcFxNkZgFx9lN4XlVM?= =?iso-8859-1?Q?jNh+ra8kw8XO9oTVI7canX+DpiTNosLrPV/Ro5nysGXowxRCdOeQYuoIfw?= =?iso-8859-1?Q?QxUrmlsSKlCtX+anEK3N7hpRm/aJwJiAcAEMWyqDh7tL3eRJ/8eQt5Qp70?= =?iso-8859-1?Q?q1prRNMyBCsLJawxLcL9KAdDEssgUs4qtE5XgWoEiijgwIzpN8C7XNZZbg?= =?iso-8859-1?Q?+yNnyXl9r24pwZOFGkumaP4yylLfrqzxGyqnCC2RQi5QhrL3CAM3gs9dGq?= =?iso-8859-1?Q?tINXHnl1dEs/WkKKfSUjX7qqW3VgM+3yDYluSpusURfaxkEhSDYfLlYJv+?= =?iso-8859-1?Q?Ot2jUICFGkAV9XkI+TmNV+c9FYU/nPW+oeP8AJdBLGwh9VqFYJaGPfKPBt?= =?iso-8859-1?Q?jYsZx2qeKSGDJq/Qep/SRh3gji2Aoj8E/w55RhnuX8S/mez8TtagJ23giz?= =?iso-8859-1?Q?zA4oDy1Qc88rJxIiJZFo1qyk1CffPCIz1c9ygH1CCcc1LOY1UU6wBBkLW3?= =?iso-8859-1?Q?iRKCArEFJy/s0XNYoSkc8R+b0z3X4EqF8fp7Q7I+VpbSbfffr5419Xl7Re?= =?iso-8859-1?Q?hHEZx2w0LXvshMO98Y2EDIe5xtRvbSj/yAeaGnOGFEUydjOc9M0YNif7W+?= =?iso-8859-1?Q?SXdZBe7aDu6pN/TkBao2rrVWN7noN2G9+qOT6BsAzwCqB0zfvtRz9+bIVG?= =?iso-8859-1?Q?vF53vVgNtTaGvBD5VnkxvueM6slyhsA9NiE37w5nPlKfZNrFzpA0IHBi4J?= =?iso-8859-1?Q?eEQtIvq3z0otQlGAYurOvegz89CmGNZVSjIDknQKWDyFmCRyl0DVgehVDJ?= =?iso-8859-1?Q?79A8C7jWzMf6T+WMo62e7uE7ya8W+27flXLgz0mlZTd+0gXdx8Efbx/bK6?= =?iso-8859-1?Q?U5MTWj2s/Ui3Fv6hd3qkI/I=3D?= x-ms-exchange-transport-forked: True Content-Type: multipart/alternative; boundary="_000_DM6PR00MB06336C79B89BB439E7E97BFCB6E61DM6PR00MB0633namp_" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR00MB0633.namprd00.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: de6ff06b-6016-4f91-15ab-08d887f269d4 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Nov 2020 16:37:29.9268 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: un6GW5N3lBt4o5jGjbtQSQDiW88RseRSfNvbdgNvHG1IZBvHgAYjau2h2zQIUFf3oC+/mgFIkgPB5UgbjVzw3jslQZb8IbJXzggHqqopsKw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR00MB0783 Archived-At: Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP Prague preliminary results X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 16:37:39 -0000 --_000_DM6PR00MB06336C79B89BB439E7E97BFCB6E61DM6PR00MB0633namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Minor correction: the most recent HyStart++ draft is draft-ietf-tcpm-hystar= tplusplus-00 - HyStart++: Modified Slow Start for TCP. Only the delay increase algorit= hm is used. We have some work planned for measurements on Cellular networks= . From: tcpPrague On Behalf Of Ingemar Johansson= S Sent: Thursday, November 12, 2020 11:27 PM To: De Schepper, Koen (Nokia - BE/Antwerp) ; Szilveszter Nadas ; tcpprague@ietf.org Cc: Ferenc Fejes ; Ingemar Johansson S ; Gombos Gergo ; LAKI Sandor Subject: [EXTERNAL] Re: [tcpPrague] A Congestion Control Independent L4S Sc= heduler - TCP Prague preliminary results Hi Just some food for thought. Hybrid Slowstart (HyStart) in Cubic is known to have the problem that it ca= n exit to congestion avoidance prematurely. We have in fact seen issues wit= h HyStart in cellular and the outcome is that it over-reacts to scheduling = jitter. This issue was mentioned in the long since expired https://tools.ie= tf.org/html/draft-johansson-cc-for-4g-5g-02 (section 4) and a presentation at ICCRG https:= //datatracker.ietf.org/meeting/96/materials/slides-96-iccrg-1 The Linux TCP stack enables HyStart with both the delay and ack train algor= ithms on by default. It could be a good idea to at some stage rerun the tes= ts with HyStart completely disabled. I am not sure in this is the reason bu= t believe that it is worth to rule out the problems with HyStart . /Ingemar PS HyStart++ developed by Praveen and others at Microsoft instead exits to= a limited slowstart https://tools.ietf.org/html/draft-balasubramanian-tcpm= -hystartplusplus-03. From: De Schepper, Koen (Nokia - BE/Antwerp) > Sent: den 12 november 2020 10:35 To: Szilveszter Nadas >; tcpprague@ietf.or= g Cc: Ferenc Fejes >; Gombos Gerg= o >; LAKI Sandor > Subject: Re: [tcpPrague] A Congestion Control Independent L4S Scheduler - T= CP Prague preliminary results Hi Szilveszter, Interesting work, thanks for sharing. There is indeed a very big difference= between how DCTCP behaves today in recent kernels, than what we used in th= e original L4S work testing (Linux kernel 3.19). That original version was = a "clean" DCTCP version that matched very well the theoretical equations: r= =3D2/(p.RTT) for uniform stable random and r=3D2/(p=B2.RTT) on/off marking.= Due to many interactions, integer range constraints, pacing/GSO interactio= ns and bugs, the performance really degraded in the recent Linux versions. = These issues were fixed within the Linux Prague version, so that is why Pra= gue should perform "better" in respect to matching the equations. It would = be interesting to get an idea of the deviation from the r=3D2/(p.RTT) equat= ion in your tests (relevant in DualPI2 as the coupling gives a smooth proba= bility). Collecting the average RTT (base + queuing latency), marking proba= bility, and rate would make it possible to evaluate this. Main reasons why we saw deviations is when the probability varies a lot in = time. As you might know DCTCP's (and Prague's) equation becomes r=3D2/(p=B2= .RTT) when it receives on/off marking episodes (on episodes in the order of= 1 RTT, off in the order of multiple RTTs). Any not so stable marking proba= bility results in a rate between those 2 boundaries. This is the theory, lo= oking forward on how your practice matches this. >From a first quick look at the results it might not deviate that much, I gu= ess. The more aggressive part is probably due to the RTT unfairness: 1 to 5= rate ratio (with a 5ms base RTT, Classic gets a queue of 5ms+20ms target, = so about 25ms and 5 times less throughput). Did you try to use the RTT inde= pendent version? If you set it to the f=3Dmax(15, RTT) mode, the minimum ef= fective RTT becomes 15ms, so the rate ratio is 3 to 5 in that case. Thanks and Regards, Koen. From: tcpPrague > On Behalf Of Szilveszter Nadas Sent: Tuesday, November 3, 2020 6:30 PM To: tcpprague@ietf.org Cc: Ferenc Fejes >; Gombos Gerg= o >; LAKI Sandor > Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler - TCP P= rague preliminary results Hi all, During the review of our article "A Congestion Control Independent L4S Sche= duler" we received comments from one of the reviewers, on why we did not al= so evaluate TCP Prague. So now we rerun the test cases with replacing DCTCP= to TCP Prague. The results are at the below link. We are somewhat surprised by much increa= sed aggressiveness of TCP Prague. Can you comment on it? The settings and s= etup we used are described in the pdf (and in the original article). Results: http://ppv.elte.hu/tcp-prague/ Original article (including YouTube presentation): http://ppv.elte.hu/cc-in= dependent-l4s/ Can you comment on the results and the settings we used? Cheers, Szilveszter --_000_DM6PR00MB06336C79B89BB439E7E97BFCB6E61DM6PR00MB0633namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Minor correction: the most recent HyStart++ draft is= draft-ietf-tcpm-hystartplusplus-00 - HyStart++: Modified Slow Start for TCP= . Only the delay increase algorithm is used. We have some work planned = for measurements on Cellular networks.

 

From: tcpPrague <tcpprague-bounces@ietf.or= g> On Behalf Of Ingemar Johansson S
Sent: Thursday, November 12, 2020 11:27 PM
To: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@noki= a-bell-labs.com>; Szilveszter Nadas <Szilveszter.Nadas=3D40ericsson.c= om@dmarc.ietf.org>; tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.elte.hu>; Ingemar Johansson S <= ingemar.s.johansson@ericsson.com>; Gombos Gergo <ggombos@inf.elte.hu&= gt;; LAKI Sandor <lakis@inf.elte.hu>
Subject: [EXTERNAL] Re: [tcpPrague] A Congestion Control Independent= L4S Scheduler - TCP Prague preliminary results

 

Hi

 

Just some food for thought.

Hybrid Slowstart (HyStart) in Cubic is known to have= the problem that it can exit to congestion avoidance prematurely. We have = in fact seen issues with HyStart in cellular and the outcome is that it ove= r-reacts to scheduling jitter. This issue was mentioned in the long since expired https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-02 (section 4)= and a presentation at ICCRG https://datatracker.ietf.org/meeting/96/materials/slides-96-iccrg-1

The Linux TCP stack enables HyStart with both the de= lay and ack train algorithms on by default. It could be a good idea to at s= ome stage rerun the tests with HyStart completely disabled. I am not sure i= n this is the reason but believe that it is worth to rule out the problems with HyStart .

 

/Ingemar

 

PS HyStart++ developed by Praveen and others at Micr= osoft  instead exits to a limited slowstart https://tools.ietf.org/html/draft-balasubramanian-tcpm-hystartplusplus-03.

 

 

 

From: De Schepper, Koen (Nokia - BE/Antwerp) = <koen.de_scheppe= r@nokia-bell-labs.com>
Sent: den 12 november 2020 10:35
To: Szilveszter Nadas <Szilveszter.Nadas=3D40ericsson.com@dmarc.ietf.or= g>; tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.= elte.hu>; Gombos Gergo <gg= ombos@inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: Re: [tcpPrague] A Congestion Control Independent L4S Schedu= ler - TCP Prague preliminary results

 

Hi Szilveszter,

 

Interesting work, thanks for sharing. There is indee= d a very big difference between how DCTCP behaves today in recent kernels, = than what we used in the original L4S work testing (Linux kernel 3.19). Tha= t original version was a “clean” DCTCP version that matched very well the theoretical equations: r=3D2/(p.RTT) fo= r uniform stable random and r=3D2/(p=B2.RTT) on/off marking. Due to many in= teractions, integer range constraints, pacing/GSO interactions and bugs, th= e performance really degraded in the recent Linux versions. These issues were fixed within the Linux Prague version, s= o that is why Prague should perform “better” in respect to matc= hing the equations. It would be interesting to get an idea of the deviation= from the r=3D2/(p.RTT) equation in your tests (relevant in DualPI2 as the coupling gives a smooth probability). Collecti= ng the average RTT (base + queuing latency), marking probability, and rate = would make it possible to evaluate this.

 

Main reasons why we saw deviations is when the proba= bility varies a lot in time. As you might know DCTCP’s (and Prague= 217;s) equation becomes r=3D2/(p=B2.RTT) when it receives on/off marking ep= isodes (on episodes in the order of 1 RTT, off in the order of multiple RTTs). Any not so stable marking probability results= in a rate between those 2 boundaries. This is the theory, looking forward = on how your practice matches this.

 

From a first quick look at the results it might not = deviate that much, I guess. The more aggressive part is probably due to the= RTT unfairness: 1 to 5 rate ratio (with a 5ms base RTT, Classic gets a que= ue of 5ms+20ms target, so about 25ms and 5 times less throughput). Did you try to use the RTT independent versi= on? If you set it to the f=3Dmax(15, RTT) mode, the minimum effective RTT b= ecomes 15ms, so the rate ratio is 3 to 5 in that case.

 

Thanks and Regards,

Koen.

 

From: tcpPrague <tcpprague-bounces@ietf.org> On Behalf Of Szilveszter Nadas
Sent: Tuesday, November 3, 2020 6:30 PM
To: tcpprague@ietf.org
Cc: Ferenc Fejes <fejes@inf.= elte.hu>; Gombos Gergo <gg= ombos@inf.elte.hu>; LAKI Sandor <lakis@inf.elte.hu>
Subject: [tcpPrague] A Congestion Control Independent L4S Scheduler = - TCP Prague preliminary results

 

Hi all,

 

During the review of our article “A Congestion= Control Independent L4S Scheduler” we received comments from one of = the reviewers, on why we did not also evaluate TCP Prague. So now we rerun = the test cases with replacing DCTCP to TCP Prague.

 

The results are at the below link. We are somewhat s= urprised by much increased aggressiveness of TCP Prague. Can you comment on= it? The settings and setup we used are described in the pdf (and in the or= iginal article).

Results: http://ppv.elte.hu/tcp-prague/

Original article (including YouTube presentation): <= a href=3D"https://nam06.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%= 2Fppv.elte.hu%2Fcc-independent-l4s%2F&data=3D04%7C01%7Cpravb%40microsof= t.com%7Ca630feb0fdfc4118c97708d887a5922b%7C72f988bf86f141af91ab2d7cd011db47= %7C1%7C0%7C637408492508531361%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL= CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3Dn3J1Txb%2B= vqsuhzmZVxCu0ch5Nr%2B5wwrn2LJNkmyWK%2Fs%3D&reserved=3D0"> http://ppv.elte.hu/cc-independent-l4s/

 

Can you comment on the results and the settings we u= sed?

 

Cheers,

Szilveszter

--_000_DM6PR00MB06336C79B89BB439E7E97BFCB6E61DM6PR00MB0633namp_--