From nobody Fri Jul 24 06:36:24 2015 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 53DC31A910E; Fri, 24 Jul 2015 04:25:06 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NE62uBf6hPV9; Fri, 24 Jul 2015 04:25:04 -0700 (PDT) Received: from ietfa.amsl.com (localhost [IPv6:::1]) by ietfa.amsl.com (Postfix) with ESMTP id 84FAC1A904F; Fri, 24 Jul 2015 04:25:04 -0700 (PDT) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit From: IETF Secretariat To: "IETF Announcement List" X-Test-IDTracker: no X-IETF-IDTracker: 6.1.0.p3 Auto-Submitted: auto-generated Precedence: bulk Message-ID: <20150724112504.14108.82770.idtracker@ietfa.amsl.com> Date: Fri, 24 Jul 2015 04:25:04 -0700 Archived-At: X-Mailman-Approved-At: Fri, 24 Jul 2015 06:36:21 -0700 Cc: ietf@bobbriscoe.net, mirja.kuehlewind@tik.ee.ethz.ch, tcpprague@ietf.org Subject: [Tcpprague] New Non-WG Mailing List: TCP Prague -- Evolution of DCTCP designed to live alongside other TCP variants and derivatives X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.15 Reply-To: ietf@ietf.org List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jul 2015 11:25:06 -0000 A new IETF non-working group email list has been created. List address: tcpprague@ietf.org Archive: https://mailarchive.ietf.org/arch/browse/tcpprague/ To subscribe: https://www.ietf.org/mailman/listinfo/tcpprague Purpose: This mailing list is to coordinate parallel implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of Data Centre TCP (DCTCP) designed to live alongside other TCP variants and derivatives. DCTCP has demonstrated how minimal latency could be, but it is only applicable in controlled environments such as data centers. This is because the congestion control in DCTCP dominates existing variants (Reno, Cubic, etc) and DCTCP redefines the meaning of TCP-ECN feedback. The defining features of TCP Prague will be: o Designed to exploit the full potential of ECN in the network o Marking frequency invariant with flow rate. For additional information, please contact the list administrators. From nobody Tue Jul 28 06:01:13 2015 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 916EB1A8AA8 for ; Tue, 28 Jul 2015 06:01:12 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.9 X-Spam-Level: X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_50=0.8, GB_I_INVITATION=-2, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vGYVhNPONJFT for ; Tue, 28 Jul 2015 06:01:01 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 59D171A8AA9 for ; Tue, 28 Jul 2015 06:00:51 -0700 (PDT) Received: from 83.58.199.146.dyn.plus.net ([146.199.58.83]:37741 helo=[192.168.0.11]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1ZK4Uw-0007Yv-Qe for tcpPrague@ietf.org; Tue, 28 Jul 2015 14:00:49 +0100 Message-ID: <55B77CFE.9090505@bobbriscoe.net> Date: Tue, 28 Jul 2015 14:00:46 +0100 From: Bob Briscoe User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 To: TCP Prague IETF List References: <55AD6C26.5050101@bobbriscoe.net> <697AD251-AF73-4DE0-896C-C00B446CC623@ifi.uio.no> <78961a4cc0cbcc1f802604072abf3cc2.squirrel@erg.abdn.ac.uk> <55AE3EA8.1070804@bobbriscoe.net> In-Reply-To: <55AE3EA8.1070804@bobbriscoe.net> Content-Type: multipart/alternative; boundary="------------050605040209060201060105" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net Archived-At: Subject: [tcpPrague] Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jul 2015 13:01:12 -0000 This is a multi-part message in MIME format. --------------050605040209060201060105 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Folks, These notes have taken a week, because I've only just put my machine back together after having to rebuild the hardware a little :| _*Notes: DCTCP Evolution Bar BoF*_ 6-7pm Tue 21 Jul 2015, Budapest room, The Hilton, Prague, CZ **Summary of Actions:* *Lars E: Set up tcpprague wiki page Bob B: Request tcpprague@ietf.org mailing list, via IETF process (requires Area Director approval) Bob B: Document Rationale - initiate a para on wiki. Lars E: fwd Dagstuhl invitee list to Bob Bob B: Set up list on wiki to assign people to invite those not in the room to join. * * 18:00 Introductions - name and interest ** ***Present: Marcelo Bagnulo Braun Praveen Balasubramanian Martin Bekker Bob Briscoe Anna Brunstrom Stuart Cheshire Koen De Schepper Fabien Duchen Phil Eardley Lars Eggert Michio Honda Per Hurtig Jana Iyengar Naeem Khademi Mirja Kuehlewind Matt Mathis > Andrew McGregor Karen Nielsen Tommy Pauly Andreas Petlund > Costin Raiciu Pasi Sarolahti Richard Scheffenegger David Schinazi Randall Stewart Dave Thaler Brian Trammell > Michael Tuexen Felix Weinrank Michael Welzl Alex Zimmermann ** Scope and Agenda Bashing** *** [Non-italic text is from the materials pre-prepared by Koen De Schepper and Bob Briscoe. /Italic text summarises conversation in the room./] Meeting is covered by the standard IETF "Note Well" concerning intellectual property. *Scope*: * Evolving the e2e DCTCP protocol for use alongside existing traffic (whether in DCs, private nets or public Internet). * Primarily to get DCTCP /developers/ involved early (Windows, FreeBSD, Linux), so that whatever we decide to standardise can be implemented in parallel (Doing implementation and standardisation in series is not desirable, in whichever order). * Primarily an organisational meeting about creating a forum / community to do this work, using people's experience to know what will work best. *Not in Scope:*** * Network changes are not in scope unless they impact the list of changes needed to DCTCP * The in-network side of the solution (two approaches exist [DCttH , Judd15 ], others may follow).* ** Identifier of DCTCP-like traffic (please discuss by email, not in this meeting) /Lars E: Informational draft recording Microsoft's DCTCP should not be stalled by this, as it has value of its own. Unanimous agreement. / /Praveen S: Microsoft has offered a royalty free license for DCTCP IPR ./ /Karen N: Is DCTCP over a non-TCP transport (e.g. SCTP) in scope//? Unanimous "Yes"// / /Outcome of discussion on the features of this DCTCP-like congestion control that define this work:// / 1. /Must use ECN, but unlike RFC3168 ECN, marking is not merely equivalent to drop,// //so ECN signals can be more plentiful and sooner than drop./ 2. /P//acket rate is proportional to 1/p//, where p is the ECN marking probability. // / /Matt M: 1/p makes congestion control scale with the bandwidth, //by making//the intensity of congestion control signals per RTT invariant//./ //// /Stuart Ch: Apple is turning on ECN by default in clients. Currently in developer seeds but probably in the next releases. Packet loss is also not a mystery./ ** 18:15 List of /must-have/ changes before deployment alongside existing traffic.** *** /Matt M: Rather than a "MUST-have" list, produce a prioritised list, because where to draw the necessity line could depend on the use-case.// / The following list wasn't formally prioritised in the meeting, but items where some people questioned necessity are shifted down. 1. Fall back to Reno or Cubic behaviour on loss; /For how long?//quick consensus: 1 RTT, but needs further discussion. ECN response continues to operate in parallel./ 2. Negotiate altered feedback semantics, to convey the extent of ECN marking, not just its existence, and this feedback needs to be robust to loss [RFC-to-be 7560 ]; /Mirja K, Richard S & Bob B plan to have spec of much simpler solution out soon. / 3. Use of a standardised packet identifier (if ECN-capable is not enough) /Identifier tbd. /*/- - - 8< - - - - - - - - highest line between "must-have for safety" and "would be nice for performance" - - - - - - 8< - - - -/* 4. Handle a window of less than 2 when the RTT is low, rather than increase the queue [TCP-sub-mss-cwnd ], like TCP Nice . /Michael W: Is this "must-have"? Quite a complicated step. // //Bob B: Yes, but, otherwise DCTCP will pollute ultra-low latency queues from the start./ 5. Average ECN feedback over its own RTT, not the hard-coded RTT suitable only for data-centres, perhaps reduce cwnd by seg-size/2 per ECN Echo, like Relentless TCP [Mathis09 ]; /???: How bad would long-RTT flows be? More generally, how can we evaluate all this?/ /Bob B: With mixed RTTs, flows with RTT > a couple of ms will respond too quickly to bursts.//Whatever, it's already been implemented by Mohammad Alizadeh in Linux, and evaluated , so this is easy./ 6. Heuristic testing for classic ECN bottlenecks //The idea would be to detect a 'classic' RFC316 bottleneck by whether appreciable delay growth accompanies the marking (originally suggested by Michael W). /Bob B: Complex and slow to detect, so it would have to learn and cache for new flows - suggest this should only be a must-have if measurements prove it to be a problem - i.e. if a significant proportion of classic ECN bottlenecks //exist// //Matt M: No need for this - rate mismatch //no worse than TCP already sees with RTT discrepancies./ /* - - - 8< - - - - - - - - lowest line between "must-have for safety" and "would be nice for performance" - - - - - - 8< - - - -*/ 7. /Costin R: //Faster-than-additive increase//(similar to Cubic) //A performance improvement, not "must-have", but would be nice to have while we're doing this./ 8. /[Not discussed in the meeting, but added by Bob B for the record]: Less drastic exit from slow-start, to match less drastic rate reduction per mark.// //Currently, because marking threshold is shallow, //slow start exits earlier than with drop, unnecessarily increasing completion time.// / / //Costin R: Any other way to evolve towards DCTCP over mixed networks, without separate queues in the network?// ////Bob B: To discuss on ML, and if we continue with the proposed approach, we must record the rationale on the WIki.// /// ** 18:30 Brainstorm to identify people not present who will be important to this.** *** Mohammad Alizadeh Grenville Armitage Fred Baker Stephen Bensley Daniel Borkmann Yuchung Cheng Kenjiro Cho 邓灵莉/Lingli Deng Eric Dumazet Gorry Fairhurst Jamal Hadi Salim Glenn Judd Midori Kato Kenneth Klette Jonassen > (already subscribed) Sridharan, Murari Hiren Panchasara Hagen Pfeifer Balaji Prabhakar KK Ramakrishnan Lawrence Stewart Dave Taht Florian Westphal /Agreed to cc to the following for awareness, but no need to invite to join the list:// /Stephen Hemminger David Miller /Missing types of organisations:// / * /Network operators (not so relevant for e2e protocol, but need to be motivated to deploy the network part)/ * /CDN//s/ /[Bob B adds: Subsequent to mtg, Erik Nygren tells me Xin Zhang leads Akamai's congestion control team. Also I noticed Hiren used to work at Limelight, so may have contacts]// //// //Lars E: Co-organising a Dagstuhl retreat around DCTCP. Will forward list of invitees to Bob to notify once the ML exists.// //Also Lars's list of FreeBSD and Linux devs.// /// ** 18:40 What is the best way to ensure the outputs from a number of separate developers all converge in parallel to standardisation?** ***/Common Test Suite// //Interop//events ////Plugfests// ////Serving paths (e.g. Google's) capable of serving this/ ** 18:50 Next steps: Actions to set up suitable MLs, tools, with timesales etc.** *** /Discussed pros and cons of hosting ML on ietf.org.// //General agreement: use ietf.org for ML - because the IPR Note Well is useful. // // /Name for ML? / //Matt M: TCP Prague (for an evolving protocol, a meaningless tag is best).// //Karen N: ecn-prague, because it's not just TCP?// // //Agreed: //tcpprague@ietf.org//// // //Actions:// //Bob B: ML - ask SpencerD/MartinS, following the documented process// //Lars E: Set up wiki page - for assigning people to send out invitations// /// ** End 19:05** *** Notes: Bob Briscoe, helped by Andrew McGregor 28 Jul 2015 ============================================================================================= Below I've pasted some of the discussion that went on between a growing group of people before the mailing list was formed. On 21/07/15 13:44, Bob Briscoe wrote: > Folks, [Adding Jana & Pasi (as tcpm co-chair)] > > I have booked the Budapest room on Lobby Level in the IETF Hotel > (Hilton Prague). > Arrive 17:50 CET (*no longer 17:40*). > I have the room from 18:00, but let's all get there to start on time. > > It may be a squeeze so arrive early. There are now 29 on the list - I > hope I have captured everyone, including those who were added, but > crossing postings dropped them off again. > > *Scope*: Evolving the e2e DCTCP protocol for use alongside existing > traffic (whether in DCs, private nets or public Internet). > > This session is primarily to get DCTCP /developers/ involved early > (Windows, FreeBSD, Linux), so that whatever we decide to standardise > can be implemented in parallel. Doing implementation and > standardisation in series is not desirable, in whichever order. > > So, it is primarily an organisational meeting about creating a forum / > community to do this work, using people's experience to know what will > work best. > > *Not in Scope:** > ** Network changes are not in scope unless they impact the list of > changes needed to DCTCP that Koen and I posted in the original email. > * I am only aware of two solutions in the network that enable DCTCP > co-existence (i) the one Koen and I link to; (ii) the one Glenn Judd > used in Morgan Stanley (Diffserv segregation or capacity). If people > think there are others, please keep today's discussion focused on > whether any different changes to DCTCP will be needed than those > listed above. > * Please /do not/ describe in-network solutions in this Bar BoF. > > *Identifier**Out Of Scope for Today? > *Please continue to discuss whether and what identifier will be needed > for different traffic treatment /by email/, but I suggest we do /not/ > discuss this in the session. We know there are at least three possible > solutions, and one will have to be picked during standardisation - so > that is not priority #1 task today. > > *Proposed Agenda (for bashing)** > * > * 18:00 Introductions - name and interest > * 18:10 List of /must-have/ changes before deployment alongside > existing traffic. > * 18:30 Brainstorm to identify people not present that will be > important to this. > * 18:40 What is the best way to ensure the outputs from a number of > separate developers all converge in parallel to standardisation? > * 18:50 Next steps: Actions to set up suitable MLs, tools, with > timesales etc. > * End 19:00 > > *List of must-have DCTCP changes* we think will be needed: > > o fall back to Reno or Cubic behaviour on loss; > o negotiate its altered feedback semantics, which conveys the extent > of ECN marking, not just its existence, and this feedback needs to > be robust to loss [I-D.ietf-tcpm-accecn-reqs]; > o handle a window of less than 2 when the RTT is low, rather than > increase the queue [TCP-sub-mss-w]. > o average ECN feedback over its own RTT, not the hard-coded RTT > suitable only for data-centres, perhaps like Relentless > TCP [Mathis09]; > > o Use of a standardised packet identifier (if ECN-capable is not enough) > *BUT no discussion of which identifier*!!! > > oHeuristic testing for classic ECN bottlenecks (optional?) > > > > Bob > > On 21/07/15 12:40, Karen Elisabeth Egede Nielsen wrote: >> >> adding Michael >> >> *From:*Naeem Khademi [mailto:naeem.khademi@gmail.com >> ] >> *Sent:* Tuesday, July 21, 2015 1:22 PM >> *To:* De Schepper, Koen (Koen) >> *Cc:* Matt Mathis; Michael Welzl; gorry@erg.abdn.ac.uk >> ; michio@netapp.com >> ; kk@cs.ucr.edu ; >> Karen E. E. Nielsen; knneth@gmail.com ; Bob >> Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen >> Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred Baker; >> Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas Petlund; Anna >> Brunstrom; Per Hurtig; Tommy Pauly; David Schinazi >> *Subject:* Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague >> >> On Tue, Jul 21, 2015 at 1:00 PM, De Schepper, Koen (Koen) >> > > wrote: >> >> Hi Naeem, >> >> >> I'd be cautious to speak of "Scalable TCP" as a separate item from >> "Scalable ECN" >> >> >> as scalable TCP will *necessarily* need ECN (with instantaneous >> making and no >> >> >> averaging on the AQM side) to work. >> >> Well, this seems to be a common misunderstanding. A scalable TCP >> (like DCTCP) needs only >> >> a DON’T think twice ECN (which I call a scalable ECN). So if you mark >> with p and drop with p² it “works”. >> >> All other things you refer to are optimizations for having real low >> queuing latency as well (which are only possible with scalable TCP). >> >> To me getting a low latency translates into: low AQM thresholds + >> small TCP sawteeth + ECN (to avoid losses with small sawteeth) => >> can be achieved with DCTCP or a beta_{ecn} that is large enough. All >> other issues are only about backward compatibility with legacy TCP >> (fairness, convergence, etc if ever needed). >> >> Cheers, >> >> Naeem >> >> On a single queue with an ARED, curvyRED, PIE or CoDel controller >> it “works” (meaning it does not cause starvation, gives fairness, >> less wobbling flow throughput and throughput independent marking >> feedback). If unavoidable, even smoothing and delays are allowed, >> with some impact on latency of course, but it “works”. >> >> This is good news, as single queue routers just need 2 random >> values for drop (making it curvyRED) and one for marking. This >> might be a very simple upgrade for current hardware. 2 random >> variables can be “produced” simply, for example by remembering >> the previous one. >> >> >> is it about improving DCTCP for data center (which its >> original design was intended >> >> >> for)? or improving it for the use in the general Internet? >> >> That is a good question. I guess it depends on whether the >> “improvement” will also improve pure datacenter deployment (then >> it is a no-brainer), or otherwise if coexistence is required in >> the data center. >> >> Regards, >> >> Koen. >> >> *From:*Naeem Khademi [mailto:naeem.khademi@gmail.com >> ] >> *Sent:* dinsdag 21 juli 2015 12:35 >> *To:* De Schepper, Koen (Koen) >> *Cc:* Matt Mathis; Michael Welzl; gorry@erg.abdn.ac.uk >> ; michio@netapp.com >> ; kk@cs.ucr.edu ; >> Karen E. E. Nielsen; knneth@gmail.com ; >> Bob Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen >> Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred >> Baker; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas >> Petlund; Anna Brunstrom >> >> >> *Subject:* Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, >> Prague >> >> On Tue, Jul 21, 2015 at 12:16 PM, De Schepper, Koen (Koen) >> > > wrote: >> >> Hi All, >> >> The meeting is indeed to discuss improvements of DCTCP. Hopefully >> it does not get completely hijacked by the deployment story ;-). >> But it is an important topic to work on and to take into account >> for some of the other safety improvements. I see there is a lot >> of interest on this topic, so enough volunteers to work this ;-). >> >> Also for people who want to experience the power of using a >> scalable TCP and a scalable ECN (L4S) compared with that of the >> classic ones, the bitsNbytes demo is up and running on location. >> Give me a call or sms (+477550347) if you want to feel it. If >> people have an application on a client and server doing DCTCP, we >> could even do interop testing. We still have a few slots in our >> switches (only wired for now). >> >> Thanks Koen for the info. >> >> Just a minor side note: I'd be cautious to speak of "Scalable >> TCP" as a separate item from "Scalable ECN" as scalable TCP will >> *necessarily* need ECN (with instantaneous making and no >> averaging on the AQM side) to work. Apart than than DCTCP has >> been around for a while, and would have perhaps been a great >> thing if legacy traffic (loss-based or legacy ECN) had never >> existed in the first place and Internet had evolved in a way that >> we only had DCTCP today. Now that we know this is not the case, >> it would perhaps be of significant importance to discuss the >> deployment path and backward compatibility in great details IMHO. >> >> Another question about BarBof: is it about improving DCTCP for >> data center (which its original design was intended for)? or >> improving it for the use in the general Internet? from my >> understanding these are two separate issues and both should be >> addressed but better to be on the same page perhaps? >> >> Cheers, >> >> Naeem >> >> Regards, >> >> Koen. >> >> *From:*Matt Mathis [mailto:matt.mathis@gmail.com] >> *Sent:* dinsdag 21 juli 2015 8:51 >> *To:* Michael Welzl >> *Cc:* gorry@erg.abdn.ac.uk ; >> michio@netapp.com ; kk@cs.ucr.edu >> ; Karen E. E. Nielsen; knneth@gmail.com >> ; De Schepper, Koen (Koen); Bob >> Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen >> Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred >> Baker; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas >> Petlund; Anna Brunstrom; Naeem Khademi >> >> >> *Subject:* Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, >> 17:40, Prague >> >> I don't think it matters.... >> >> Given that TCP friendliness is but an illusion, I don't >> believe that differing ECN interpretations matter as much as >> people think. >> >> In the core any congestion causes egregious unfairness, no >> matter what the network and stacks try to do. Strive to >> eliminate congestion in the core. >> >> Near the network edges, there is an extremely powerful >> workaround that is already in play: have enough capacity and >> avoid uncontrolled background traffic when there is something >> important in the foreground. >> >> These are powerful enough to bridge most of any transition.... >> >> The danger does not come from deploying multiple versions of >> ECN at the edges, but deploying ECN in "substitute for loss" >> mode in the network. Network devices that independently mark >> and drop are not grossly incompatible with either edge >> behavior (although admittedly not optimal). >> >> Thanks, >> >> --MM-- >> Matt Mathis Home/Cell: 650-417-3029 >> ------------------------------------------- >> Evil is defined by mortals who think they know "The Truth" >> and use force to apply it to others. >> >> On Mon, Jul 20, 2015 at 11:42 PM, Michael Welzl >> > wrote: >> >> +1, and cc the people that have been newly added because they >> missed this thread. Read the bottom email first, then the one >> above, which is Koen's answer to me with Gorry's answer to >> that inline. >> >> Cheers, >> Michael >> >> >> >> > On 21. jul. 2015, at 08.29, gorry@erg.abdn.ac.uk >> wrote: >> > >> > Hi all, >> > >> > So I thought this was a Bar-BOF to discuss how to fix DCTP >> and take on a >> > road map to get this made robust. I would have been >> interested in this. >> > However, as I read on I now perceive the BOF as a trick to >> introduce your >> > own new idea for how to replace the operation of AQM >> within routers - >> > something quite different. >> > >> >> Hi Michael, >> >> >> >> According to me it means that there is still the >> possibility to hijack the >> >> Classic ECN bits and to use them for Scalable ECN. >> >> >> > I think this is still craziness. >> > >> >> It might be simpler >> >> today to contain Classic ECN in a controlled environment >> (with diffserv >> >> codepoint?) than future Scalable ECN. I suppose that if >> there is Classic >> >> ECN deployment today, it is already in a very controlled >> environment, and >> >> we hope that it will get deprecated soon as the advantages >> of scalable ECN >> >> and congestion control are huge compared to the Classic >> ones... >> >> >> > This we have talked about - my take this is that this a >> plain ridiculous >> > position. >> > >> >> I guess the three options are ignore Classic ECN, contain >> Scalable ECN, or >> >> contain Classic ECN. In the last 2 cases, we need to >> define that >> >> container. >> >> >> >> Regards, >> >> Koen. >> >> >> > Gorry >> > >> >>> -----Original Message----- >> >>> From: Michael Welzl [mailto:michawe@ifi.uio.no >> ] >> >>> Sent: maandag 20 juli 2015 23:58 >> >>> To: Bob Briscoe >> >>> Cc: Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen >> >>> Balasubramanian; Alex Zimmermann; Richard Scheffenegger; >> Fred Baker; >> >>> Matt Mathis; Andrew McGregor; Dave Taht; Stuart Cheshire; >> Andreas >> >>> Petlund; Gorry Fairhurst; Anna Brunstrom; De Schepper, >> Koen (Koen); >> >>> Naeem Khademi >> >>> Subject: Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, >> 17:40, Prague >> >>> >> >>> [ + Naeem Khademi, who I think is also interested ] >> >>> >> >>> >> >>> Hi, >> >>> >> >> >>> I’m reacting to: >> >>> >> >>>> PS. Below is some background, and some agenda ideas. Pls >> discuss, >> >>> bash and add your own. >> >>> >> >>> when I say: >> >>> >> >>> This is about something that’s not compatible at all >> with today’s >> >>> ECN. >> >>> That is, senders that use ECN and react to it like the >> spec so far says >> >>> will be beat to death by DCTCP with this scheme, unless >> we have a way >> >>> to classify this as being as something else, and queue >> this traffic >> >>> separately. Luckily, in AQM today, you said at the mic >> that you’re >> >>> considering usage of ECT(1) and probably a DSCP. >> >>> >> >>> But then I don’t understand this: >> >>> >> >>>>> We're trying to move fast because if we can get on top >> of other >> >>> developments (e.g. Apple's recent release of ECN), it >> will mean less >> >>> messy classification code in the AQM. >> >>> >> >>> You say “get on top of other developments†and you >> want to avoid >> >>> “less >> >>> messy classification code†. What exactly do you mean? >> >>> >> >>> Cheers, >> >>> Michael >> >> >> >> >> > >> > > -------- Forwarded Message -------- Subject: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague Date: Mon, 20 Jul 2015 22:46:14 +0100 From: Bob Briscoe To: Mirja Kuehlewind , EGGERT, Lars , Dave Thaler , Praveen Balasubramanian , Alex Zimmermann , Richard Scheffenegger , Fred Baker , Matt Mathis , Andrew McGregor , Dave Taht , Stuart Cheshire , Michael WELZL , Andreas Petlund , Gorry Fairhurst , Anna Brunstrom CC: De Schepper, Koen (Koen) Folks, DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague Location: Unless I have emailed with a room location before then, pls meet at the IETF reception. Koen & I are trying to get together people in Prague who are involved in development of DCTCP across platforms (Windows, Free BSD, Linux, etc), and who are interested in evolving it for use on heterogeneous networks, e.g. * data centres with a mix of TCP flavours, not just DCTCP * private networks * the public Internet Pls fwd this invite to anyone in Prague who ought to be involved that I've missed (pls cc everyone else too). Sorry for short notice. One purpose of the session will be to build a community beyond the IETF, so I'd like us to compose an email to a wider set of people after the session, e.g.: Stephen Bensley Glenn Judd Daniel Borkmann Florian Westphal 邓灵莉/Lingli Deng Mohammad Alizadeh Stephen Hemminger David S. Miller Sridharan, Murari > Yuchung Cheng > Koen & Bob PS. Below is some background, and some agenda ideas. Pls discuss, bash and add your own. > We've recently developed an AQM that allows DCTCP to co-exist with > Cubic/Reno etc. with zero config. Links below. > > We have to add some necessary mechanisms to DCTCP (listed below) so it > will be safe alongside other traffic. Two questions: > > Q1. We don't want to fork DCTCP. Does anyone think a fork optimised > for homogeneous DCTCP would be better? > > Q2. Anyone interested in helping? > We have an idea how to do each one, but sharing the load would be > great - there's Linux, FreeBSD, Windows, etc. to cover. > > List of the 4 essential 'safety' mods to DCTCP (copied from the IETF > Internet Draft linked below) and one that might need to be essential: > o fall back to Reno or Cubic behaviour on loss; > > o negotiate its altered feedback semantics, which conveys the extent > of ECN marking, not just its existence, and this feedback needs to > be robust to loss [I-D.ietf-tcpm-accecn-reqs]; > > o handle a window of less than 2 when the RTT is low, rather than > increase the queue [TCP-sub-mss-w]. > > o average ECN feedback over its own RTT, not the hard-coded RTT > suitable only for data-centres, perhaps like Relentless > TCP [Mathis09]; > > > o Use of a standardised packet identifier (if ECN-capable is not enough) > > > oHeuristic testing for classic ECN bottlenecks (optional?) > > > > > We're trying to move fast because if we can get on top of other > developments (e.g. Apple's recent release of ECN), it will mean less > messy classification code in the AQM. > So the links below are not on official sites yet. > > ‘Data Centre to the Home’: Ultra-Low Latency for All > > > Highlights: > * 1ms 99%-ile queuing delay for all DCTCP traffic in thousands of > expts incl. high load, > over an e2e test network with real broadband equipment. > * DCTCP co-existence with Reno & Cubic, with no transport ID inspection. > * less ops per packet than RED > * Zero config > > IETF Draft to standardise those parts of the AQM relevant to > interop(not yet submitted to IETF): > > > Koen & Bob --------------050605040209060201060105 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Folks,

These notes have taken a week, because I've only just put my machine back together after having to rebuild the hardware a little :|


Notes: DCTCP Evolution Bar BoF
6-7pm Tue 21 Jul 2015, Budapest room, The Hilton, Prague, CZ

Summary of Actions:
Lars E: Set up tcpprague wiki page
Bob B: Request tcpprague@ietf.org mailing list, via IETF process (requires Area Director approval)
Bob B: Document Rationale - initiate a para on wiki.
Lars E: fwd Dagstuhl invitee list to Bob
Bob B: Set up list on wiki to assign people to invite those not in the room to join.

* 18:00 Introductions - name and interest
Present:
Marcelo    Bagnulo Braun    <marcelo@it.uc3m.es>
Praveen    Balasubramanian    <pravb@microsoft.com>
Martin    Bekker    <martin.becke@haw-hamburg.de>
Bob    Briscoe    <ietf@bobbriscoe.net>
Anna    Brunstrom    <anna.brunstrom@kau.se>
Stuart    Cheshire    <cheshire@apple.com>
Koen    De Schepper    <koen.de_schepper@alcatel-lucent.com>
Fabien    Duchen    <fabien.duchene@uclouvain.be>
Phil    Eardley    <philip.eardley@bt.com>
Lars    Eggert    <lars@netapp.com>
Michio    Honda    <michio@netapp.com>
Per    Hurtig    <Per.Hurtig@kau.se>
Jana    Iyengar    <jri@google.com>
Naeem    Khademi    <naeem.khademi@gmail.com>
Mirja    Kuehlewind    <mirja.kuehlewind@tik.ee.ethz.ch>
Matt    Mathis    <mattmathis@google.com>
Andrew    McGregor    <andrewmcgr@google.com>
Karen    Nielsen    <karen.nielsen@tieto.com>
Tommy    Pauly    <tpauly@apple.com>
Andreas Petlund <apetlund@ifi.uio.no>
Costin    Raiciu    <costin.raiciu@cs.pub.ro>
Pasi    Sarolahti    <pasi.sarolahti@iki.fi>
Richard    Scheffenegger    <rs@netapp.com>
David    Schinazi    <dschinazi@apple.com>
Randall    Stewart    <randall@lakerest.net>
Dave    Thaler    <dthaler@microsoft.com>
Brian    Trammell   <ietf@trammell.ch>
Michael    Tuexen    <Michael.Tuexen@lurchi.franken.de>
Felix    Weinrank    <weinrank@fh-munster.de>
Michael    Welzl    <michawe@ifi.uio.no>
Alex    Zimmermann    <alexander.zimmermann@netapp.com>

* Scope and Agenda Bashing
[Non-italic text is from the materials pre-prepared by Koen De Schepper and Bob Briscoe.
Italic text summarises conversation in the room.]

Meeting is covered by the standard IETF "Note Well" concerning intellectual property.

Scope:
* Evolving the e2e DCTCP protocol for use alongside existing traffic (whether in DCs, private nets or public Internet).
* Primarily to get DCTCP /developers/ involved early (Windows, FreeBSD, Linux), so that whatever we decide to standardise can be implemented in parallel
  (Doing implementation and standardisation in series is not desirable, in whichever order).
* Primarily an organisational meeting about creating a forum / community to do this work, using people's experience to know what will work best.

Not in Scope:
* Network changes are not in scope unless they impact the list of changes needed to DCTCP
* The in-network side of the solution (two approaches exist [DCttH, Judd15], others may follow).
* Identifier of DCTCP-like traffic (please discuss by email, not in this meeting)

Lars E: Informational draft recording Microsoft's DCTCP should not be stalled by this, as it has value of its own.
   Unanimous agreement.

Praveen S: Microsoft has offered a royalty free license for DCTCP IPR.

Karen N: Is DCTCP over a non-TCP transport (e.g. SCTP) in scope?
   Unanimous "Yes"

Outcome of discussion on the features of this DCTCP-like congestion control that define this work:
  1. Must use ECN, but unlike RFC3168 ECN, marking is not merely equivalent to drop,
    so ECN signals can be more plentiful and sooner than drop.
  2. Packet rate is proportional to 1/p, where p is the ECN marking probability.
Matt M: 1/p makes congestion control scale with the bandwidth, by making the intensity of congestion control signals per RTT invariant.

Stuart Ch: Apple is turning on ECN by default in clients. Currently in developer seeds but probably in the next releases.  Packet loss is also not a mystery.

* 18:15 List of /must-have/ changes before deployment alongside existing traffic.
Matt M: Rather than a "MUST-have" list, produce a prioritised list, because where to draw the necessity line could depend on the use-case.

The following list wasn't formally prioritised in the meeting, but items where some people questioned necessity are shifted down.
  1. Fall back to Reno or Cubic behaviour on loss;
    For how long? quick consensus: 1 RTT, but needs further discussion. ECN response continues to operate in parallel.
  2. Negotiate altered feedback semantics, to convey the extent of ECN marking, not just its existence, and this feedback needs to be robust to loss [RFC-to-be 7560];
    Mirja K, Richard S & Bob B plan to have spec of much simpler solution out soon.
  3. Use of a standardised packet identifier (if ECN-capable is not enough)
    Identifier tbd.
    - - - 8< - - - - - - - - highest line between "must-have for safety" and "would be nice for performance" - - - - - -  8< - - - -
  4. Handle a window of less than 2 when the RTT is low, rather than increase the queue [TCP-sub-mss-cwnd], like TCP Nice.
    Michael W: Is this "must-have"? Quite a complicated step.
    Bob B: Yes, but, otherwise DCTCP will pollute ultra-low latency queues from the start.
  5. Average ECN feedback over its own RTT, not the hard-coded RTT suitable only for data-centres, perhaps reduce cwnd by seg-size/2 per ECN Echo, like Relentless TCP [Mathis09];
    ???: How bad would long-RTT flows be?  More generally, how can we evaluate all this?
    Bob B: With mixed RTTs, flows with RTT > a couple of ms will respond too quickly to bursts. Whatever, it's already been implemented by Mohammad Alizadeh in Linux, and evaluated, so this is easy.
  6. Heuristic testing for classic ECN bottlenecks
    The idea would be to detect a 'classic' RFC316 bottleneck by whether appreciable delay growth accompanies the marking (originally suggested by Michael W).
    Bob B: Complex and slow to detect, so it would have to learn and cache for new flows - suggest this should only be a must-have if measurements prove it to be a problem - i.e. if a significant proportion of classic ECN bottlenecks     exist
    Matt M: No need for this - rate mismatch no worse than TCP already sees with RTT discrepancies.
     - - - 8< - - - - - - - - lowest line between "must-have for safety" and "would be nice for performance" - - - - - -  8< - - - -
  7. Costin R: Faster-than-additive increase (similar to Cubic)
    A performance improvement, not "must-have", but would be nice to have while we're doing this.
  8. [Not discussed in the meeting, but added by Bob B for the record]: Less drastic exit from slow-start, to match less drastic rate reduction per mark.
    Currently, because marking threshold is shallow, slow start exits earlier than with drop, unnecessarily increasing completion time.

Costin R: Any other way to evolve towards DCTCP over mixed networks, without separate queues in the network?
 Bob B: To discuss on ML, and if we continue with the proposed approach, we must record the rationale on the WIki.
* 18:30 Brainstorm to identify people not present who will be important to this.
Mohammad    Alizadeh    <alizadeh.mr@gmail.com>
Grenville    Armitage    <garmitage@swin.edu.au>
Fred    Baker    <fred@cisco.com>
Stephen    Bensley    <sbens@microsoft.com>
Daniel    Borkmann    <daniel.borkmann@alumni.ethz.ch>
Yuchung    Cheng    <ycheng@google.com>
Kenjiro    Cho    <kjc@iijlab.net>
邓灵莉/Lingli    Deng    <denglingli@chinamobile.com>
Eric    Dumazet    <edumazet@gmail.com>
Gorry    Fairhurst    <gorry@erg.abdn.ac.uk>
Jamal    Hadi Salim    <hadi@mojatatu.com>
Glenn    Judd    <glenn.judd@morganstanley.com>
Midori    Kato    <katoon@sfc.wide.ad.jp>
Kenneth    Klette Jonassen    <kennetkl@ifi.uio.no>    (already subscribed)
Sridharan,    Murari    <muraris@microsoft.com>
Hiren    Panchasara    <hiren.panchasara@gmail.com>
Hagen     Pfeifer    <hagen@jauu.net>
Balaji    Prabhakar    <balaji@ee.stanford.edu>
KK    Ramakrishnan    <kk@cs.ucr.edu>
Lawrence    Stewart    <lstewart@netflix.com>
Dave    Taht    <dave.taht@gmail.com>
Florian    Westphal    <fw@strlen.de>

Agreed to cc to the following for awareness, but no need to invite to join the list:
Stephen    Hemminger    <stephen@networkplumber.org>
David    Miller    <davem@davemloft.net>

Missing types of organisations:
  • Network operators (not so relevant for e2e protocol, but need to be motivated to deploy the network part)
  • CDNs
[Bob B adds: Subsequent to mtg, Erik Nygren tells me Xin Zhang leads Akamai's congestion control team. Also I noticed Hiren used to work at Limelight, so may have contacts]
Lars E: Co-organising a Dagstuhl retreat around DCTCP. Will forward list of invitees to Bob to notify once the ML exists.
Also Lars's list of FreeBSD and Linux devs.
* 18:40 What is the best way to ensure the outputs from a number of separate developers all converge in parallel to standardisation?
 Common Test Suite
 Interop events
 Plugfests
 Serving paths (e.g. Google's) capable of serving this

* 18:50 Next steps: Actions to set up suitable MLs, tools, with timesales etc.
Discussed pros and cons of hosting ML on ietf.org.
 General agreement: use ietf.org for ML - because the IPR Note Well is useful.

Name for ML?
Matt M: TCP Prague (for an evolving protocol, a meaningless tag is best).
Karen N: ecn-prague, because it's not just TCP?

Agreed: tcpprague@ietf.org

Actions:
 Bob B: ML - ask SpencerD/MartinS, following the documented process
 Lars E: Set up wiki page - for assigning people to send out invitations
* End 19:05

Notes: Bob Briscoe, helped by Andrew McGregor
28 Jul 2015


=============================================================================================
Below I've pasted some of the discussion that went on between a growing group of people before the mailing list was formed.

On 21/07/15 13:44, Bob Briscoe wrote:
Folks, [Adding Jana & Pasi (as tcpm co-chair)]

I have booked the Budapest room on Lobby Level in the IETF Hotel (Hilton Prague).
Arrive 17:50 CET (no longer 17:40).
I have the room from 18:00, but let's all get there to start on time.

It may be a squeeze so arrive early. There are now 29 on the list - I hope I have captured everyone, including those who were added, but crossing postings dropped them off again.

Scope: Evolving the e2e DCTCP protocol for use alongside existing traffic (whether in DCs, private nets or public Internet).

This session is primarily to get DCTCP /developers/ involved early (Windows, FreeBSD, Linux), so that whatever we decide to standardise can be implemented in parallel. Doing implementation and standardisation in series is not desirable, in whichever order.

So, it is primarily an organisational meeting about creating a forum / community to do this work, using people's experience to know what will work best.

Not in Scope:
* Network changes are not in scope unless they impact the list of changes needed to DCTCP that Koen and I posted in the original email.
* I am only aware of two solutions in the network that enable DCTCP co-existence (i) the one Koen and I link to; (ii) the one Glenn Judd used in Morgan Stanley (Diffserv segregation or capacity). If people think there are others, please keep today's discussion focused on whether any different changes to DCTCP will be needed than those listed above.
* Please /do not/ describe in-network solutions in this Bar BoF.

Identifier Out Of Scope for Today?
Please continue to discuss whether and what identifier will be needed for different traffic treatment /by email/, but I suggest we do /not/ discuss this in the session. We know there are at least three possible solutions, and one will have to be picked during standardisation - so that is not priority #1 task today.

Proposed Agenda (for bashing)

* 18:00 Introductions - name and interest
* 18:10 List of /must-have/ changes before deployment alongside existing traffic.
* 18:30 Brainstorm to identify people not present that will be important to this.
* 18:40 What is the best way to ensure the outputs from a number of separate developers all converge in parallel to standardisation?
* 18:50 Next steps: Actions to set up suitable MLs, tools, with timesales etc.
* End 19:00

List of must-have DCTCP changes we think will be needed:

   o  fall back to Reno or Cubic behaviour on loss;
   o  negotiate its altered feedback semantics, which conveys the extent
      of ECN marking, not just its existence, and this feedback needs to
      be robust to loss [I-D.ietf-tcpm-accecn-reqs];
   o  handle a window of less than 2 when the RTT is low, rather than
      increase the queue [TCP-sub-mss-w].
   o  average ECN feedback over its own RTT, not the hard-coded RTT
      suitable only for data-centres, perhaps like Relentless
      TCP [Mathis09];

   o  Use of a standardised packet identifier (if ECN-capable is not enough)
 BUT no discussion of which identifier!!!

   o  Heuristic testing for classic ECN bottlenecks (optional?)



Bob

On 21/07/15 12:40, Karen Elisabeth Egede Nielsen wrote:

adding Michael

 

From: Naeem Khademi [mailto:naeem.khademi@gmail.com]
Sent: Tuesday, July 21, 2015 1:22 PM
To: De Schepper, Koen (Koen)
Cc: Matt Mathis; Michael Welzl; gorry@erg.abdn.ac.uk; michio@netapp.com; kk@cs.ucr.edu; Karen E. E. Nielsen; knneth@gmail.com; Bob Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred Baker; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas Petlund; Anna Brunstrom; Per Hurtig; Tommy Pauly; David Schinazi
Subject: Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague

 

 

 

On Tue, Jul 21, 2015 at 1:00 PM, De Schepper, Koen (Koen) <koen.de_schepper@alcatel-lucent.com> wrote:

Hi Naeem,

 

>> I'd be cautious to speak of "Scalable TCP" as a separate item from "Scalable ECN"

>> as scalable TCP will *necessarily* need ECN (with instantaneous making and no

>> averaging on the AQM side) to work.

Well, this seems to be a common misunderstanding. A scalable TCP (like DCTCP) needs only

a DON’T think twice ECN (which I call a scalable ECN). So if you mark with p and drop with p² it “works”.

All other things you refer to are optimizations for having real low queuing latency as well (which are only possible with scalable TCP).

 

 

To me getting a low latency translates into: low AQM thresholds + small TCP sawteeth + ECN  (to avoid losses with small sawteeth) => can be achieved with DCTCP or a beta_{ecn} that is large enough. All other issues are only about backward compatibility with legacy TCP (fairness, convergence, etc if ever needed). 

 

Cheers,

Naeem   

 

 

On a single queue with an ARED, curvyRED, PIE or CoDel controller it “works” (meaning it does not cause starvation, gives fairness, less wobbling flow throughput and throughput independent marking feedback). If unavoidable, even smoothing and delays are allowed, with some impact on latency of course, but it “works”.

 

This is good news, as single queue routers just need 2 random values for drop (making it curvyRED) and one for marking. This might be a very simple upgrade for current hardware. 2 random variables can be “produced” simply, for example by remembering the previous one.

 

>> is it about improving DCTCP for data center (which its original design was intended

>> for)? or improving it for the use in the general Internet?

That is a good question. I guess it depends on whether the “improvement” will also improve pure datacenter deployment (then it is a no-brainer), or otherwise if coexistence is required in the data center.

 

Regards,

Koen.

 

 

From: Naeem Khademi [mailto:naeem.khademi@gmail.com]
Sent: dinsdag 21 juli 2015 12:35
To: De Schepper, Koen (Koen)
Cc: Matt Mathis; Michael Welzl; gorry@erg.abdn.ac.uk; michio@netapp.com; kk@cs.ucr.edu; Karen E. E. Nielsen; knneth@gmail.com; Bob Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred Baker; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas Petlund; Anna Brunstrom


Subject: Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague

 

 

 

On Tue, Jul 21, 2015 at 12:16 PM, De Schepper, Koen (Koen) <koen.de_schepper@alcatel-lucent.com> wrote:

Hi All,

 

The meeting is indeed to discuss improvements of DCTCP. Hopefully it does not get completely hijacked by the deployment story ;-). But it is an important topic to work on and to take into account for some of the other safety improvements. I see there is a lot of interest on this topic, so enough volunteers to work this ;-).

 

Also for people who want to experience the power of using a scalable TCP and a scalable ECN (L4S) compared with that of the classic ones, the bitsNbytes demo is up and running on location. Give me a call or sms (+477550347) if you want to feel it. If people have an application on a client and server doing DCTCP, we could even do interop testing. We still have a few slots in our switches (only wired for now).

 

Thanks Koen for the info. 

 

Just a minor side note: I'd be cautious to speak of "Scalable TCP" as a separate item from "Scalable ECN" as scalable TCP will *necessarily* need ECN (with instantaneous making and no averaging on the AQM side) to work. Apart than than DCTCP has been around for a while, and would have perhaps been a great thing if legacy traffic (loss-based or legacy ECN) had never existed in the first place and Internet had evolved in a way that we only had DCTCP today. Now that we know this is not the case, it would perhaps be of significant importance to discuss the deployment path and backward compatibility in great details IMHO. 

 

Another question about BarBof: is it about improving DCTCP for data center (which its original design was intended for)? or improving it for the use in the general Internet? from my understanding these are two separate issues and both should be addressed but better to be on the same page perhaps? 

 

Cheers,

Naeem

 

 

 

Regards,

Koen.

 

 

 

 

From: Matt Mathis [mailto:matt.mathis@gmail.com]
Sent: dinsdag 21 juli 2015 8:51
To: Michael Welzl
Cc: gorry@erg.abdn.ac.uk; michio@netapp.com; kk@cs.ucr.edu; Karen E. E. Nielsen; knneth@gmail.com; De Schepper, Koen (Koen); Bob Briscoe; Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred Baker; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas Petlund; Anna Brunstrom; Naeem Khademi


Subject: Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague

 

I don't think it matters....

 

Given that TCP friendliness is but an illusion, I don't believe that differing ECN interpretations matter as much as people think.

 

In the core any congestion causes egregious unfairness, no matter what the network and stacks try to do.  Strive to eliminate congestion in the core.

 

Near the network edges, there is an extremely powerful workaround that is already in play: have enough capacity and avoid uncontrolled background traffic when there is something important in the foreground.

 

These are powerful enough to bridge most of any transition....

 

The danger does not come from deploying multiple versions of ECN at the edges, but deploying ECN in "substitute for loss" mode in the network.    Network devices that independently mark and drop are not grossly incompatible with either edge behavior (although admittedly not optimal).

 

Thanks,

--MM--
Matt Mathis    Home/Cell: 650-417-3029
-------------------------------------------
Evil is defined by mortals who think they know "The Truth" and use force to apply it to others.

 

On Mon, Jul 20, 2015 at 11:42 PM, Michael Welzl <michawe@ifi.uio.no> wrote:

+1, and cc the people that have been newly added because they missed this thread. Read the bottom email first, then the one above, which is Koen's answer to me with Gorry's answer to that inline.

Cheers,
Michael



> On 21. jul. 2015, at 08.29, gorry@erg.abdn.ac.uk wrote:
>
> Hi all,
>
> So  I thought this was a Bar-BOF to discuss how to fix DCTP and take on a
> road map to get this made robust. I would have been interested in this.
> However, as I read on I now perceive the BOF as a trick to introduce your
> own new idea for how to replace the operation of AQM  within routers -
> something quite different.
>
>> Hi Michael,
>>
>> According to me it means that there is still the possibility to hijack the
>> Classic ECN bits and to use them for Scalable ECN.
>>
> I think this is still craziness.
>
>> It might be simpler
>> today to contain Classic ECN in a controlled environment (with diffserv
>> codepoint?) than future Scalable ECN. I suppose that if there is Classic
>> ECN deployment today, it is already in a very controlled environment, and
>> we hope that it will get deprecated soon as the advantages of scalable ECN
>> and congestion control are huge compared to the Classic ones...
>>
> This we have talked about - my take this is that this a plain ridiculous
> position.
>
>> I guess the three options are ignore Classic ECN, contain Scalable ECN, or
>> contain Classic ECN. In the last 2 cases, we need to define that
>> container.
>>
>> Regards,
>> Koen.
>>
> Gorry
>
>>> -----Original Message-----
>>> From: Michael Welzl [mailto:michawe@ifi.uio.no]
>>> Sent: maandag 20 juli 2015 23:58
>>> To: Bob Briscoe
>>> Cc: Mirja Kuehlewind; EGGERT, Lars; Dave Thaler; Praveen
>>> Balasubramanian; Alex Zimmermann; Richard Scheffenegger; Fred Baker;
>>> Matt Mathis; Andrew McGregor; Dave Taht; Stuart Cheshire; Andreas
>>> Petlund; Gorry Fairhurst; Anna Brunstrom; De Schepper, Koen (Koen);
>>> Naeem Khademi
>>> Subject: Re: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
>>>
>>> [ + Naeem Khademi, who I think is also interested ]
>>>
>>>
>>> Hi,
>>>

>>> I’m reacting to:
>>>
>>>> PS. Below is some background, and some agenda ideas. Pls discuss,
>>> bash and add your own.
>>>
>>> when I say:
>>>
>>> This is about something that’s not compatible at all with today’s
>>> ECN.
>>> That is, senders that use ECN and react to it like the spec so far says
>>> will be beat to death by DCTCP with this scheme, unless we have a way
>>> to classify this as being as something else, and queue this traffic
>>> separately. Luckily, in AQM today, you said at the mic that you’re
>>> considering usage of ECT(1) and probably a DSCP.
>>>
>>> But then I don’t understand this:
>>>
>>>>> We're trying to move fast because if we can get on top of other
>>> developments (e.g. Apple's recent release of ECN), it will mean less
>>> messy classification code in the AQM.
>>>
>>> You say “get on top of other developments† and you want to avoid
>>> “less
>>> messy classification code†. What exactly do you mean?
>>>
>>> Cheers,
>>> Michael
>>
>>
>

 

 

 




-------- Forwarded Message --------
Subject: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Date: Mon, 20 Jul 2015 22:46:14 +0100
From: Bob Briscoe <ietf@bobbriscoe.net>
To: Mirja Kuehlewind <mirja.kuehlewind@tik.ee.ethz.ch>, EGGERT, Lars <lars@netapp.com>, Dave Thaler <dthaler@microsoft.com>, Praveen Balasubramanian <pravb@microsoft.com>, Alex Zimmermann <alexander.zimmermann@netapp.com>, Richard Scheffenegger <rs@netapp.com>, Fred Baker <fred@cisco.com>, Matt Mathis <matt.mathis@gmail.com>, Andrew McGregor <andrewmcgr@google.com>, Dave Taht <dave.taht@gmail.com>, Stuart Cheshire <cheshire@apple.com>, Michael WELZL <michawe@ifi.uio.no>, Andreas Petlund <andreas@petlund.no>, Gorry Fairhurst <gorry@erg.abdn.ac.uk>, Anna Brunstrom <anna.brunstrom@kau.se>
CC: De Schepper, Koen (Koen) <koen.de_schepper@alcatel-lucent.com>


Folks,

DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Location: Unless I have emailed with a room location before then, pls meet at the IETF reception.

Koen & I are trying to get together people in Prague who are involved in development of DCTCP across platforms (Windows, Free BSD, Linux, etc), and who are interested in evolving it for use on heterogeneous networks, e.g.
* data centres with a mix of TCP flavours, not just DCTCP
* private networks
* the public Internet

Pls fwd this invite to anyone in Prague who ought to be involved that I've missed (pls cc everyone else too).

Sorry for short notice.

One purpose of the session will be to build a community beyond the IETF, so I'd like us to compose an email to a wider set of people after the session, e.g.:

Stephen Bensley <sbens@microsoft.com>
Glenn Judd <glenn.judd@morganstanley.com>
Daniel Borkmann <daniel.borkmann@alumni.ethz.ch>
Florian Westphal <fw@strlen.de>
邓灵莉/Lingli Deng <denglingli@chinamobile.com>
Mohammad Alizadeh <alizadeh.mr@gmail.com>
Stephen Hemminger <stephen@networkplumber.org>
David S. Miller <davem@davemloft.net>
Sridharan, Murari <muraris@microsoft.com>
Yuchung Cheng <ycheng@google.com>


Koen & Bob

PS. Below is some background, and some agenda ideas. Pls discuss, bash and add your own.


We've recently developed an AQM that allows DCTCP to co-exist with Cubic/Reno etc. with zero config. Links below.

We have to add some necessary mechanisms to DCTCP (listed below) so it will be safe alongside other traffic. Two questions:

Q1. We don't want to fork DCTCP. Does anyone think a fork optimised for homogeneous DCTCP would be better?

Q2. Anyone interested in helping?
We have an idea how to do each one, but sharing the load would be great - there's Linux, FreeBSD, Windows, etc. to cover.

List of the 4 essential 'safety' mods to DCTCP (copied from the IETF Internet Draft linked below) and one that might need to be essential: 
   o  fall back to Reno or Cubic behaviour on loss;
 
   o  negotiate its altered feedback semantics, which conveys the extent
      of ECN marking, not just its existence, and this feedback needs to
      be robust to loss [I-D.ietf-tcpm-accecn-reqs];
 
   o  handle a window of less than 2 when the RTT is low, rather than
      increase the queue [TCP-sub-mss-w].
 
   o  average ECN feedback over its own RTT, not the hard-coded RTT
      suitable only for data-centres, perhaps like Relentless
      TCP [Mathis09];


   o  Use of a standardised packet identifier (if ECN-capable is not enough)


   o  Heuristic testing for classic ECN bottlenecks (optional?)

We're trying to move fast because if we can get on top of other developments (e.g. Apple's recent release of ECN), it will mean less messy classification code in the AQM.
So the links below are not on official sites yet.

‘Data Centre to the Home’: Ultra-Low Latency for All
<http://www.bobbriscoe.net/projects/latency/dctth_preprint.pdf>

Highlights:
* 1ms 99%-ile queuing delay for all DCTCP traffic in thousands of expts incl. high load,
   over an e2e test network with real broadband equipment.
* DCTCP co-existence with Reno & Cubic, with no transport ID inspection.
* less ops per packet than RED
* Zero config

IETF Draft to standardise those parts of the AQM relevant to interop (not yet submitted to IETF):
<http://www.bobbriscoe.net/projects/latency/draft-briscoe-aqm-dualq-coupled-00.txt>



Koen & Bob --------------050605040209060201060105-- From nobody Tue Jul 28 07:37:24 2015 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6BC4F1AC3CA for ; Tue, 28 Jul 2015 07:37:22 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.6 X-Spam-Level: X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lfldgc52hNW7 for ; Tue, 28 Jul 2015 07:37:20 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3239D1A8A7A for ; Tue, 28 Jul 2015 07:37:20 -0700 (PDT) Received: from 83.58.199.146.dyn.plus.net ([146.199.58.83]:37832 helo=[192.168.0.11]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1ZK60M-0007vL-I7 for tcpPrague@ietf.org; Tue, 28 Jul 2015 15:37:18 +0100 Message-ID: <55B7939D.9070308@bobbriscoe.net> Date: Tue, 28 Jul 2015 15:37:17 +0100 From: Bob Briscoe User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 To: TCP Prague IETF List References: <55B77CFE.9090505@bobbriscoe.net> In-Reply-To: <55B77CFE.9090505@bobbriscoe.net> X-Forwarded-Message-Id: <55B77CFE.9090505@bobbriscoe.net> Content-Type: multipart/alternative; boundary="------------030704040503060800090604" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net Archived-At: Subject: [tcpPrague] Fwd: Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jul 2015 14:37:22 -0000 This is a multi-part message in MIME format. --------------030704040503060800090604 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Folks, To those at the Bar Bof, I meant to ask if you can check thru the notes and point out any mistakes or gaps: Cheers Bob -------- Forwarded Message -------- Subject: Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague Date: Tue, 28 Jul 2015 14:00:46 +0100 From: Bob Briscoe To: TCP Prague IETF List [snip] --------------030704040503060800090604 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit Folks,

To those at the Bar Bof, I meant to ask if you can check thru the notes and point out any mistakes or gaps:
<https://mailarchive.ietf.org/arch/msg/tcpprague/mwWncQg3egPd15FItYWiEvRDrvA>

Cheers


Bob


-------- Forwarded Message --------
Subject: Notes: DCTCP evolution 'bar BoF': Tue 21 Jul 2015, 17:40, Prague
Date: Tue, 28 Jul 2015 14:00:46 +0100
From: Bob Briscoe <ietf@bobbriscoe.net>
To: TCP Prague IETF List <tcpPrague@ietf.org>

[snip]



--------------030704040503060800090604-- From nobody Thu Jul 30 05:22:28 2015 Return-Path: X-Original-To: tcpprague@ietfa.amsl.com Delivered-To: tcpprague@ietfa.amsl.com Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 235A21A1AD9 for ; Thu, 30 Jul 2015 04:53:03 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: 0.1 X-Spam-Level: X-Spam-Status: No, score=0.1 tagged_above=-999 required=5 tests=[BAYES_50=0.8, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EHZGjjojc_hN for ; Thu, 30 Jul 2015 04:53:00 -0700 (PDT) Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EDA891A1B3E for ; Thu, 30 Jul 2015 04:52:59 -0700 (PDT) Received: from [81.174.189.118] (port=39155 helo=[192.168.0.11]) by server.dnsblock1.com with esmtpsa (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1ZKmOQ-0006n3-AC; Thu, 30 Jul 2015 12:52:58 +0100 Message-ID: <55BA1019.7030600@bobbriscoe.net> Date: Thu, 30 Jul 2015 12:52:57 +0100 From: Bob Briscoe User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 To: Costin RAICIU , TCP Prague IETF List Content-Type: multipart/alternative; boundary="------------080900060301060703060509" X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - ietf.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net Archived-At: X-Mailman-Approved-At: Thu, 30 Jul 2015 05:22:27 -0700 Subject: [tcpPrague] Is there another way to do this? X-BeenThere: tcpprague@ietf.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jul 2015 11:53:03 -0000 This is a multi-part message in MIME format. --------------080900060301060703060509 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Costin & all on the tcpprague list, You asked "Is there another way to do this?" in the bar BoF in Prague. I am taking "this" to mean the features of the proposed new service (L4S),. Then the questions becomes, "Are all the elements needed at once?": * Low Loss, * Low Latency * Scalable throughput. [For those on the list who were not in Prague, a way to achieve L4S is described in this pre-print: "Data Centre to the Home "] There are a number of elements to "this", so I'll take them in vaguely decreasing order of obviousness. Then everyone can identify any unnecessary assumptions. A. ECN-capable (sndr, rcvr, aqm)? B. Shift ECN smoothing from AQM to source (aqm, sndr)? C. Second queue (aqm)? D. Scalable congestion control (sndr)? E. Accurate ECN feedback (sndr, rcvr)? F. Finally, a more intangible question: significantly better service vs incrementally better? The further features we discussed in the Bar BoF are secondary to all these questions. Taking each in turn... *A. Is ECN essential? * (sndr, rcvr, aqm) I think we're all agreed that ECN is essential. Without ECN: #1. as load increases (i.e. as each TCP flow's rate has to decrease), an AQM has to introduce excessive drop to keep queuing delay low. (If you weren't in the AQM wg, see "questioning a fixed delay target " or the tech report that backs it up). #2. Throughput scaling requires roughly 1 congestion signal per RTT at high flow rates, at least within an order of magnitude. As load increases, segments per RTT (the window) has to reduce, so the drop ratio has to increase. With low RTT (assuming we have achieved v low queuing delay) the window is also smaller. See also {Note 1} *B. Is shifting smoothing of ECN signals from the AQM to the source essential?* (aqm, sndr) There's no reason to smooth ECN in the AQM, and little harm from not smoothing it. A newly implemented source can smooth incoming ECN feedback itself. If a classic TCP-ECN source does not smooth ECN feedback, it's not a killer, because it will ignore >1 feedback per RTT anyway. If a source smooths ECN that has already been smoothed in the AQM, its response could be sluggish, but we can make this unlikely as follows: Smoothing (burst allowance) should be turned off by default for ECN in all AQMs (RED, PIE, CoDel, etc) now. *C. Is a second queue essential?* (aqm) With one queue, 'new' traffic shares the latency of the queue with 'classic' Internet traffic. So to achieve low latency with one queue, the AQM threshold for all traffic has to be shallow. To achieve really low latency, this means classic traffic would severely underutilise capacity. Colleagues in the RITE project tried shifting the AQM threshold down only if there was no ECN traffic, but they couldn't get it to work. Is full utilisation important? Well, probably not as important as it used to be. But network operators (ISPs or DC operators) are not going to deploy an AQM that prevents existing traffic using a large proportion of the link capacity. *D. **Is scalable **congestion control essential?** *(sndr) As under question A, throughput scaling requires roughly 1 congestion signal per RTT at high flow rates, at least within an order of magnitude. I.e. if the congestion control algo gives packet rate proportional to 1/p^b, where p is drop probability, then scalable means b=1 For now, we can probably make do with b<1 as long as b~=1, e.g. b=0.8 or something similar. However, we will need b=1 in the future. So it all depends whether we can map out a route to increase to b=1 later, bearing in mind this traffic will still have to co-exist with b=0.5 (Reno) as well as b=0.75 (Cubic). *E. Is Accurate ECN feedback essential?* (sndr, rcvr) Without AccECN, each downward rate response to one ECN feedback has to be greater, because it has to be enough for a whole round trip. This makes the sawteeth bigger, which creates the dilemma between higher delay and lower utilisation. This is primarily important when numerous flows are arriving and departing but the total number is not huge (i.e. typical conditions on an access link, or in a data centre). *F. Is sIgnificantly better service essential (vs incrementally better)?** *Altho this is the most controversial question, I think it is the most important: * Without the wow factor of the potential for new products and applications, deployment has to be pushed by a few individual visionary engineers, rather than pulled by product and marketing /teams/. * Starting with a new queue with near-zero queuing delay creates 'social pressure' to keep it that way (similar to the TCP friendliness meme). This may be sufficient to avoid the need for behaviour policing. Whereas, incrementally better delay gets polluted by incrementally worse behaviour, so it either never gets anywhere, or it requires complex behaviour policing. HTH Bob {Note 1} Both the dilemmas A#1 & A#2 are inherent to any congestion control of the form r = K/(R^c * p^b), where r : packet rate K: constant R: RTT p: drop probability c & b are constants of the congestion control algo. Even if c=0 (i.e. the congestion control is RTT-independent), dilemma #1 becomes the same as dilemma #2. -- ________________________________________________________________ Bob Briscoe http://bobbriscoe.net/ --------------080900060301060703060509 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Costin & all on the tcpprague list,

You asked "Is there another way to do this?" in the bar BoF in Prague. I am taking "this" to mean the features of the proposed new service (L4S),. Then the questions becomes, "Are all the elements needed at once?":
* Low Loss,
* Low Latency
* Scalable throughput.

[For those on the list who were not in Prague, a way to achieve L4S is described in this pre-print: "Data Centre to the Home"]

There are a number of elements to "this", so I'll take them in vaguely decreasing order of obviousness. Then everyone can identify any unnecessary assumptions.
A. ECN-capable (sndr, rcvr, aqm)?
B. Shift ECN smoothing from AQM to source (aqm, sndr)?
C. Second queue (aqm)?
D. Scalable congestion control (sndr)?
E. Accurate ECN feedback (sndr, rcvr)?
F. Finally, a more intangible question: significantly better service vs incrementally better?

The further features we discussed in the Bar BoF are secondary to all these questions.

Taking each in turn...

A. Is ECN essential?
(sndr, rcvr, aqm)
I think we're all agreed that ECN is essential. Without ECN:
#1. as load increases (i.e. as each TCP flow's rate has to decrease), an AQM has to introduce excessive drop to keep queuing delay low.
  (If you weren't in the AQM wg, see "questioning a fixed delay target" or the tech report that backs it up).
#2. Throughput scaling requires roughly 1 congestion signal per RTT at high flow rates, at least within an order of magnitude. As load increases, segments per RTT (the window) has to reduce, so the drop ratio has to increase. With low RTT (assuming we have achieved v low queuing delay) the window is also smaller.

See also {Note 1}

B. Is shifting smoothing of ECN signals from the AQM to the source essential?
(aqm, sndr)
There's no reason to smooth ECN in the AQM, and little harm from not smoothing it.

A newly implemented source can smooth incoming ECN feedback itself.
If a classic TCP-ECN source does not smooth ECN feedback, it's not a killer, because it will ignore >1 feedback per RTT anyway.
If a source smooths ECN that has already been smoothed in the AQM, its response could be sluggish, but we can make this unlikely as follows:

Smoothing (burst allowance) should be turned off by default for ECN in all AQMs (RED, PIE, CoDel, etc) now.

C. Is a second queue essential?
(aqm)
With one queue, 'new' traffic shares the latency of the queue with 'classic' Internet traffic.
So to achieve low latency with one queue, the AQM threshold for all traffic has to be shallow. To achieve really low latency, this means classic traffic would severely underutilise capacity. Colleagues in the RITE project tried shifting the AQM threshold down only if there was no ECN traffic, but they couldn't get it to work.

Is full utilisation important? Well, probably not as important as it used to be. But network operators (ISPs or DC operators) are not going to deploy an AQM that prevents existing traffic using a large proportion of the link capacity.

D. Is scalable congestion control essential?
(sndr)
As under question A, throughput scaling requires roughly 1 congestion signal per RTT at high flow rates, at least within an order of magnitude.
I.e. if the congestion control algo gives packet rate proportional to 1/p^b, where p is drop probability, then scalable means b=1

For now, we can probably make do with b<1 as long as b~=1, e.g. b=0.8 or something similar.
However, we will need b=1 in the future. So it all depends whether we can map out a route to increase to b=1 later, bearing in mind this traffic will still have to co-exist with b=0.5 (Reno) as well as b=0.75 (Cubic).

E. Is Accurate ECN feedback essential?
(sndr, rcvr)
Without AccECN, each downward rate response to one ECN feedback has to be greater, because it has to be enough for a whole round trip. This makes the sawteeth bigger, which creates the dilemma between higher delay and lower utilisation. This is primarily important when numerous flows are arriving and departing but the total number is not huge (i.e. typical conditions on an access link, or in a data centre).

F. Is sIgnificantly better service essential (vs incrementally better)?
Altho this is the most controversial question, I think it is the most important:
* Without the wow factor of the potential for new products and applications, deployment has to be pushed by a few individual visionary engineers, rather than pulled by product and marketing /teams/.
* Starting with a new queue with near-zero queuing delay creates 'social pressure' to keep it that way (similar to the TCP friendliness meme). This may be sufficient to avoid the need for behaviour policing. Whereas, incrementally better delay gets polluted by incrementally worse behaviour, so it either never gets anywhere, or it requires complex behaviour policing.

HTH


Bob

{Note 1} Both the dilemmas A#1 & A#2 are inherent to any congestion control of the form r = K/(R^c * p^b), where
  r : packet rate
  K: constant
  R: RTT
  p: drop probability
  c & b are constants of the congestion control algo.
Even if c=0 (i.e. the congestion control is RTT-independent), dilemma #1 becomes the same as dilemma #2.

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/
--------------080900060301060703060509--