From owner-ecm@wyvern.aciri.org Sun Oct 1 21:26:43 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA14370 for ; Sun, 1 Oct 2000 21:26:42 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id SAA50069 for ecm-outgoing; Sun, 1 Oct 2000 18:23:09 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from hotbot.com (sdn-ar-003flflauP241.dialsprint.net [168.191.85.3]) by wyvern.aciri.org (8.9.3/8.9.3) with SMTP id SAA50058; Sun, 1 Oct 2000 18:23:01 -0700 (PDT) (envelope-from HerbV399@hotbot.com) From: HerbV399@hotbot.com Subject: Herbal Viagra Date: Sun, 1 Oct 2000 21:19:01 Message-Id: <429.729305.644304@unknown> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-ecm@aciri.org Precedence: bulk X-MIME-Autoconverted: from 8bit to quoted-printable by wyvern.aciri.org id SAB50069 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id VAA14370 Herbal V: An Incredible All-Natural Healthy Alternative to Viagra Herbal V is the All Natural Approach to Male Virility, Vitality and Pleasure. Available N o w ! Welcome to the New Sexual Revolution. It's the all natural male potency and pleasure pill that men everywhere are buzzing about. Herbal V isafe, natural and specifically formulated to help support male sexual function and pleasure. You just take two easy-to-swallow tablets one hour before sex. And there's more great news - you can get Herbal V for less than $1 a pill. Amazing word of mouth praise on Herbal V has been spreading like wildfire-already over 1,500,000 men have chosen Herbal V. Since it is 100% natural you will never have to worry about safety. Try doctor-recommended Herbal V today and have the greatest night of your life! Herbal V... Bringing Back the Magic! 1,585,000 men can't be wrong. To date over 1 million men have tried the super supplement Herbal V. Here is why: No Doctor Visit Required Available Over the Counter Not a Drug 100% Natural Safe, No Worries Highest Quality Pharmaceutical-Grade Pure Nutriceuticals Guaranteed Potency & Purity Be a Real Man Again! Questions and Answers What is Herbal V? Herbal V is a proprietary blend that was specifically developed as a safe alternative for men who prefer an all-natural approach to address impotence and boost sexual performance. This amazing formula first became popular with Hollywood insiders and the wealthy elite. They were maximizing their sex lives, long before it was available to the general public. How does Herbal V work? Developed by a team whose goal was to create the perfect all-natural aphrodisiac. Herbal V is the result of that remarkable effort. The Herbal V formula contains a precise blend of cutting edge pro-sexual nutrients from around the world that provide nutritional support, making it possible for a man to have a pleasurable sexual experience. What can Herbal V do for me? Herbal V helps support male sexual function and pleasure in a safe and natural manner. Simply put, it can make your sex life incredible. Is Herbal V Safe? One of the great things about Herbal V is that it is not a drug. It is an incredible herbal dietary supplement that provides nutritional support for male sexual function and pleasure. One of the most comforting features of Herbal V is that you never have to worry about safety. Herbal V: Safe - Natural - Exciting Many have speculated that because Herbal V is so popular with men, it must contain prescription drugs or chemical components. Herbal V does not contain any elements or traces of any prescription drug. Herbal V is made using the world's most technologically advanced state-of-the-art cold processing equipment to ensure maximum purity. Herbal V has been independently analyzed by the nation's premier testing facility to ensure purity, quality and to end the rumors that, because it is so popular, it must somehow be chemical. It is not. Herbal V is natural - just as it says on the label. Herbal V is simply fantastic! Herbal V: Ingredients Yohimbe, saw palmetto, avena sativa, androstenedione, guarana, taurine, siberian ginseng, tribulus terrestris. Tribulus Terrestis is certified to enhanced testosterone levels by increasing Luteinzing hormone (LH) levels. Androstenedione which is a precursor to testosterone unlocks bound testosterone and makes it biologically active again quickly. This means a dramatic surge in desire. Avena Sativa Stimulates the neurotransmitter pleasure centers to maximum capacity. This greatly intensifies pleasure. Just listen to what Herbal V has done for the sex lives of people like you! “On a scale of 1 to 10, it's a 15. Electrifying. It's like a wonder pill!” — Justin Q B., New Haven, Texas “I haven't had sexual relations in 11 years. Then with Herbal V it was... wow! It works again!” — Sid R., Lakeland, Florida “I had sex four times in one night. It made me feel like a 19-year-old again.” — Chip S, Beech Mountain, North Carolina “Herbal V has turned my husband into a Sexual Superman! I like the fact that it's all natural and has no side effects. It's bringing back the good old days.” — Jennifer B, Beverly Hills, California The above testimonials are from product literature, and we have not independently verified them. However, the following testimonial is from a "senior" gentleman who has purchased his second bottle of Herbal V. When we heard his words with our own ears, we asked his permission to print them here. “Man! I'm wild as I can be! I feel like I'm 25 years old again! I'm not believing this!” — Mr. Murphy, age 64, Lampart, IL. Risk Free: Double Your Money Back Guarantee If Herbal V does not give the desired results as stated above, simply return the unused portion for a double-your money back refund. No questions asked ! Order Now: Safe, Fast, Secure, Private Herbal V with its DOUBLE YOUR MONEY BACK GUARANTEE is available only through this special promotional offer. Herbal V arrives in plain packaging for your privacy. Any and all information is kept strictly confidential. Payment Methods You may FAX or Postal Mail Checks, MasterCard, Visa, & American Express.payments. Money Orders are accepted only by Postal Mail. Each bottle of Herbal V contains 30 tablets, approximately a 1 month supply. Step 1: Place a check by your desired quanity. ______ 1 Bottle of Herbal V $28 ______ 2 Bottles of Herbal V $48 ______ 3 Bottles of Herbal V $63 Please add $6 shipping and handling for any size order. [ Total cost including shipping & handling, 1 bottle=$34, 2 bottles=$54, 3 bottles=$69 ] Step 2: Place a check by your desired payment method and complete fields if necessary. _____Check or CHECK-BY-FAX [details below] _____Money Order _____American Express Account Number__________________ Exp____/____ _____Visa Account Number__________________ Exp____/____ _____MasterCard Account Number__________________ Exp____/____ Please make your check or money order payable to "Lion Sciences National". Charges will appear discreetly as "Lion S National". Step 3: Please complete and print the following fields clearly. Name ___________________________________________________ Address _________________________________________________ City ____________________________________________________ State ___________________________________________________ Zip _____________________________________________________ E-mail __________________________________________________ Signature _________________________________________________ [ required for check and credit card orders] Toll Free FAX Order Line: 1-800-940-6590 If faxing in your order, please state whether you require a fax, email, or no confirmation at all. Allow up to one day for confirmation, if requested. FAX orders are processed immediately. Or, print & mail to: LSN 273 S. State Rd. 7 #193 Margate, Fl 33068-5727 ______________________________________________________ *CHECK BY FAX ORDERS: Complete the check as normal. Tape the check in the area below. Below the check, clearly write the check number, all numbers at the bottom of the check, & your name. Tape the check below and fax the check to the toll free FAX number above. Void the check. Our merchant will electronically debit your account for the amount of the check; your reference number for this transaction will be your check number. Nothing could be safer & easier ! TAPE CHECK BELOW _____________________________________________________________ This is a one time mailing: Removal is automatic and no further contact is necessary. Please Note: Herbal V is not intended to diagnose, treat, cure or prevent any disease. As individuals differ, so will results. Herbal V helps provide herbal and nutritional support for male sexual performance. The FDA has not evaluated these statements. For details about our double your money back guarantee, please write to the above address, attention consumer affairs department; enclose a self addressed stamped envelope for this and any requested contact information. Thank You. From owner-ecm@wyvern.aciri.org Sun Oct 8 12:05:55 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA09883 for ; Sun, 8 Oct 2000 12:05:54 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id JAA15664 for ecm-outgoing; Sun, 8 Oct 2000 09:03:23 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from mail.sinet.net.cn (szptt103-190.szptt.net.cn [202.103.190.4] (may be forged)) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id JAA15659 for ; Sun, 8 Oct 2000 09:03:21 -0700 (PDT) (envelope-from rsb@docsj.de) From: rsb@docsj.de Received: from isd.com.cn ([202.103.148.117]) by mail.sinet.net.cn (8.9.3/8.9.3) with SMTP id XAA04834; Sun, 8 Oct 2000 23:59:44 +0800 (CST) Date: Sun, 8 Oct 2000 23:59:44 +0800 (CST) Message-Id: <200010081559.XAA04834@mail.sinet.net.cn> Received: from pavilion ([206.214.131.49]) by isd.com.cn (Lotus SMTP MTA v4.6.1 (569.2 2-6-1998)) with SMTP id 48256972.0057E4EE; Thu, 8 Oct 1970 23:58:29 +0800 To: rsb@docsj.de Subject: At last, HERBAL V the all natural alternative! Sender: owner-ecm@aciri.org Precedence: bulk Herbal V: An Incredible All-Natural Healthy Alternative Herbal V is the All Natural Approach to Male Virility, Vitality and Pleasure. Available N o w ! Welcome to the New Sexual Revolution. It's the all natural male potency and pleasure pill that men everywhere are buzzing about. Herbal V is safe, natural and specifically formulated to help support male sexual function and pleasure. You just take two easy-to-swallow tablets one hour before sex. And there's more great news - you can get Herbal V for less than $1 a pill. Amazing word of mouth praise on Herbal V has been spreading like wildfire-already over 1,500,000 men have chosen Herbal V. Since it is 100% natural you will never have to worry about safety. Try doctor-recommended Herbal V today and have the greatest night of your life! Herbal V... Bringing Back the Magic! 1,585,000 men can't be wrong. To date over 1 million men have tried the super supplement Herbal V. Here is why: No Doctor Visit Required Available Over the Counter Not a Drug 100% Natural Safe, No Worries Highest Quality Pharmaceutical-Grade Pure Nutriceuticals Guaranteed Potency & Purity Be a Real Man Again! Questions and Answers What is Herbal V? Herbal V is a proprietary blend that was specifically developed as a safe alternative for men who prefer an all-natural approach to address impotence and boost sexual performance. This amazing formula first became popular with Hollywood insiders and the wealthy elite. They were maximizing their sex lives, long before it was available to the general public. How does Herbal V work? Developed by a team whose goal was to create the perfect all-natural aphrodisiac. Herbal V is the result of that remarkable effort. The Herbal V formula contains a precise blend of cutting edge pro-sexual nutrients from around the world that provide nutritional support, making it possible for a man to have a pleasurable sexual experience. What can Herbal V do for me? Herbal V helps support male sexual function and pleasure in a safe and natural manner. Simply put, it can make your sex life incredible. Is Herbal V Safe? One of the great things about Herbal V is that it is not a drug. It is an incredible herbal dietary supplement that provides nutritional support for male sexual function and pleasure. One of the most comforting features of Herbal V is that you never have to worry about safety. Herbal V: Safe - Natural - Exciting Many have speculated that because Herbal V is so popular with men, it must contain prescription drugs or chemical components. Herbal V does not contain any elements or traces of any prescription drug. Herbal V is made using the world's most technologically advanced state-of-the-art cold processing equipment to ensure maximum purity. Herbal V has been independently analyzed by the nation's premier testing facility to ensure purity, quality and to end the rumors that, because it is so popular, it must somehow be chemical. It is not. Herbal V is natural - just as it says on the label. Herbal V is simply fantastic! Herbal V: Ingredients Yohimbe, saw palmetto, avena sativa, androstenedione, guarana, taurine, siberian ginseng, tribulus terrestris. Tribulus Terrestis is certified to enhanced testosterone levels by increasing Luteinzing hormone (LH) levels. Androstenedione which is a precursor to testosterone unlocks bound testosterone and makes it biologically active again quickly. This means a dramatic surge in desire. Avena Sativa Stimulates the neurotransmitter pleasure centers to maximum capacity. This greatly intensifies pleasure. Just listen to what Herbal V has done for the sex lives of people like you! “On a scale of 1 to 10, it's a 15. Electrifying. It's like a wonder pill!” — Justin Q B., New Haven, Texas “I haven't had sexual relations in 11 years. Then with Herbal V it was... wow! It works again!” — Sid R., Lakeland, Florida “I had sex four times in one night. It made me feel like a 19-year-old again.” — Chip S, Beech Mountain, North Carolina “Herbal V has turned my husband into a Sexual Superman! I like the fact that it's all natural and has no side effects. It's bringing back the good old days.” — Jennifer B, Beverly Hills, California The above testimonials are from product literature, and we have not independently verified them. However, the following testimonial is from a "senior" gentleman who has purchased his second bottle of Herbal V. When we heard his words with our own ears, we asked his permission to print them here. “Man! I'm wild as I can be! I feel like I'm 25 years old again! I'm not believing this!” — Mr. Murphy, age 64, Lampart, IL. Risk Free: Double Your Money Back Guarantee If Herbal V does not give the desired results as stated above, simply return the unused portion for a double-your money back refund. No questions asked ! Order Now: Safe, Fast, Secure, Private Herbal V with its DOUBLE YOUR MONEY BACK GUARANTEE is available only through this special promotional offer. Herbal V arrives in plain packaging for your privacy. Any and all information is kept strictly confidential. Payment Methods You may FAX or Postal Mail Checks, MasterCard, Visa, & American Express.payments. Money Orders are accepted only by Postal Mail. Each bottle of Herbal V contains 30 tablets, approximately a 1 month supply. Step 1: Place a check by your desired quanity. ______ 1 Bottle of Herbal V $24 ______ 2 Bottles of Herbal V $44 ______ 3 Bottles of Herbal V $59 Please add $6 shipping and handling for any size order. [ Total cost including shipping & handling, 1 bottle=$30, 2 bottles=$50, 3 bottles=$65 ] International Orders Please add $16 shipping and handling for any size order. [ Total cost including shipping & handling, 1 bottle=$40, 2 bottles=$60, 3 bottles=$75 ] Step 2: Place a check by your desired payment method and complete fields if necessary. _____Check or CHECK-BY-FAX [details below] _____Money Order _____American Express Account Number__________________ Exp____/____ _____Visa Account Number__________________ Exp____/____ _____MasterCard Account Number__________________ Exp____/____ Please make your check or money order payable to "Lion Sciences National". Step 3: Please complete and print the following fields clearly. Name ___________________________________________________ Address _________________________________________________ City ____________________________________________________ State ___________________________________________________ Zip _____________________________________________________ E-mail __________________________________________________ Signature _________________________________________________ [ required for check and credit card orders] Toll Free FAX Order Line: 1-800-940-6590 If faxing in your order, please state whether you require a fax, email, or no confirmation at all. Allow up to one day for confirmation, if requested. FAX orders are processed immediately. Or, print & mail to: LSN 3502 N. Powerline Rd. #525 Pompano Beach, FL 33069 ______________________________________________________ *CHECK BY FAX ORDERS: Complete the check as normal. Tape the check in the area below. Below the check, clearly write the check number, all numbers at the bottom of the check, & your name. Tape the check below and fax the check to the toll free FAX number above. Void the check. Our merchant will electronically debit your account for the amount of the check; your reference number for this transaction will be your check number. Nothing could be safer & easier ! TAPE CHECK BELOW _____________________________________________________________ This is a one time mailing: Removal is automatic and no further contact is necessary. Please Note: Herbal V is not intended to diagnose, treat, cure or prevent any disease. As individuals differ, so will results. Herbal V helps provide herbal and nutritional support for male sexual performance. The FDA has not evaluated these statements. For details about our double your money back guarantee, please write to the above address, attention consumer affairs department; enclose a self addressed stamped envelope for this and any requested contact information. Thank You. From owner-ecm@wyvern.aciri.org Sun Oct 8 18:53:42 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA11870 for ; Sun, 8 Oct 2000 18:53:41 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id PAA17328 for ecm-outgoing; Sun, 8 Oct 2000 15:52:54 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from mail0.mia.bellsouth.net (mail0.mia.bellsouth.net [205.152.144.12]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id PAA17323 for ; Sun, 8 Oct 2000 15:52:53 -0700 (PDT) (envelope-from bubblehead32@hotmail.com) From: bubblehead32@hotmail.com Received: from www.goldendeckcasino.com (adsl-61-143-206.mia.bellsouth.net [208.61.143.206]) by mail0.mia.bellsouth.net (3.3.5alt/0.75.2) with SMTP id SAA03851; Sun, 8 Oct 2000 18:51:52 -0400 (EDT) Message-Id: <200010082251.SAA03851@mail0.mia.bellsouth.net> To: <> Subject: WOW!!! Highest Payouts Around!!!!!!!! Date: Sun, 08 Oct 2000 18:38:55 -0400 X-Sender: bubblehead32@hotmail.com X-Mailer: QUALCOMM Windows Eudora Pro Version 4.1 Content-Type: text/plain; charset="us-ascii" X-Priority: 3 X-MSMail-Priority: Normal Sender: owner-ecm@aciri.org Precedence: bulk Its just like being there. Go to www.goldendeckcasino.com/goldendeckcasino/links/2769.html. If you would like to be removed from these mailings in the future please mailto:bubblehead32@hotmail.com?subject=remove From owner-ecm@wyvern.aciri.org Mon Oct 16 01:14:13 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA16964 for ; Mon, 16 Oct 2000 01:14:12 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id WAA76432 for ecm-outgoing; Sun, 15 Oct 2000 22:11:52 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id WAA76424 for ; Sun, 15 Oct 2000 22:11:51 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: from breeze.lcs.mit.edu (IDENT:hari@localhost.localdomain [127.0.0.1]) by breeze.lcs.mit.edu (8.11.0/8.9.3) with ESMTP id e9G1xBv20056 for ; Sun, 15 Oct 2000 21:59:11 -0400 Message-Id: <200010160159.e9G1xBv20056@breeze.lcs.mit.edu> X-Mailer: exmh version 2.1.1 10/15/1999 X-url: http://nms.lcs.mit.edu/~hari/ From: Hari Balakrishnan Reply-to: Hari Balakrishnan To: ecm@aciri.org Subject: Next draft of ECM WG congestion manager document Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 15 Oct 2000 21:59:07 -0400 Sender: owner-ecm@aciri.org Precedence: bulk On Thu, 03 Aug 2000 00:33:20 PDT, Vern Paxson wrote: > The chair then made the following proposal for moving forward with ecm-cm-01: > > 1. Add opaque (scheduler) data to the API when creating a macroflow. > 2. Add specification of how to compute the RTT variation. > 3. Change CM_PERSISTENT to CM_LOST_FEEDBACK, etc. > 4. Add a comment to the document that some key experience we don't > yet have is with scheduling APIs, and that this may lead to > possible changes in the CM API. > 5. Resolve the issue Joe raised regarding sending multiple packets. > > After addressing these, there would be one more WG Last Call. The chair > then asked for a show of WG consensus for this plan, with it being understood > that the chair would interpret consensus for the plan followed by a > successful last call as consensus for the document. A show of hands > revealed good consensus and no opposition. At long last, there's now a new draft of the IETF ECM WG Congestion Manager draft. Many apologies for the delay. It is called draft-ietf-ecm-cm-02.txt and is attached to the end of this message for comments (it is also available from http://nms.lcs.mit.edu/papers/draft-ietf-ecm-cm-02.txt). It hasn't been submitted to the IETF yet; the intention is to get some discussion going and obtain comments well in advance of the deadline of the next IETF. There were 6 pending issues, 5 of which are mentioned above. Of the ones above, items 2, 3, and 4 have been incorporated into the document. A sixth item, brought up by Tim Shepard, concerning NATs and the behavior of the CM in that context has been added to section 4.5 (sharing granularity). We have thought a great deal about issues 1 and 5, and the rest of this note concerns them. We would appreciate any thoughts you might have on this. > 1. Add opaque (scheduler) data to the API when creating a macroflow. While we do agree that a sophisticated scheduler can enable a lot of new functionality, the CM itself (as described in this document) does not require anything more than a trivial round-robin scheduler to be valuable. That's because even such a trivial scheduler provides the machinery to do proper congestion control and provides an API for adaptation. Throughout the document, we have strived to maintain the modularity of the different CM components, in particular the congestion controller, the scheduler, and the API. Thus, rather than add opaque scheduler data to the cm_open() call, which many (simple) schedulers may not even care to look at, we strongly suggest that the right approach (to achieve what Joe wants) would be to: 1. call cm_open(stream_info) and get back a streamid. 2. call a scheduler API function (which would in general be dependent on the particular scheduler), passing the streamid (and perhaps the macroflowid if required), in order for the appropriate scheduler parameters to be initialized. Note that although this will require an additional system call, the additional call overhead is observed only once, at stream setup time. In addition, the separated versions of the two calls (i.e., the non-cm_open version to invoke the scheduler) must be supported, since an application may want to change the scheduler parameters during the middle of the stream. In fact, we believe that this is a common mode; a Web server might decide on the priority/bandwidth of a stream only after it sees the full GET request. The specific details of the scheduler contents are obviously scheduler-dependent. In fact, it may not be a good idea to standardize too much of this, since the details of a good scheduler may allow vendors to differentiate themselves. > 5. Resolve the issue Joe raised regarding sending multiple packets. The issue here is to make the cm_request(id) call, which currently takes no other arguments, richer. In particular, the suggestion is: cm_request(id, npkts, size) where npkts is the number of packets to be sent and size the size of each packet. The apparent reason for this is the smaller number of cm_request() calls. I tried to articulate (at the last IETF) why this may not be a good idea. Here's the reasoning: 1. The current API is not as inefficient as one might believe. It may be tempting to think that the current API causes a user-kernel crossing for each packet. That's not necessarily true. A user-level library that implements the API can package multiple such calls into one system call. For in-kernel apps like TCP, cm_request(id) is only a function call. And if you believe that a function call is expensive, inlining it (and inlining the in-kernel function callback) is pretty straightforward. 2. If we add a "MAY" clause for cm_request(id, npkts, size), it MUST require that applications negotiate the specific supported mode in the API. And each application would have to be written to handle both modes, or face the risk of not running on some platform. I am not sure at all that complicating the API to accommodate this, in the absence of significant experience that dictates that it's necessary, is a good idea. Furthermore, as Vern has pointed out and we as fully also agree with, this is not cast in stone. As we gain more experience with CM implementations, changes will both be made and be welcome. Adding this option to the cm_request() call may well be a prime candidate, when such experience is obtained. Note that several non-ALF applications may not care to receive a callback for each transmission. They can, and will be expected to, use the congestion-controlled UDP sockets option (desribed in Section 6.1.2). TCP applications don't have a problem because it is possible to write a TCP-over-CM that performs virtually identically to a vanilla TCP despite using these per-function (in-kernel) callbacks. Please send your comments and observations to the group, so that we may discuss these before the next IETF. Thanks! Hari & Srini From owner-ecm@wyvern.aciri.org Mon Oct 16 01:14:13 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA16965 for ; Mon, 16 Oct 2000 01:14:12 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id WAA76437 for ecm-outgoing; Sun, 15 Oct 2000 22:11:53 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id WAA76430 for ; Sun, 15 Oct 2000 22:11:52 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: from breeze.lcs.mit.edu (IDENT:hari@localhost.localdomain [127.0.0.1]) by breeze.lcs.mit.edu (8.11.0/8.9.3) with ESMTP id e9G207v20076 for ; Sun, 15 Oct 2000 22:00:07 -0400 Message-Id: <200010160200.e9G207v20076@breeze.lcs.mit.edu> X-Mailer: exmh version 2.1.1 10/15/1999 X-url: http://nms.lcs.mit.edu/~hari/ From: Hari Balakrishnan Reply-to: Hari Balakrishnan To: ecm@aciri.org Subject: CM draft Mime-Version: 1.0 Content-Type: multipart/mixed ; boundary="==_Exmh_-20288952360" Date: Sun, 15 Oct 2000 22:00:07 -0400 Sender: owner-ecm@aciri.org Precedence: bulk This is a multipart MIME message. --==_Exmh_-20288952360 Content-Type: text/plain; charset=us-ascii Oops, I forgot to attach the draft to my previous note. Here is is. --==_Exmh_-20288952360 Content-Type: text/plain ; name="draft-ietf-ecm-cm-02.txt"; charset=us-ascii Content-Description: draft-ietf-ecm-cm-02.txt Content-Disposition: attachment; filename="draft-ietf-ecm-cm-02.txt" Content-Transfer-Encoding: base64 SW50ZXJuZXQgRW5naW5lZXJpbmcgVGFzayBGb3JjZSAgICAgICAgICAgICAgICAgICAgICAg SGFyaSBCYWxha3Jpc2huYW4KSU5URVJORVQgRFJBRlQgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIE1JVCBMQ1MKRG9jdW1lbnQ6IGRyYWZ0LWll dGYtZWNtLWNtLTAyLnR4dCAgICAgICAgICAgICAgICAgICAgU3Jpbml2YXNhbiBTZXNoYW4K ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBDTVUKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIE9jdG9iZXIsIDIwMDAKCQkJCQkJICAgIEV4cGlyZXM6 IEFwcmlsIDIwMDEKCiAgICAgICAgICAgICAgICAgICAgICAgIFRoZSBDb25nZXN0aW9uIE1h bmFnZXIKClN0YXR1cyBvZiB0aGlzIE1lbW8KCiAgIFRoaXMgZG9jdW1lbnQgaXMgYW4gSW50 ZXJuZXQtRHJhZnQgYW5kIGlzIGluIGZ1bGwgY29uZm9ybWFuY2Ugd2l0aAogICAgICBhbGwg cHJvdmlzaW9ucyBvZiBTZWN0aW9uIDEwIG9mIFJGQy0yMDI2IFtCcmFkbmVyOTZdLgoKICAg SW50ZXJuZXQtRHJhZnRzIGFyZSB3b3JraW5nIGRvY3VtZW50cyBvZiB0aGUgSW50ZXJuZXQg RW5naW5lZXJpbmcKICAgVGFzayBGb3JjZSAoSUVURiksIGl0cyBhcmVhcywgYW5kIGl0cyB3 b3JraW5nIGdyb3Vwcy4gTm90ZSB0aGF0CiAgIG90aGVyIGdyb3VwcyBtYXkgYWxzbyBkaXN0 cmlidXRlIHdvcmtpbmcgZG9jdW1lbnRzIGFzIEludGVybmV0LQogICBEcmFmdHMuIEludGVy bmV0LURyYWZ0cyBhcmUgZHJhZnQgZG9jdW1lbnRzIHZhbGlkIGZvciBhIG1heGltdW0gb2YK ICAgc2l4IG1vbnRocyBhbmQgbWF5IGJlIHVwZGF0ZWQsIHJlcGxhY2VkLCBvciBvYnNvbGV0 ZWQgYnkgb3RoZXIKICAgZG9jdW1lbnRzIGF0IGFueSB0aW1lLiBJdCBpcyBpbmFwcHJvcHJp YXRlIHRvIHVzZSBJbnRlcm5ldC0gRHJhZnRzCiAgIGFzIHJlZmVyZW5jZSBtYXRlcmlhbCBv ciB0byBjaXRlIHRoZW0gb3RoZXIgdGhhbiBhcyAid29yayBpbgogICBwcm9ncmVzcy4iCiAg IFRoZSBsaXN0IG9mIGN1cnJlbnQgSW50ZXJuZXQtRHJhZnRzIGNhbiBiZSBhY2Nlc3NlZCBh dAogICBodHRwOi8vd3d3LmlldGYub3JnL2lldGYvMWlkLWFic3RyYWN0cy50eHQKICAgVGhl IGxpc3Qgb2YgSW50ZXJuZXQtRHJhZnQgU2hhZG93IERpcmVjdG9yaWVzIGNhbiBiZSBhY2Nl c3NlZCBhdAogICBodHRwOi8vd3d3LmlldGYub3JnL3NoYWRvdy5odG1sLgoKCgoxLiAgICAg IEFic3RyYWN0CgogICBUaGlzIGRvY3VtZW50IGRlc2NyaWJlcyB0aGUgQ29uZ2VzdGlvbiBN YW5hZ2VyIChDTSksIGFuIGVuZC1zeXN0ZW0KICAgbW9kdWxlIHRoYXQ6CgogICAoaSkgRW5h YmxlcyBhbiBlbnNlbWJsZSBvZiBtdWx0aXBsZSBjb25jdXJyZW50IHN0cmVhbXMgZnJvbSBh CiAgIHNlbmRlciBkZXN0aW5lZCB0byB0aGUgc2FtZSByZWNlaXZlciBhbmQgc2hhcmluZyB0 aGUgc2FtZQogICBjb25nZXN0aW9uIHByb3BlcnRpZXMgdG8gcGVyZm9ybSBwcm9wZXIgY29u Z2VzdGlvbiBhdm9pZGFuY2UgYW5kCiAgIGNvbnRyb2wsIGFuZAoKICAgKGlpKSBBbGxvd3Mg YXBwbGljYXRpb25zIHRvIGVhc2lseSBhZGFwdCB0byBuZXR3b3JrIGNvbmdlc3Rpb24uCiAg IAogICBUaGUgZnJhbWV3b3JrIGRlc2NyaWJlZCBpbiB0aGlzIGRvY3VtZW50IGludGVncmF0 ZXMgY29uZ2VzdGlvbgogICBtYW5hZ2VtZW50IGFjcm9zcyBhbGwgYXBwbGljYXRpb25zIGFu ZCB0cmFuc3BvcnQgcHJvdG9jb2xzLiBUaGUgQ00KICAgbWFpbnRhaW5zIGNvbmdlc3Rpb24g cGFyYW1ldGVycyAoYXZhaWxhYmxlIGFnZ3JlZ2F0ZSBhbmQgcGVyLXN0cmVhbQogICBiYW5k d2lkdGgsIHBlci1yZWNlaXZlciByb3VuZC10cmlwIHRpbWVzLCBldGMuKSBhbmQgZXhwb3J0 cyBhbiBBUEkKICAgdGhhdCBlbmFibGVzIGFwcGxpY2F0aW9ucyB0byBsZWFybiBhYm91dCBu ZXR3b3JrIGNoYXJhY3RlcmlzdGljcywKICAgcGFzcyBpbmZvcm1hdGlvbiB0byB0aGUgQ00s IHNoYXJlIGNvbmdlc3Rpb24gaW5mb3JtYXRpb24gd2l0aCBlYWNoCiAgIG90aGVyLCBhbmQg c2NoZWR1bGUgZGF0YSB0cmFuc21pc3Npb25zLiAgVGhpcyBkb2N1bWVudCBmb2N1c2VzIG9u CiAgIGFwcGxpY2F0aW9ucyBhbmQgdHJhbnNwb3J0IHByb3RvY29scyB3aXRoIHRoZWlyIG93 biBpbmRlcGVuZGVudAogICBwZXItYnl0ZSBvciBwZXItcGFja2V0IHNlcXVlbmNlIG51bWJl ciBpbmZvcm1hdGlvbiwgYW5kIGRvZXMgbm90CiAgIHJlcXVpcmUgbW9kaWZpY2F0aW9ucyB0 byB0aGUgcmVjZWl2ZXIgcHJvdG9jb2wgc3RhY2suICBIb3dldmVyLCB0aGUKICAgcmVjZWl2 aW5nIGFwcGxpY2F0aW9uIG11c3QgcHJvdmlkZSBmZWVkYmFjayB0byB0aGUgc2VuZGluZwog ICBhcHBsaWNhdGlvbiBhYm91dCByZWNlaXZlZCBwYWNrZXRzIGFuZCBsb3NzZXMsIGFuZCB0 aGUgbGF0dGVyIGlzCiAgIGV4cGVjdGVkIHRvIHVzZSB0aGUgQ00gQVBJIHRvIHVwZGF0ZSBD TSBzdGF0ZS4gIFRoaXMgZG9jdW1lbnQgZG9lcwogICBub3QgYWRkcmVzcyBuZXR3b3JrcyB3 aXRoIHJlc2VydmF0aW9ucyBvciBzZXJ2aWNlIGRpZmZlcmVudGlhdGlvbi4KCjIuICAgICAg Q29udmVudGlvbnMgdXNlZCBpbiB0aGlzIGRvY3VtZW50OgogICBUaGUga2V5IHdvcmRzICJN VVNUIiwgIk1VU1QgTk9UIiwgIlJFUVVJUkVEIiwgIlNIQUxMIiwgIlNIQUxMIE5PVCIsCiAg ICJTSE9VTEQiLCAiU0hPVUxEIE5PVCIsICJSRUNPTU1FTkRFRCIsICJNQVkiLCBhbmQgIk9Q VElPTkFMIiBpbgogICB0aGlzIGRvY3VtZW50IGFyZSB0byBiZSBpbnRlcnByZXRlZCBhcyBk ZXNjcmliZWQgaW4gUkZDLTIxMTkKICAgW0JyYWRuZXI5N10uCgogICBTVFJFQU0gCglBIGdy b3VwIG9mIHBhY2tldHMgdGhhdCBhbGwgc2hhcmUgdGhlIHNhbWUgc291cmNlIGFuZAogICAg ICAgIGRlc3RpbmF0aW9uIElQIGFkZHJlc3MsIElQIHR5cGUtb2Ytc2VydmljZSwgdHJhbnNw b3J0CiAgICAgICAgcHJvdG9jb2wsIGFuZCBzb3VyY2UgYW5kIGRlc3RpbmF0aW9uIHRyYW5z cG9ydC1sYXllciBwb3J0CiAgICAgICAgbnVtYmVycy4KCiAgIE1BQ1JPRkxPVyAKCUEgZ3Jv dXAgb2Ygc3RyZWFtcyB0aGF0IGFsbCB1c2UgdGhlIHNhbWUgY29uZ2VzdGlvbiBtYW5hZ2Vt ZW50CiAgICAgICAgYW5kIHNjaGVkdWxpbmcgYWxnb3JpdGhtcywgYW5kIHNoYXJlIGNvbmdl c3Rpb24gc3RhdGUKICAgICAgICBpbmZvcm1hdGlvbi4gIEN1cnJlbnRseSwgc3RyZWFtcyBk ZXN0aW5lZCB0byBkaWZmZXJlbnQKICAgICAgICByZWNlaXZlcnMgYmVsb25nIHRvIGRpZmZl cmVudCBtYWNyb2Zsb3dzLiAgU3RyZWFtcyBkZXN0aW5lZCB0bwogICAgICAgIHRoZSBzYW1l IHJlY2VpdmVyIE1BWSBiZWxvbmcgdG8gZGlmZmVyZW50IG1hY3JvZmxvd3MuICBTdHJlYW1z CiAgICAgICAgdGhhdCBleHBlcmllbmNlIGlkZW50aWNhbCBjb25nZXN0aW9uIGJlaGF2aW9y IGluIHRoZSBJbnRlcm5ldAogICAgICAgIGFuZCB1c2UgdGhlIHNhbWUgY29uZ2VzdGlvbiBj b250cm9sIGFsZ29yaXRobSBTSE9VTEQgYmVsb25nIHRvCiAgICAgICAgdGhlIHNhbWUgbWFj cm9mbG93LgoKICAgQVBQTElDQVRJT04gCglBbnkgc29mdHdhcmUgbW9kdWxlIHRoYXQgdXNl cyB0aGUgQ00uICBUaGlzIGluY2x1ZGVzCiAgICAgICAgdXNlci1sZXZlbCBhcHBsaWNhdGlv bnMgc3VjaCBhcyBXZWIgc2VydmVycyBvciBhdWRpby92aWRlbwogICAgICAgIHNlcnZlcnMs IGFzIHdlbGwgYXMgaW4ta2VybmVsIHByb3RvY29scyBzdWNoIGFzIFRDUCBbUG9zdGVsODFd CiAgICAgICAgdGhhdCB1c2UgdGhlIENNIGZvciBjb25nZXN0aW9uIGNvbnRyb2wuCgogICBX RUxMLUJFSEFWRUQgQVBQTElDQVRJT04KICAgICAgICBBbiBhcHBsaWNhdGlvbiB0aGF0IG9u bHkgdHJhbnNtaXRzIHdoZW4gYWxsb3dlZCBieSB0aGUgQ00gYW5kCiAgICAgICAgYWNjdXJh dGVseSBhY2NvdW50cyBmb3IgYWxsIGRhdGEgdGhhdCBpdCBoYXMgc2VudCB0byB0aGUKICAg ICAgICByZWNlaXZlciBieSBpbmZvcm1pbmcgdGhlIENNIHVzaW5nIHRoZSBDTSBBUEkuCgog ICBQQVRIIE1BWElNVU0gVFJBTlNNSVNTSU9OIFVOSVQgKFBNVFUpCiAgICAgICAgVGhlIHNp emUgb2YgdGhlIGxhcmdlc3QgcGFja2V0IHRoYXQgdGhlIHNlbmRlciBjYW4gdHJhbnNtaXQK ICAgICAgICB3aXRob3V0IGl0IGJlaW5nIGZyYWdtZW50ZWQgZW4gcm91dGUgdG8gdGhlIHJl Y2VpdmVyLiAgSXQKICAgICAgICBpbmNsdWRlcyB0aGUgc2l6ZXMgb2YgYWxsIGhlYWRlcnMg YW5kIGRhdGEgZXhjZXB0IHRoZSBJUAogICAgICAgIGhlYWRlci4KCiAgIENPTkdFU1RJT04g V0lORE9XIChjd25kKQogICAgICAgIEEgQ00gc3RhdGUgdmFyaWFibGUgdGhhdCBtb2R1bGF0 ZXMgdGhlIGFtb3VudCBvZiBvdXRzdGFuZGluZwogICAgICAgIGRhdGEgYmV0d2VlbiBzZW5k ZXIgYW5kIHJlY2VpdmVyLgoKICAgT1VUU1RBTkRJTkcgV0lORE9XIChvd25kKQogICAgICAg IFRoZSBudW1iZXIgb2YgYnl0ZXMgdGhhdCBoYXMgYmVlbiB0cmFuc21pdHRlZCBieSB0aGUg c291cmNlLAogICAgICAgIGJ1dCBub3Qga25vd24gdG8gaGF2ZSBiZWVuIGVpdGhlciByZWNl aXZlZCBieSB0aGUgZGVzdGluYXRpb24KICAgICAgICBvciBsb3N0IGluIHRoZSBuZXR3b3Jr LgoKICAgSU5JVElBTCBXSU5ET1cgKElXKQogICAgICAgIFRoZSBzaXplIG9mIHRoZSBzZW5k ZXIncyBjb25nZXN0aW9uIHdpbmRvdyBhdCB0aGUgYmVnaW5uaW5nIG9mCiAgICAgICAgYSBt YWNyb2Zsb3cuCgogICBEQVRBIFRZUEUgU1lOVEFYCiAgICAgICAgICAgV2UgdXNlICJ1NjQi IGZvciB1bnNpZ25lZCA2NC1iaXQsICJ1MzIiIGZvciB1bnNpZ25lZCAzMi0KICAgYml0LCAi dTE2IiBmb3IgdW5zaWduZWQgMTYtYml0LCAidTgiIGZvciB1bnNpZ25lZCA4LWJpdCwgImkz MiIgZm9yCiAgIHNpZ25lZCAzMi1iaXQsICJpMTYiIGZvciBzaWduZWQgMTYtYml0IHF1YW50 aXRpZXMsICJmbG9hdCIgZm9yIElFRUUKICAgZmxvYXRpbmcgcG9pbnQgdmFsdWVzLiBUaGUg dHlwZSAidm9pZCIgaXMgdXNlZCB0byBpbmRpY2F0ZSB0aGF0IG5vCiAgIHJldHVybiB2YWx1 ZSBpcyBleHBlY3RlZCBmcm9tIGEgY2FsbC4gUG9pbnRlcnMgYXJlIHJlZmVycmVkIHRvCiAg IHVzaW5nICIqIiBzeW50YXgsIGZvbGxvd2luZyBDIGxhbmd1YWdlIGNvbnZlbnRpb24uCgoJ ICAgV2UgZW1waGFzaXplIHRoYXQgYWxsIHRoZSBBUEkgZnVuY3Rpb25zIGRlc2NyaWJlZCBp biB0aGlzCiAgIGRvY3VtZW50IGFyZSAiYWJzdHJhY3QiIGNhbGxzIGFuZCB0aGF0IGNvbmZv cm1hbnQgQ00KICAgaW1wbGVtZW50YXRpb25zIG1heSBkaWZmZXIgaW4gc3BlY2lmaWMgaW1w bGVtZW50YXRpb24gZGV0YWlscy4KCgozLiAgICAgIEludHJvZHVjdGlvbgoKICAgVGhlIENN IGlzIGFuIGVuZC1zeXN0ZW0gbW9kdWxlIHRoYXQgZW5hYmxlcyBhbiBlbnNlbWJsZSBvZiBt dWx0aXBsZQogICBjb25jdXJyZW50IHN0cmVhbXMgdG8gcGVyZm9ybSBzdGFibGUgY29uZ2Vz dGlvbiBhdm9pZGFuY2UgYW5kCiAgIGNvbnRyb2wsIGFuZCBhbGxvd3MgYXBwbGljYXRpb25z IHRvIGVhc2lseSBhZGFwdCB0aGVpcgogICB0cmFuc21pc3Npb25zIHRvIHByZXZhaWxpbmcg bmV0d29yayBjb25kaXRpb25zLiAgSXQgaW50ZWdyYXRlcwogICBjb25nZXN0aW9uIG1hbmFn ZW1lbnQgYWNyb3NzIGFsbCBhcHBsaWNhdGlvbnMgYW5kIHRyYW5zcG9ydAogICBwcm90b2Nv bHMuICBJdCBtYWludGFpbnMgY29uZ2VzdGlvbiBwYXJhbWV0ZXJzIChhdmFpbGFibGUgYWdn cmVnYXRlCiAgIGFuZCBwZXItc3RyZWFtIGJhbmR3aWR0aCwgcGVyLXJlY2VpdmVyIHJvdW5k LXRyaXAgdGltZXMsIGV0Yy4pIGFuZAogICBleHBvcnRzIGFuIEFQSSB0aGF0IGVuYWJsZXMg YXBwbGljYXRpb25zIHRvIGxlYXJuIGFib3V0IG5ldHdvcmsKICAgY2hhcmFjdGVyaXN0aWNz LCBwYXNzIGluZm9ybWF0aW9uIHRvIHRoZSBDTSwgc2hhcmUgY29uZ2VzdGlvbgogICBpbmZv cm1hdGlvbiB3aXRoIGVhY2ggb3RoZXIsIGFuZCBzY2hlZHVsZSBkYXRhIHRyYW5zbWlzc2lv bnMuICBBbGwKICAgZGF0YSB0cmFuc21pc3Npb25zIE1VU1QgYmUgZG9uZSB3aXRoIHRoZSBl eHBsaWNpdCBjb25zZW50IG9mIHRoZSBDTQogICB2aWEgdGhpcyBBUEkgdG8gZW5zdXJlIHBy b3BlciBjb25nZXN0aW9uIGJlaGF2aW9yLgoKICAgVGhpcyBkb2N1bWVudCBmb2N1c2VzIG9u IGFwcGxpY2F0aW9ucyBhbmQgbmV0d29ya3Mgd2hlcmUgdGhlCiAgIGZvbGxvd2luZyBjb25k aXRpb25zIGhvbGQ6CgogICAxLiBBcHBsaWNhdGlvbnMgYXJlIHdlbGwtYmVoYXZlZCB3aXRo IHRoZWlyIG93biBpbmRlcGVuZGVudAogICAgICBwZXItYnl0ZSBvciBwZXItcGFja2V0IHNl cXVlbmNlIG51bWJlciBpbmZvcm1hdGlvbiwgYW5kIHVzZSB0aGUKICAgICAgQ00gQVBJIHRv IHVwZGF0ZSBpbnRlcm5hbCBzdGF0ZSBpbiB0aGUgQ00uCgogICAyLiBOZXR3b3JrcyBhcmUg YmVzdC1lZmZvcnQgd2l0aG91dCBzZXJ2aWNlIGRpc2NyaW1pbmF0aW9uIG9yCiAgICAgIHJl c2VydmF0aW9ucy4gIEluIHBhcnRpY3VsYXIsIGl0IGRvZXMgbm90IGFkZHJlc3Mgc2l0dWF0 aW9ucwogICAgICB3aGVyZSBkaWZmZXJlbnQgc3RyZWFtcyBiZXR3ZWVuIHRoZSBzYW1lIHBh aXIgb2YgaG9zdHMgdHJhdmVyc2UKICAgICAgcGF0aHMgd2l0aCBkaWZmZXJpbmcgY2hhcmFj dGVyaXN0aWNzLgoKICAgVGhlIENvbmdlc3Rpb24gTWFuYWdlciBmcmFtZXdvcmsgY2FuIGJl IGV4dGVuZGVkIHRvIHN1cHBvcnQKICAgYXBwbGljYXRpb25zIHRoYXQgZG8gbm90IHByb3Zp ZGUgdGhlaXIgb3duIGZlZWRiYWNrIGFuZCB0bwogICBkaWZmZXJlbnRpYWxseS1zZXJ2ZWQg bmV0d29ya3MuICBUaGVzZSBleHRlbnNpb25zIHdpbGwgYmUgYWRkcmVzc2VkCiAgIGluIGxh dGVyIGRvY3VtZW50cy4KCiAgIFRoZSBDTSBpcyBtb3RpdmF0ZWQgYnkgdHdvIG1haW4gZ29h bHM6CgogICAoaSkgRW5hYmxlIGVmZmljaWVudCBtdWx0aXBsZXhpbmcuICBJbmNyZWFzaW5n bHksIHRoZSB0cmVuZCBvbiB0aGUKICAgSW50ZXJuZXQgaXMgZm9yIHVuaWNhc3QgZGF0YSBz ZW5kZXJzIChlLmcuLCBXZWIgc2VydmVycykgdG8KICAgdHJhbnNtaXQgaGV0ZXJvZ2VuZW91 cyB0eXBlcyBvZiBkYXRhIHRvIHJlY2VpdmVycywgcmFuZ2luZyBmcm9tCiAgIHVucmVsaWFi bGUgcmVhbC10aW1lIHN0cmVhbWluZyBjb250ZW50IHRvIHJlbGlhYmxlIFdlYiBwYWdlcyBh bmQKICAgYXBwbGV0cy4gIEFzIGEgcmVzdWx0LCBtYW55IGxvZ2ljYWxseSBkaWZmZXJlbnQg c3RyZWFtcyBzaGFyZSB0aGUKICAgc2FtZSBwYXRoIGJldHdlZW4gc2VuZGVyIGFuZCByZWNl aXZlci4gIEZvciB0aGUgSW50ZXJuZXQgdG8gcmVtYWluCiAgIHN0YWJsZSwgZWFjaCBvZiB0 aGVzZSBzdHJlYW1zIG11c3QgaW5jb3Jwb3JhdGUgY29udHJvbCBwcm90b2NvbHMKICAgdGhh dCBzYWZlbHkgcHJvYmUgZm9yIHNwYXJlIGJhbmR3aWR0aCBhbmQgcmVhY3QgdG8KICAgY29u Z2VzdGlvbi4gVW5mb3J0dW5hdGVseSwgdGhlc2UgY29uY3VycmVudCBzdHJlYW1zIHR5cGlj YWxseSBjb21wZXRlCiAgIHdpdGggZWFjaCBvdGhlciBmb3IgbmV0d29yayByZXNvdXJjZXMs IHJhdGhlciB0aGFuIHNoYXJlIHRoZW0KICAgZWZmZWN0aXZlbHkuIEZ1cnRoZXJtb3JlLCB0 aGV5IGRvIG5vdCBsZWFybiBmcm9tIGVhY2ggb3RoZXIgYWJvdXQKICAgdGhlIHN0YXRlIG9m IHRoZSBuZXR3b3JrLiBFdmVuIGlmIHRoZXkgZWFjaCBpbmRlcGVuZGVudGx5IGltcGxlbWVu dAogICBjb25nZXN0aW9uIGNvbnRyb2wgKGUuZy4sIGEgZ3JvdXAgb2YgVENQIGNvbm5lY3Rp b25zIGVhY2gKICAgaW1wbGVtZW50aW5nIHRoZSBhbGdvcml0aG1zIGluIFtKYWNvYnNvbjg4 LCBBbGxtYW45OV0pLCB0aGUKICAgZW5zZW1ibGUgb2Ygc3RyZWFtcyB0ZW5kcyB0byBiZSBt b3JlIGFnZ3Jlc3NpdmUgaW4gdGhlIGZhY2Ugb2YKICAgY29uZ2VzdGlvbiB0aGFuIGEgc2lu Z2xlIFRDUCBjb25uZWN0aW9uIGltcGxlbWVudGluZyBzdGFuZGFyZCBUQ1AKICAgY29uZ2Vz dGlvbiBjb250cm9sIGFuZCBhdm9pZGFuY2UgW0JhbGFrcmlzaG5hbjk4XS4KCiAgIChpaSkg RW5hYmxlIGFwcGxpY2F0aW9uIGFkYXB0YXRpb24gdG8gY29uZ2VzdGlvbi4gSW5jcmVhc2lu Z2x5CiAgIHBvcHVsYXIgcmVhbC10aW1lIHN0cmVhbWluZyBhcHBsaWNhdGlvbnMgcnVuIG92 ZXIgVURQIHVzaW5nIHRoZWlyCiAgIG93biB1c2VyLWxldmVsIHRyYW5zcG9ydCBwcm90b2Nv bHMgZm9yIGdvb2QgYXBwbGljYXRpb24KICAgcGVyZm9ybWFuY2UsIGJ1dCBpbiBtb3N0IGNh c2VzIHRvZGF5IGRvIG5vdCBhZGFwdCBvciByZWFjdCBwcm9wZXJseQogICB0byBuZXR3b3Jr IGNvbmdlc3Rpb24uICBCeSBpbXBsZW1lbnRpbmcgYSBzdGFibGUgY29udHJvbCBhbGdvcml0 aG0KICAgYW5kIGV4cG9zaW5nIGFuIGFkYXB0YXRpb24gQVBJLCB0aGUgQ00gZW5hYmxlcyBl YXN5IGFwcGxpY2F0aW9uCiAgIGFkYXB0YXRpb24gdG8gY29uZ2VzdGlvbi4gIEFwcGxpY2F0 aW9ucyBhZGFwdCB0aGUgZGF0YSB0aGV5CiAgIHRyYW5zbWl0IHRvIHRoZSBjdXJyZW50IG5l dHdvcmsgY29uZGl0aW9ucy4KCiAgIFRoZSBDTSBmcmFtZXdvcmsgYnVpbGRzIG9uIHJlY2Vu dCB3b3JrIG9uIFRDUCBjb250cm9sIGJsb2NrIHNoYXJpbmcKICAgW1RvdWNoOTddLCBpbnRl Z3JhdGVkIFRDUCBjb25nZXN0aW9uIGNvbnRyb2wgKFRDUC1JbnQpCiAgIFtCYWxha3Jpc2hu YW45OF0gYW5kIFRDUCBzZXNzaW9ucyBbUGFkbWFuYWJoYW45OF0uICBbVG91Y2g5N10KICAg YWR2b2NhdGVzIHRoZSBzaGFyaW5nIG9mIHNvbWUgb2YgdGhlIHN0YXRlIGluIHRoZSBUQ1Ag Y29udHJvbCBibG9jawogICB0byBpbXByb3ZlIHRyYW5zaWVudCB0cmFuc3BvcnQgcGVyZm9y bWFuY2UgYW5kIGRlc2NyaWJlcyBzaGFyaW5nCiAgIGFjcm9zcyBhbiBlbnNlbWJsZSBvZiBU Q1AgY29ubmVjdGlvbnMuICBbQmFsYWtyaXNobmFuOThdLAogICBbUGFkbWFuYWJoYW45OF0s IGFuZCBbRWdnZXJ0MDBdIGRlc2NyaWJlIHNldmVyYWwgZXhwZXJpbWVudHMgdGhhdAogICBx dWFudGlmeSB0aGUgYmVuZWZpdHMgb2Ygc2hhcmluZyBjb25nZXN0aW9uIHN0YXRlLCBpbmNs dWRpbmcKICAgaW1wcm92ZWQgc3RhYmlsaXR5IGluIHRoZSBmYWNlIG9mIGNvbmdlc3Rpb24g YW5kIGJldHRlciBsb3NzCiAgIHJlY292ZXJ5LiAgSW50ZWdyYXRpbmcgbG9zcyByZWNvdmVy eSBhY3Jvc3MgY29uY3VycmVudCBjb25uZWN0aW9ucwogICBzaWduaWZpY2FudGx5IGltcHJv dmVzIHBlcmZvcm1hbmNlIGJlY2F1c2UgbG9zc2VzIG9uIG9uZSBjb25uZWN0aW9uCiAgIGNh biBiZSBkZXRlY3RlZCBieSBub3RpY2luZyB0aGF0IGxhdGVyIGRhdGEgc2VudCBvbiBhbm90 aGVyCiAgIGNvbm5lY3Rpb24gaGFzIGJlZW4gcmVjZWl2ZWQgYW5kIGFja25vd2xlZGdlZC4g IFRoZSBDTSBmcmFtZXdvcmsKICAgZXh0ZW5kcyB0aGVzZSBpZGVhcyBpbiB0d28gc2lnbmlm aWNhbnQgd2F5czogKGkpIGl0IGV4dGVuZHMKICAgY29uZ2VzdGlvbiBtYW5hZ2VtZW50IHRv IG5vbi1UQ1Agc3RyZWFtcywgd2hpY2ggYXJlIGJlY29taW5nCiAgIGluY3JlYXNpbmdseSBj b21tb24gYW5kIG9mdGVuIGRvIG5vdCBpbXBsZW1lbnQgcHJvcGVyIGNvbmdlc3Rpb24KICAg bWFuYWdlbWVudCwgYW5kIChpaSkgaXQgcHJvdmlkZXMgYW4gQVBJIGZvciBhcHBsaWNhdGlv bnMgdG8gYWRhcHQKICAgdGhlaXIgdHJhbnNtaXNzaW9ucyB0byBjdXJyZW50IG5ldHdvcmsg Y29uZGl0aW9ucy4gIEZvciBhbiBleHRlbmRlZAogICBkaXNjdXNzaW9uIG9mIHRoZSBtb3Rp dmF0aW9uIGZvciB0aGUgQ00sIGl0cyBhcmNoaXRlY3R1cmUsIEFQSSwKICAgYW5kIGFsZ29y aXRobXMsIHNlZSBbQmFsYWtyaXNobmFuOTldOyBmb3IgYSBkZXNjcmlwdGlvbiBvZiBhbgog ICBpbXBsZW1lbnRhdGlvbiBhbmQgcGVyZm9ybWFuY2UgcmVzdWx0cywgc2VlIFtBbmRlcnNl bjAwXS4KCiAgIFRoZSByZXN1bHRpbmcgZW5kLWhvc3QgcHJvdG9jb2wgYXJjaGl0ZWN0dXJl IGF0IHRoZSBzZW5kZXIgaXMgc2hvd24KICAgaW4gRmlndXJlIDEuICBUaGUgQ00gaGVscHMg YWNoaWV2ZSBuZXR3b3JrIHN0YWJpbGl0eSBieQogICBpbXBsZW1lbnRpbmcgc3RhYmxlIGNv bmdlc3Rpb24gYXZvaWRhbmNlIGFuZCBjb250cm9sIGFsZ29yaXRobXMKICAgdGhhdCBhcmUg IlRDUC1mcmllbmRseSIgW01haGRhdmk5OF0gYmFzZWQgb24gYWxnb3JpdGhtcyBkZXNjcmli ZWQgaW4KICAgW0FsbG1hbjk5XS4gIEhvd2V2ZXIsIGl0IGRvZXMgbm90IGF0dGVtcHQgdG8g ZW5mb3JjZSBwcm9wZXIKICAgY29uZ2VzdGlvbiBiZWhhdmlvciBmb3IgYWxsIGFwcGxpY2F0 aW9ucyAoYnV0IGl0IGRvZXMgbm90IHByZWNsdWRlCiAgIGEgcG9saWNlciBvbiB0aGUgaG9z dCB0aGF0IHBlcmZvcm1zIHRoaXMgdGFzaykuICBOb3RlIHRoYXQgd2hpbGUKICAgdGhlIHBv bGljZXIgYXQgdGhlIGVuZC1ob3N0IGNhbiB1c2UgQ00sIHRoZSBuZXR3b3JrIGhhcyB0byBi ZQogICBwcm90ZWN0ZWQgYWdhaW5zdCBjb21wcm9taXNlcyB0byB0aGUgQ00gYW5kIHRoZSBw b2xpY2VyIGF0IHRoZSBlbmQKICAgaG9zdHMsIGEgdGFzayB0aGF0IHJlcXVpcmVzIHJvdXRl ciBtYWNoaW5lcnkgW0Zsb3lkOTlhXS4gV2UgZG8gbm90CiAgIGFkZHJlc3MgdGhpcyBpc3N1 ZSBmdXJ0aGVyIGluIHRoaXMgZG9jdW1lbnQuCgwKCiAgIHwtLS0tLS0tLXwgfC0tLS0tLS0t fCB8LS0tLS0tLS18IHwtLS0tLS0tLXwgICAgICAgfC0tLS0tLS0tLS0tLS0tfAogICB8ICBI VFRQICB8IHwgIEZUUCAgIHwgfCAgUlRQIDEgfCB8ICBSVFAgMiB8ICAgICAgIHwgICAgICAg ICAgICAgIHwKICAgfC0tLS0tLS0tfCB8LS0tLS0tLS18IHwtLS0tLS0tLXwgfC0tLS0tLS0t fCAgICAgICB8ICAgICAgICAgICAgICB8CiAgICAgICB8ICAgICAgICAgIHwgICAgICAgICB8 ICBeICAgICAgIHwgIF4gICAgICAgICAgfCAgICAgICAgICAgICAgfAogICAgICAgfCAgICAg ICAgICB8ICAgICAgICAgfCAgfCAgICAgICB8ICB8ICAgICAgICAgIHwgICBTY2hlZHVsZXIg IHwKICAgICAgIHwgICAgICAgICAgfCAgICAgICAgIHwgIHwgICAgICAgfCAgfCAgfC0tLXwg ICB8ICAgICAgICAgICAgICB8CiAgICAgICB8ICAgICAgICAgIHwgICAgICAgICB8ICB8LS0t LS0tLXwtLSstPnwgICB8ICAgfCAgICAgICAgICAgICAgfAogICAgICAgfCAgICAgICAgICB8 ICAgICAgICAgfCAgICAgICAgICB8ICAgICB8ICAgfDwtLXwgICAgICAgICAgICAgIHwKICAg ICAgIHYgICAgICAgICAgdiAgICAgICAgIHYgICAgICAgICAgdiAgICAgfCAgIHwgICB8LS0t LS0tLS0tLS0tLS18CiAgIHwtLS0tLS0tLXwgfC0tLS0tLS0tfCAgfC0tLS0tLS0tLS0tLS18 ICAgIHwgICB8ICAgICAgICAgICBeCiAgIHwgIFRDUCAxIHwgfCAgVENQIDIgfCAgfCAgICBV RFAgMSAgICB8ICAgIHwgQSB8ICAgICAgICAgICB8CiAgIHwtLS0tLS0tLXwgfC0tLS0tLS0t fCAgfC0tLS0tLS0tLS0tLS18ICAgIHwgICB8ICAgICAgICAgICB8CiAgICAgIF4gICB8ICAg ICAgXiAgIHwgICAgICAgICAgICAgIHwgICAgICAgIHwgICB8ICAgfC0tLS0tLS0tLS0tLS0t fAogICAgICB8ICAgfCAgICAgIHwgICB8ICAgICAgICAgICAgICB8ICAgICAgICB8IFAgfC0t PnwgICAgICAgICAgICAgIHwKICAgICAgfCAgIHwgICAgICB8ICAgfCAgICAgICAgICAgICAg fCAgICAgICAgfCAgIHwgICB8ICAgICAgICAgICAgICB8CiAgICAgIHwtLS18LS0tLS0tKy0t LXwtLS0tLS0tLS0tLS0tLXwtLS0tLS0tPnwgICB8ICAgfCAgQ29uZ2VzdGlvbiAgfAogICAg ICAgICAgfCAgICAgICAgICB8ICAgICAgICAgICAgICB8ICAgICAgICB8IEkgfCAgIHwgICAg ICAgICAgICAgIHwKICAgICAgICAgIHYgICAgICAgICAgdiAgICAgICAgICAgICAgdiAgICAg ICAgfCAgIHwgICB8ICBDb250cm9sbGVyICB8CiAgICAgfC0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tfCAgIHwgICB8ICAgfCAgICAgICAgICAgICAgfAogICAgIHwgICAg ICAgICAgICAgICBJUCAgICAgICAgICAgICAgICAgIHwtLT58ICAgfCAgIHwgICAgICAgICAg ICAgIHwKICAgICB8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18ICAgfCAg IHwgICB8LS0tLS0tLS0tLS0tLS18CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIHwtLS18CgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IEZpZ3VyZSAxCgogICBUaGUga2V5IGNvbXBvbmVudHMgb2YgdGhlIENNIGZyYW1ld29yayBh cmUgKGkpIHRoZSBBUEksIChpaSkgdGhlCiAgIGNvbmdlc3Rpb24gY29udHJvbGxlciwgYW5k IChpaWkpIHRoZSBzY2hlZHVsZXIuICBUaGUgQVBJIGlzIChpbgogICBwYXJ0KSBtb3RpdmF0 ZWQgYnkgdGhlIHJlcXVpcmVtZW50cyBvZiBhcHBsaWNhdGlvbi1sZXZlbCBmcmFtaW5nCiAg IChBTEYpIFtDbGFyazkwXSwgYW5kIGlzIGRlc2NyaWJlZCBpbiBTZWN0aW9uIDQuICBUaGUg Q00gaW50ZXJuYWxzCiAgIChTZWN0aW9uIDUpIGluY2x1ZGUgYSBjb25nZXN0aW9uIGNvbnRy b2xsZXIgKFNlY3Rpb24gNS4xKSBhbmQgYQogICBzY2hlZHVsZXIgdG8gb3JjaGVzdHJhdGUg ZGF0YSB0cmFuc21pc3Npb25zIGJldHdlZW4gY29uY3VycmVudAogICBzdHJlYW1zIGluIGEg bWFjcm9mbG93IChTZWN0aW9uIDUuMikuICBUaGUgY29uZ2VzdGlvbiBjb250cm9sbGVyCiAg IGFkanVzdHMgdGhlIGFnZ3JlZ2F0ZSB0cmFuc21pc3Npb24gcmF0ZSBiZXR3ZWVuIHNlbmRl ciBhbmQgcmVjZWl2ZXIKICAgYmFzZWQgb24gaXRzIGVzdGltYXRlIG9mIGNvbmdlc3Rpb24g aW4gdGhlIG5ldHdvcmsuICBJdCBvYnRhaW5zCiAgIGZlZWRiYWNrIGFib3V0IGl0cyBwYXN0 IHRyYW5zbWlzc2lvbnMgZnJvbSBhcHBsaWNhdGlvbnMgdGhlbXNlbHZlcwogICB2aWEgdGhl IEFQSS4gIFRoZSBzY2hlZHVsZXIgYXBwb3J0aW9ucyBhdmFpbGFibGUgYmFuZHdpZHRoIGFt b25nc3QKICAgdGhlIGRpZmZlcmVudCBzdHJlYW1zIHdpdGhpbiBlYWNoIG1hY3JvZmxvdyBh bmQgbm90aWZpZXMKICAgYXBwbGljYXRpb25zIHdoZW4gdGhleSBhcmUgcGVybWl0dGVkIHRv IHNlbmQgZGF0YS4gIFRoaXMgZG9jdW1lbnQKICAgZm9jdXNlcyBvbiB3ZWxsLWJlaGF2ZWQg YXBwbGljYXRpb25zOyBhIGZ1dHVyZSBvbmUgd2lsbCBkZXNjcmliZQogICB0aGUgc2VuZGVy LXJlY2VpdmVyIHByb3RvY29sIGFuZCBoZWFkZXIgZm9ybWF0cyB0aGF0IHdpbGwgaGFuZGxl CiAgIGFwcGxpY2F0aW9ucyB0aGF0IGRvIG5vdCBpbmNvcnBvcmF0ZSB0aGVpciBvd24gZmVl ZGJhY2sgdG8gdGhlIENNLgoKNC4gICAgICBDTSBBUEkKCiAgIFVzaW5nIHRoZSBDTSBBUEks IHN0cmVhbXMgY2FuIGRldGVybWluZSB0aGVpciBzaGFyZSBvZiB0aGUgYXZhaWxhYmxlCiAg IGJhbmR3aWR0aCwgcmVxdWVzdCBhbmQgaGF2ZSB0aGVpciBkYXRhIHRyYW5zbWlzc2lvbnMg c2NoZWR1bGVkLAogICBpbmZvcm0gdGhlIENNIGFib3V0IHN1Y2Nlc3NmdWwgdHJhbnNtaXNz aW9ucywgYW5kIGJlIGluZm9ybWVkIHdoZW4KICAgdGhlIENNJ3MgZXN0aW1hdGUgb2YgcGF0 aCBiYW5kd2lkdGggY2hhbmdlcy4gVGh1cywgdGhlIENNIGZyZWVzCiAgIGFwcGxpY2F0aW9u cyBmcm9tIGhhdmluZyB0byBtYWludGFpbiBpbmZvcm1hdGlvbiBhYm91dCB0aGUgc3RhdGUg b2YKICAgY29uZ2VzdGlvbiBhbmQgYXZhaWxhYmxlIGJhbmR3aWR0aCBhbG9uZyBhbnkgcGF0 aC4KCiAgIFRoZSBmdW5jdGlvbiBwcm90b3R5cGVzIGJlbG93IGZvbGxvdyBzdGFuZGFyZCBD IGxhbmd1YWdlCiAgIGNvbnZlbnRpb24uICBXZSBlbXBoYXNpemUgdGhhdCB0aGVzZSBBUEkg ZnVuY3Rpb25zIGFyZSBhYnN0cmFjdAogICBjYWxscyBhbmQgY29uZm9ybWFudCBDTSBpbXBs ZW1lbnRhdGlvbnMgbWF5IGRpZmZlciBpbiBzcGVjaWZpYwogICBkZXRhaWxzLCBhcyBsb25n IGFzIGVxdWl2YWxlbnQgZnVuY3Rpb25hbGl0eSBpcyBwcm92aWRlZC4KCiAgIFdoZW4gYSBu ZXcgc3RyZWFtIGlzIGNyZWF0ZWQgYnkgYW4gYXBwbGljYXRpb24sIGl0IHBhc3NlcyBzb21l CiAgIGluZm9ybWF0aW9uIHRvIHRoZSBDTSB2aWEgdGhlIGNtX29wZW4oc3RyZWFtX2luZm8p IEFQSSBjYWxsLgogICBDdXJyZW50bHksIHN0cmVhbV9pbmZvIGNvbnNpc3RzIG9mIHRoZSBm b2xsb3dpbmcgaW5mb3JtYXRpb246IChpKQogICB0aGUgc291cmNlIElQIGFkZHJlc3MsIChp aSkgdGhlIHNvdXJjZSBwb3J0LCAoaWlpKSB0aGUgZGVzdGluYXRpb24KICAgSVAgYWRkcmVz cywgKGl2KSB0aGUgZGVzdGluYXRpb24gcG9ydCwgYW5kICh2KSB0aGUgSVAgcHJvdG9jb2wK ICAgbnVtYmVyLgogICAKICAgNC4xIFN0YXRlIG1haW50ZW5hbmNlCgogICAxLiBPcGVuOiBB bGwgYXBwbGljYXRpb25zIE1VU1QgY2FsbCBjbV9vcGVuKHN0cmVhbV9pbmZvKSBiZWZvcmUK ICAgICAgdXNpbmcgdGhlIENNIEFQSS4gIFRoaXMgcmV0dXJucyBhIGhhbmRsZSwgY21fc3Ry ZWFtaWQsIGZvciB0aGUKICAgICAgYXBwbGljYXRpb24gdG8gdXNlIGZvciBhbGwgZnVydGhl ciBDTSBBUEkgaW52b2NhdGlvbnMgZm9yIHRoYXQKICAgICAgc3RyZWFtLiAgSWYgdGhlIHJl dHVybmVkIGNtX3N0cmVhbWlkIGlzIC0xLCB0aGVuIHRoZSBjbV9vcGVuKCkKICAgICAgZmFp bGVkIGFuZCB0aGF0IHN0cmVhbSBjYW5ub3QgdXNlIHRoZSBDTS4KCiAgICAgIEFsbCBvdGhl ciBjYWxscyB0byB0aGUgQ00gZm9yIGEgc3RyZWFtIHVzZSB0aGUgY21fc3RyZWFtaWQKICAg ICAgcmV0dXJuZWQgZnJvbSB0aGUgY21fb3BlbigpIGNhbGwuCgogICAyLiBDbG9zZTogV2hl biBhIHN0cmVhbSB0ZXJtaW5hdGVzLCB0aGUgYXBwbGljYXRpb24gU0hPVUxEIGludm9rZQog ICAgICBjbV9jbG9zZShjbV9zdHJlYW1pZCkgdG8gaW5mb3JtIHRoZSBDTSBhYm91dCB0aGUg dGVybWluYXRpb24KICAgICAgb2YgdGhlIHN0cmVhbS4KCiAgIDMuIFBhY2tldCBzaXplOiBj bV9tdHUoY21fc3RyZWFtaWQpIHJldHVybnMgdGhlIGVzdGltYXRlZCBQTVRVIG9mCiAgICAg IHRoZSBwYXRoIGJldHdlZW4gc2VuZGVyIGFuZCByZWNlaXZlci4gIEludGVybmFsbHksIHRo aXMKICAgICAgaW5mb3JtYXRpb24gU0hPVUxEIGJlIG9idGFpbmVkIHZpYSBwYXRoIE1UVSBk aXNjb3ZlcnkKICAgICAgW01vZ3VsOTBdLiAgSXQgTUFZIGJlIHN0YXRpY2FsbHkgY29uZmln dXJlZCBpbiB0aGUgYWJzZW5jZSBvZgogICAgICBzdWNoIGEgbWVjaGFuaXNtLgoKICAgNC4y IERhdGEgdHJhbnNtaXNzaW9uCgogICBUaGUgQ00gYWNjb21tb2RhdGVzIHR3byB0eXBlcyBv ZiBhZGFwdGl2ZSBzZW5kZXJzLCBlbmFibGluZwogICBhcHBsaWNhdGlvbnMgdG8gZHluYW1p Y2FsbHkgYWRhcHQgdGhlaXIgY29udGVudCBiYXNlZCBvbgogICBwcmV2YWlsaW5nIG5ldHdv cmsgY29uZGl0aW9ucywgYW5kIHN1cHBvcnRpbmcgQUxGLWJhc2VkCiAgIGFwcGxpY2F0aW9u cy4gCgogICAxLiBDYWxsYmFjay1iYXNlZCB0cmFuc21pc3Npb24uIFRoZSBjYWxsYmFjay1i YXNlZCB0cmFuc21pc3Npb24gQVBJCiAgIHB1dHMgdGhlIHN0cmVhbSBpbiBmaXJtIGNvbnRy b2wgb2YgZGVjaWRpbmcgd2hhdCB0byB0cmFuc21pdCBhdAogICBlYWNoIHBvaW50IGluIHRp bWUuIFRvIGFjaGlldmUgdGhpcywgdGhlIENNIGRvZXMgbm90IGJ1ZmZlciBhbnkKICAgZGF0 YTsgaW5zdGVhZCwgaXQgYWxsb3dzIHN0cmVhbXMgdGhlIG9wcG9ydHVuaXR5IHRvIGFkYXB0 IHRvCiAgIHVuZXhwZWN0ZWQgbmV0d29yayBjaGFuZ2VzIGF0IHRoZSBsYXN0IHBvc3NpYmxl IGluc3RhbnQuICBUaHVzLAogICB0aGlzIGVuYWJsZXMgc3RyZWFtcyB0byAicHVsbCBvdXQi IGFuZCByZXBhY2tldGl6ZSBkYXRhIHVwb24KICAgbGVhcm5pbmcgYWJvdXQgYW55IHJhdGUg Y2hhbmdlLCB3aGljaCBpcyBoYXJkIHRvIGRvIG9uY2UgdGhlIGRhdGEKICAgaGFzIGJlZW4g YnVmZmVyZWQuICBUaGUgQ00gbXVzdCBpbXBsZW1lbnQgYSBjbV9yZXF1ZXN0KGkzMgogICBj bV9zdHJlYW1pZCkgY2FsbCBmb3Igc3RyZWFtcyB3aXNoaW5nIHRvIHNlbmQgZGF0YSBpbiB0 aGlzIHN0eWxlLgogICBBZnRlciBzb21lIHRpbWUsIGRlcGVuZGluZyBvbiB0aGUgcmF0ZSwg dGhlIENNIE1VU1QgCiAgIGludm9rZSBhIGNhbGxiYWNrIHVzaW5nIGNtYXBwX3NlbmQoKSwg d2hpY2ggaXMKICAgYSBncmFudCBmb3IgdGhlIHN0cmVhbSB0byBzZW5kIHVwIHRvIFBNVFUg Ynl0ZXMuICBUaGUKICAgY2FsbGJhY2stc3R5bGUgQVBJIGlzIHRoZSByZWNvbW1lbmRlZCBj aG9pY2UgZm9yIEFMRi1iYXNlZCBzdHJlYW1zLgogICBOb3RlIHRoYXQgY21fcmVxdWVzdCgp IGRvZXMgbm90IHRha2UgdGhlIG51bWJlciBvZiBieXRlcyBvcgogICBNVFUtc2l6ZWQgdW5p dHMgYXMgYW4gYXJndW1lbnQ7IGVhY2ggY2FsbCB0byBjbV9yZXF1ZXN0KCkgaXMgYW4KICAg aW1wbGljaXQgcmVxdWVzdCBmb3Igc2VuZGluZyB1cCB0byBQTVRVIGJ5dGVzLiBUaGUgQ00g TUFZIHByb3ZpZGUKICAgYW4gYWx0ZXJuYXRlIGludGVyZmFjZSwgY21fcmVxdWVzdChpbnQg aykuIFRoZSBjbWFwcF9zZW5kIGNhbGxiYWNrCiAgIGZvciB0aGlzIHJlcXVlc3QgaXMgZ3Jh bnRlZCB0aGUgcmlnaHQgdG8gc2VuZCB1cCB0byBrIFBNVFUgc2l6ZWQKICAgc2VnbWVudHMu ICBTZWN0aW9uIDQuMyBkaXNjdXNzZXMgdGhlIHRpbWUgZHVyYXRpb24gZm9yIHdoaWNoIHRo ZSAKICAgdHJhbnNtaXNzaW9uIGdyYW50IGlzIHZhbGlkLCB3aGlsZSBTZWN0aW9uIDUuMiBk ZXNjcmliZXMgaG93IHRoZXNlIAogICByZXF1ZXN0cyBhcmUgc2NoZWR1bGVkIGFuZCBjYWxs YmFja3MgbWFkZS4KCiAgIDIuIFN5bmNocm9ub3VzLXN0eWxlLiAgVGhlIGFib3ZlIGNhbGxi YWNrLWJhc2VkIEFQSSBhY2NvbW1vZGF0ZXMgYQogICBjbGFzcyBvZiBBTEYgc3RyZWFtcyB0 aGF0IGFyZSAiYXN5bmNocm9ub3VzLiIgIEFzeW5jaHJvbm91cwogICB0cmFuc21pdHRlcnMg ZG8gbm90IHRyYW5zbWl0IGJhc2VkIG9uIGEgcGVyaW9kaWMgY2xvY2ssIGJ1dCBkbyBzbwog ICB0cmlnZ2VyZWQgYnkgYXN5bmNocm9ub3VzIGV2ZW50cyBsaWtlIGZpbGUgcmVhZHMgb3Ig Y2FwdHVyZWQKICAgZnJhbWVzLiAgT24gdGhlIG90aGVyIGhhbmQsIHRoZXJlIGFyZSBtYW55 IHN0cmVhbXMgdGhhdCBhcmUKICAgInN5bmNocm9ub3VzIiB0cmFuc21pdHRlcnMsIHdoaWNo IHRyYW5zbWl0IHBlcmlvZGljYWxseSBiYXNlZCBvbgogICB0aGVpciBvd24gaW50ZXJuYWwg dGltZXJzIChlLmcuLCBhbiBhdWRpbyBzZW5kZXJzIHRoYXQgc2VuZHMgYXQgYQogICBjb25z dGFudCBzYW1wbGluZyByYXRlKS4gIFdoaWxlIENNIGNhbGxiYWNrcyBjb3VsZCBiZSBjb25m aWd1cmVkIHRvCiAgIHBlcmlvZGljYWxseSBpbnRlcnJ1cHQgc3VjaCB0cmFuc21pdHRlcnMs IHRoZSB0cmFuc21pdCBsb29wIG9mIHN1Y2gKICAgYXBwbGljYXRpb25zIGlzIGxlc3MgYWZm ZWN0ZWQgaWYgdGhleSByZXRhaW4gdGhlaXIgb3JpZ2luYWwKICAgdGltZXItYmFzZWQgbG9v cC4gIEluIGFkZGl0aW9uLCBpdCBjb21wbGljYXRlcyB0aGUgQ00gQVBJIHRvIGhhdmUgYQog ICBzdHJlYW0gZXhwcmVzcyB0aGUgcGVyaW9kaWNpdHkgYW5kIGdyYW51bGFyaXR5IG9mIGl0 cyBjYWxsYmFja3MuCiAgIFRodXMsIHRoZSBDTSBNVVNUIGV4cG9ydCBhbiBBUEkgdGhhdCBh bGxvd3Mgc3VjaCBzdHJlYW1zIHRvIGJlIGluZm9ybWVkCiAgIG9mIGNoYW5nZXMgaW4gcmF0 ZXMgdXNpbmcgdGhlIGNtYXBwX3VwZGF0ZSh1NjQgbmV3cmF0ZSwgdTMyIHNydHQsCiAgIHUz MiBydHRkZXYpIGNhbGxiYWNrIGZ1bmN0aW9uLCB3aGVyZSBuZXdyYXRlIGlzIHRoZSBuZXcg cmF0ZSBpbgogICBiaXRzIHBlciBzZWNvbmQgZm9yIHRoaXMgc3RyZWFtLCBzcnR0IGlzIHRo ZSBjdXJyZW50IHNtb290aGVkIHJvdW5kCiAgIHRyaXAgdGltZSBlc3RpbWF0ZSBpbiBtaWNy b3NlY29uZHMsIGFuZCBydHRkZXYgaXMgdGhlIHNtb290aGVkCiAgIGxpbmVhciBkZXZpYXRp b24gaW4gdGhlIHJvdW5kLXRyaXAgdGltZSBlc3RpbWF0ZSBjYWxjdWxhdGVkIHVzaW5nCiAg IHRoZSBzYW1lIGFsZ29yaXRobSBhcyBpbiBUQ1AgW1BheHNvbjAwXS4gIFRoZSBuZXdyYXRl IHZhbHVlIHJlcG9ydHMKICAgYW4gaW5zdGFudGFuZW91cyByYXRlIGNhbGN1bGF0ZWQsIGZv ciBleGFtcGxlLCBieSB0YWtpbmcgdGhlIHJhdGlvCiAgIG9mIGN3bmQgYW5kIHNydHQsIGFu ZCBkaXZpZGluZyBieSB0aGUgZnJhY3Rpb24gb2YgdGhhdCByYXRpbwogICBhbGxvY2F0ZWQg dG8gdGhlIHN0cmVhbS4gIEluIHJlc3BvbnNlLCB0aGUgc3RyZWFtIE1VU1QgYWRhcHQgaXRz CiAgIHBhY2tldCBzaXplIG9yIGNoYW5nZSBpdHMgdGltZXIgaW50ZXJ2YWwgdG8gY29uZm9y bSB0byAoaS5lLiwgbm90CiAgIGV4Y2VlZCkgdGhlIGFsbG93ZWQgcmF0ZS4gIE9mIGNvdXJz ZSwgaXQgbWF5IGNob29zZSBub3QgdG8gdXNlIGFsbAogICBvZiB0aGlzIHJhdGUuICBOb3Rl IHRoYXQgdGhlIENNIGlzIG5vdCBvbiB0aGUgZGF0YSBwYXRoIG9mIHRoZQogICBhY3R1YWwg dHJhbnNtaXNzaW9uLgoKICAgVG8gYXZvaWQgdW5uZWNlc3NhcnkgY21hcHBfdXBkYXRlKCkg Y2FsbGJhY2tzIHRoYXQgdGhlIGFwcGxpY2F0aW9uCiAgIHdpbGwgb25seSBpZ25vcmUsIHRo ZSBDTSBNVVNUIHByb3ZpZGUgYSBjbV90aHJlc2goZmxvYXQKICAgcmF0ZV9kb3dudGhyZXNo LCBmbG9hdCByYXRlX3VwdGhyZXNoLCBmbG9hdCBydHRfZG93bnRocmVzaCwgZmxvYXQKICAg cnR0X3VwdGhyZXNoKSBmdW5jdGlvbiB0aGF0IGEgc3RyZWFtIGNhbiB1c2UgYXQgYW55IHN0 YWdlIGluIGl0cyBleGVjdXRpb24uCiAgIEluIHJlc3BvbnNlLCB0aGUgQ00gU0hPVUxEIGlu dm9rZSB0aGUgY2FsbGJhY2sgb25seSB3aGVuIHRoZSByYXRlIGRlY3JlYXNlcyAKICAgdG8g bGVzcyB0aGFuIChyYXRlX2Rvd250aHJlc2ggKiBsYXN0cmF0ZSkgb3IgaW5jcmVhc2VzIHRv IG1vcmUgdGhhbgogICAocmF0ZV91cHRocmVzaCAqIGxhc3RyYXRlKSwgd2hlcmUgbGFzdHJh dGUgaXMgdGhlIHJhdGUgbGFzdAogICBub3RpZmllZCB0byB0aGUgc3RyZWFtLCBvciB3aGVu IHRoZSByb3VuZC10cmlwIHRpbWUgY2hhbmdlcwogICBjb3JyZXNwb25kaW5nbHkgYnkgdGhl IHJlcXVpc2l0ZSB0aHJlc2hvbGRzLiAgVGhpcyBpbmZvcm1hdGlvbiBpcwogICB1c2VkIGFz IGEgaGludCBieSB0aGUgQ00sIGluIHRoZSBzZW5zZSB0aGUgY21hcHBfdXBkYXRlKCkgY2Fu IGJlCiAgIGNhbGxlZCBldmVuIGlmIHRoZXNlIGNvbmRpdGlvbnMgYXJlIG5vdCBtZXQuCgog ICBUaGUgQ00gTVVTVCBpbXBsZW1lbnQgYSBjbV9xdWVyeShpMzIgY21fc3RyZWFtaWQsIHU2 NCogcmF0ZSwgCiAgIHUzMiogc3J0dCwgdTMyKiBydHRkZXYpIHRvIGFsbG93IGFuIGFwcGxp Y2F0aW9uIHRvIHF1ZXJ5IAogICB0aGUgY3VycmVudCBDTSBzdGF0ZS4gIFRoaXMgc2V0cyB0 aGUgcmF0ZSB2YXJpYWJsZSB0byAKICAgdGhlIGN1cnJlbnQgcmF0ZSBlc3RpbWF0ZSBpbiBi aXRzIHBlciBzZWNvbmQsIHRoZQogICBzcnR0IHZhcmlhYmxlIHRvIHRoZSBjdXJyZW50IHNt b290aGVkIHJvdW5kLXRyaXAgdGltZSBlc3RpbWF0ZSBpbgogICBtaWNyb3NlY29uZHMsIGFu ZCBydHRkZXYgdG8gdGhlIG1lYW4gbGluZWFyIGRldmlhdGlvbi4gIElmIHRoZSBDTQogICBk b2VzIG5vdCBoYXZlIHZhbGlkIGVzdGltYXRlcyBmb3IgdGhlIG1hY3JvZmxvdywgaXQgZmls bHMgaW4KICAgbmVnYXRpdmUgdmFsdWVzIGZvciB0aGUgcmF0ZSwgc3J0dCwgYW5kIHJ0dGRl di4KCiAgIE5vdGUgdGhhdCBhIHN0cmVhbSBjYW4gdXNlIG1vcmUgdGhhbiBvbmUgb2YgdGhl IGFib3ZlIHRyYW5zbWlzc2lvbgogICBBUElzIGF0IHRoZSBzYW1lIHRpbWUuICBJbiBwYXJ0 aWN1bGFyLCB0aGUga25vd2xlZGdlIG9mIHN1c3RhaW5hYmxlCiAgIHJhdGUgaXMgdXNlZnVs IGZvciBhc3luY2hyb25vdXMgc3RyZWFtcyBhcyB3ZWxsIGFzIHN5bmNocm9ub3VzCiAgIG9u ZXM7IGUuZy4sIGFuIGFzeW5jaHJvbm91cyBXZWIgc2VydmVyIGRpc3NlbWluYXRpbmcgaW1h Z2VzIHVzaW5nCiAgIFRDUCBtYXkgdXNlIGNtYXBwX3NlbmQoKSB0byBzY2hlZHVsZSBpdHMg dHJhbnNtaXNzaW9ucyBhbmQKICAgY21hcHBfdXBkYXRlKCkgdG8gZGVjaWRlIHdoZXRoZXIg dG8gc2VuZCBhIGxvdy1yZXNvbHV0aW9uIG9yCiAgIGhpZ2gtcmVzb2x1dGlvbiBpbWFnZS4g IEEgVENQIGltcGxlbWVudGF0aW9uIHVzaW5nIHRoZSBDTSBpcwogICBkZXNjcmliZWQgaW4g U2VjdGlvbiA2LjEuMSwgd2hlcmUgdGhlIGJlbmVmaXQgb2YgdGhlIGNtX3JlcXVlc3QoKQog ICBjYWxsYmFjayBBUEkgZm9yIFRDUCB3aWxsIGJlY29tZSBhcHBhcmVudC4KCiAgIFRoZSBy ZWFkZXIgd2lsbCBub3RpY2UgdGhhdCB0aGUgYmFzaWMgQ00gQVBJIGRvZXMgbm90IHByb3Zp ZGUgYW4KICAgaW50ZXJmYWNlIGZvciBidWZmZXJlZCBjb25nZXN0aW9uLWNvbnRyb2xsZWQg dHJhbnNtaXNzaW9ucy4gIFRoaXMKICAgaXMgaW50ZW50aW9uYWwsIHNpbmNlIHRoaXMgdHJh bnNtaXNzaW9uIG1vZGUgY2FuIGJlIGltcGxlbWVudGVkCiAgIHVzaW5nIHRoZSBjYWxsYmFj ay1iYXNlZCBwcmltaXRpdmUuICBTZWN0aW9uIDYuMS4yIGRlc2NyaWJlcyBob3cKICAgY29u Z2VzdGlvbi1jb250cm9sbGVkIFVEUCBzb2NrZXRzIG1heSBiZSBpbXBsZW1lbnRlZCB1c2lu ZyB0aGUgQ00KICAgQVBJLgoKICAgNC4zIEFwcGxpY2F0aW9uIG5vdGlmaWNhdGlvbgoKICAg V2hlbiBhIHN0cmVhbSByZWNlaXZlcyBmZWVkYmFjayBmcm9tIHJlY2VpdmVycywgaXQgTVVT VCB1c2UKICAgY21fdXBkYXRlKGkzMiBjbV9zdHJlYW1pZCwgdTMyIG5yZWNkLCB1MzIgbmxv c3QsIHU4IGxvc3Ntb2RlLCBpMzIKICAgcnR0KSB0byBpbmZvcm0gdGhlIENNIGFib3V0IGV2 ZW50cyBzdWNoIGFzIGNvbmdlc3Rpb24gbG9zc2VzLAogICBzdWNjZXNzZnVsIHJlY2VwdGlv bnMsIHR5cGUgb2YgbG9zcyAodGltZW91dCBldmVudCwgRXhwbGljaXQKICAgQ29uZ2VzdGlv biBOb3RpZmljYXRpb24gW1JhbWFrcmlzaG5hbjk4XSwgZXRjLikgYW5kIHJvdW5kLXRyaXAg dGltZQogICBzYW1wbGVzLiAgVGhlIG5yZWNkIHBhcmFtZXRlciBpbmRpY2F0ZXMgaG93IG1h bnkgYnl0ZXMgd2VyZQogICBzdWNjZXNzZnVsbHkgcmVjZWl2ZWQgYnkgdGhlIHJlY2VpdmVy IHNpbmNlIHRoZSBsYXN0IGNtX3VwZGF0ZQogICBjYWxsLCB3aGlsZSB0aGUgbnJlY2QgcGFy YW1ldGVyIGlkZW50aWZpZXMgaG93IG1hbnkgYnl0ZXMgd2VyZQogICByZWNlaXZlZCB3ZXJl IGxvc3QgZHVyaW5nIHRoZSBzYW1lIHRpbWUgcGVyaW9kLiBUaGUgcnR0IHZhbHVlCiAgIGlu ZGljYXRlcyB0aGUgcm91bmQtdHJpcCB0aW1lIG1lYXN1cmVkIGR1cmluZyB0aGUgdHJhbnNt aXNzaW9uIG9mCiAgIHRoZXNlIGJ5dGVzLiAgVGhlIHJ0dCB2YWx1ZSBtdXN0IGJlIHNldCB0 byAtMSBpZiBubyB2YWxpZAogICByb3VuZC10cmlwIHNhbXBsZSB3YXMgb2J0YWluZWQgYnkg dGhlIGFwcGxpY2F0aW9uLiAgVGhlIGxvc3Ntb2RlCiAgIHBhcmFtZXRlciBwcm92aWRlcyBh biBpbmRpY2F0b3Igb2YgaG93IGEgbG9zcyB3YXMgZGV0ZWN0ZWQuICBBCiAgIHZhbHVlIG9m IENNX05PX0ZFRURCQUNLIGluZGljYXRlcyB0aGF0IHRoZSBhcHBsaWNhdGlvbiBoYXMgcmVj ZWl2ZWQKICAgbm8gZmVlZGJhY2sgZm9yIGFsbCBpdHMgb3V0c3RhbmRpbmcgZGF0YSwgYW5k IGlzIHJlcG9ydGluZyB0aGlzIHRvCiAgIHRoZSBDTS4gIEZvciBleGFtcGxlLCBhIFRDUCB0 aGF0IGhhcyBleHBlcmllbmNlZCBhIHRpbWVvdXQgd291bGQKICAgdXNlIHRoaXMgcGFyYW1l dGVyIHRvIGluZm9ybSB0aGUgQ00gb2YgdGhpcy4gIEEgdmFsdWUgb2YKICAgQ01fTE9TU19G RUVEQkFDSyBpbmRpY2F0ZXMgdGhhdCB0aGUgYXBwbGljYXRpb24gaGFzIGV4cGVyaWVuY2Vk CiAgIHNvbWUgbG9zcywgd2hpY2ggaXQgYmVsaWV2ZXMgdG8gYmUgZHVlIHRvIGNvbmdlc3Rp b24sIGJ1dCBub3QgYWxsCiAgIG91dHN0YW5kaW5nIGRhdGEgaGFzIGJlZW4gbG9zdC4gIEZv ciBleGFtcGxlLCBhIFRDUCBzZWdtZW50IGxvc3MKICAgZGV0ZWN0ZWQgdXNpbmcgZHVwbGlj YXRlIChzZWxlY3RpdmUpIGFja25vd2xlZGdlbWVudHMgb3Igb3RoZXIKICAgZGF0YS1kcml2 ZW4gdGVjaG5pcXVlcyBmaXRzIHRoaXMgY2F0ZWdvcnkuICBBIHZhbHVlIG9mCiAgIENNX0VY UExJQ0lUX0NPTkdFU1RJT04gaW5kaWNhdGVzIHRoYXQgdGhlIHJlY2VpdmVyIGVjaG9lZCBh bgogICBleHBsaWNpdCBjb25nZXN0aW9uIG5vdGlmaWNhdGlvbiBtZXNzYWdlLiAgRmluYWxs eSwgYSB2YWx1ZSBvZgogICBDTV9OT19DT05HRVNUSU9OIGluZGljYXRlcyB0aGF0IG5vIGNv bmdlc3Rpb24tcmVsYXRlZCBsb3NzIGhhcwogICBvY2N1cnJlZC4gIFRoZSBsb3NzbW9kZSBw YXJhbWV0ZXIgTVVTVCBiZSByZXBvcnRlZCBhcyBhIGJpdC12ZWN0b3IKICAgd2hlcmUgdGhl IGJpdHMgY29ycmVzcG9uZCB0byBDTV9OT19GRUVEQkFDSywgQ01fTE9TU19GRUVEQkFDSywK ICAgQ01fRVhQTElDSVRfQ09OR0VTVElPTiwgYW5kIENNX05PX0NPTkdFU1RJT04uICBOb3Rl IHRoYXQgb3ZlciBsaW5rcwogICAocGF0aHMpIHRoYXQgZXhwZXJpZW5jZSBsb3NzZXMgZm9y IHJlYXNvbnMgb3RoZXIgdGhhbiBjb25nZXN0aW9uLAogICBhbiBhcHBsaWNhdGlvbiBTSE9V TEQgaW5mb3JtIHRoZSBDTSBvZiBsb3NzZXMsIHdpdGggdGhlCiAgIENNX05PX0NPTkdFU1RJ T04gZmllbGQgc2V0LgoKICAgY21fbm90aWZ5KGkzMiBjbV9zdHJlYW1pZCwgdTMyIG5zZW50 KSBNVVNUIGJlIGNhbGxlZCB3aGVuIGRhdGEgaXMKICAgdHJhbnNtaXR0ZWQgZnJvbSB0aGUg aG9zdCAoZS5nLiwgaW4gdGhlIElQIG91dHB1dCByb3V0aW5lKSB0bwogICBpbmZvcm0gdGhl IENNIHRoYXQgbnNlbnQgYnl0ZXMgd2VyZSBqdXN0IHRyYW5zbWl0dGVkIG9uIGEgZ2l2ZW4K ICAgc3RyZWFtLiAgVGhpcyBhbGxvd3MgdGhlIENNIHRvIHVwZGF0ZSBpdHMgZXN0aW1hdGUg b2YgdGhlIG51bWJlciBvZgogICBvdXRzdGFuZGluZyBieXRlcyBmb3IgdGhlIG1hY3JvZmxv dyBhbmQgZm9yIHRoZSBzdHJlYW0uICAKCiAgIEEgY21hcHBfc2VuZCgpIGdyYW50IGZyb20g dGhlIENNIHRvIGFuIGFwcGxpY2F0aW9uIGlzIHZhbGlkIG9ubHkKICAgZm9yIGFuIGV4cGly YXRpb24gdGltZSwgZXF1YWwgdG8gdGhlIGxhcmdlciBvZiB0aGUgcm91bmQtdHJpcCB0aW1l CiAgIGFuZCBhbiBpbXBsZW1lbnRhdGlvbi1kZXBlbmRlbnQgdGhyZXNob2xkIGNvbW11bmlj YXRlZCBhcyBhbgogICBhcmd1bWVudCB0byB0aGUgY21hcHBfc2VuZCgpIGNhbGxiYWNrIGZ1 bmN0aW9uLiAgVGhlIGFwcGxpY2F0aW9uCiAgIE1VU1QgTk9UIHNlbmQgZGF0YSBiYXNlZCBv biB0aGlzIGNhbGxiYWNrIGFmdGVyIHRoaXMgdGltZSBoYXMKICAgZXhwaXJlZC4gIEZ1cnRo ZXJtb3JlLCBpZiB0aGUgYXBwbGljYXRpb24gZGVjaWRlcyBub3QgdG8gc2VuZCBkYXRhCiAg IGFmdGVyIHJlY2VpdmluZyB0aGlzIGNhbGxiYWNrLCBpdCBTSE9VTEQgY2FsbAogICBjbV9u b3RpZnkoc3RyZWFtX2luZm8sIDApIHRvIGFsbG93IHRoZSBDTSB0byBwZXJtaXQgb3RoZXIg c3RyZWFtcwogICBpbiB0aGUgbWFjcm9mbG93IHRvIHRyYW5zbWl0IGRhdGEuICBUaGUgQ00g Y29uZ2VzdGlvbiBjb250cm9sbGVyCiAgIE1VU1QgYmUgcm9idXN0IHRvIGFwcGxpY2F0aW9u cyBmb3JnZXR0aW5nIHRvIGludm9rZQogICBjbV9ub3RpZnkoc3RyZWFtX2luZm8sIDApIGNv cnJlY3RseSwgb3IgYXBwbGljYXRpb25zIHRoYXQgY3Jhc2ggb3IKICAgZGlzYXBwZWFyIGFm dGVyIGhhdmluZyBtYWRlIGEgY21fcmVxdWVzdCgpIGNhbGwuCgogICA0LjQgUXVlcnlpbmcK CiAgIElmIGFwcGxpY2F0aW9ucyB3aXNoIHRvIGxlYXJuIGFib3V0IHBlci1zdHJlYW0gYXZh aWxhYmxlIGJhbmR3aWR0aAogICBhbmQgcm91bmQtdHJpcCB0aW1lLCB0aGV5IGNhbiB1c2Ug dGhlIENNJ3MgY21fcXVlcnkoaTMyCiAgIGNtX3N0cmVhbWlkLCBpNjQqIHJhdGUsIGkzMiog c3J0dCwgaTMyKiBydHRkZXYpIGNhbGwsIHdoaWNoIGZpbGxzCiAgIGluIHRoZSBkZXNpcmVk IHF1YW50aXRpZXMuICBJZiB0aGUgQ00gZG9lcyBub3QgaGF2ZSB2YWxpZCBlc3RpbWF0ZXMK ICAgZm9yIHRoZSBtYWNyb2Zsb3csIGl0IGZpbGxzIGluIG5lZ2F0aXZlIHZhbHVlcyBmb3Ig dGhlIHJhdGUsIHNydHQsCiAgIGFuZCBydHRkZXYuCgogICA0LjUgU2hhcmluZyBncmFudWxh cml0eQoKICAgT25lIG9mIHRoZSBkZWNpc2lvbnMgdGhlIENNIG5lZWRzIHRvIG1ha2UgaXMg dGhlIGdyYW51bGFyaXR5IGF0CiAgIHdoaWNoIGEgbWFjcm9mbG93IGlzIGNvbnN0cnVjdGVk LCBieSBkZWNpZGluZyB3aGljaCBzdHJlYW1zIGJlbG9uZwogICB0byB0aGUgc2FtZSBtYWNy b2Zsb3cgYW5kIHNoYXJlIGNvbmdlc3Rpb24gaW5mb3JtYXRpb24uICBUaGUgQVBJCiAgIHBy b3ZpZGVzIHR3byBmdW5jdGlvbnMgdGhhdCBhbGxvdyBhcHBsaWNhdGlvbnMgdG8gZGVjaWRl IHdoaWNoIG9mCiAgIHRoZWlyIHN0cmVhbXMgb3VnaHQgdG8gYmVsb25nIHRvIHRoZSBzYW1l IG1hY3JvZmxvdy4KCiAgIGNtX2dldG1hY3JvZmxvdyhpMzIgY21fc3RyZWFtaWQpIHJldHVy bnMgYSB1bmlxdWUgaTMyIG1hY3JvZmxvdwogICBpZGVudGlmaWVyLiAgY21fc2V0bWFjcm9m bG93KGkzMiBjbV9tYWNyb2Zsb3dpZCwgaTMyIGNtX3N0cmVhbWlkKQogICBzZXRzIHRoZSBt YWNyb2Zsb3cgb2YgdGhlIHN0cmVhbSBjbV9zdHJlYW1pZCB0byBjbV9tYWNyb2Zsb3dpZC4g IElmIHRoZQogICBjbV9tYWNyb2Zsb3dpZCB0aGF0IGlzIHBhc3NlZCB0byBjbV9zZXRtYWNy b2Zsb3coKSBpcyAtMSwgdGhlbiBhCiAgIG5ldyBtYWNyb2Zsb3cgaXMgY29uc3RydWN0ZWQg YW5kIHRoaXMgaXMgcmV0dXJuZWQgdG8gdGhlIGNhbGxlci4KICAgRWFjaCBjYWxsIHRvIGNt X3NldG1hY3JvZmxvdygpIG92ZXJyaWRlcyB0aGUgcHJldmlvdXMgbWFjcm9mbG93CiAgIGFz c29jaWF0aW9uIGZvciB0aGUgc3RyZWFtLCBzaG91bGQgb25lIGV4aXN0LgoKICAgVGhlIGRl ZmF1bHQgc3VnZ2VzdGVkIGFnZ3JlZ2F0aW9uIG1ldGhvZCBpcyB0byBhZ2dyZWdhdGUgYnkK ICAgZGVzdGluYXRpb24gSVAgYWRkcmVzczsgaS5lLiwgYWxsIHN0cmVhbXMgdG8gdGhlIHNh bWUgZGVzdGluYXRpb24KICAgYWRkcmVzcyBhcmUgYWdncmVnYXRlZCB0byBhIHNpbmdsZSBt YWNyb2Zsb3cgYnkgZGVmYXVsdC4gIFRoZQogICBjbV9nZXRtYWNyb2Zsb3coKSBhbmQgY21f c2V0bWFjcm9mbG93KCkgY2FsbHMgY2FuIHRoZW4gYmUgdXNlZCB0bwogICBjaGFuZ2UgdGhp cyBhcyBuZWVkZWQuICBXZSBkbyBub3RlIHRoYXQgdGhlcmUgYXJlIHNvbWUgY2FzZXMgd2hl cmUKICAgdGhpcyBtYXkgbm90IGJlIG9wdGltYWwsIGV2ZW4gb3ZlciBiZXN0LWVmZm9ydCBu ZXR3b3Jrcy4gIEZvcgogICBleGFtcGxlLCB3aGVuIGEgZ3JvdXAgb2YgcmVjZWl2ZXJzIGFy ZSBiZWhpbmQgYSBOQVQgZGV2aWNlLCB0aGUKICAgc2VuZGVyIHdpbGwgc2VlIHRoZW0gYWxs IGFzIG9uZSBhZGRyZXNzLiAgSWYgdGhlIGhvc3RzIGJlaGluZCB0aGUKICAgTkFUIGFyZSBp biBmYWN0IGNvbm5lY3RlZCBvdmVyIGRpZmZlcmVudCBib3R0bGVuZWNrIGxpbmtzLCBzb21l IG9mCiAgIHRob3NlIGhvc3RzIGNvdWxkIHNlZSB3b3JzZSBwZXJmb3JtYW5jZSB0aGFuIGJl Zm9yZS4gIEl0IGlzCiAgIHBvc3NpYmxlIHRvIGRldGVjdCBzdWNoIGhvc3RzIHdoZW4gdXNp bmcgZGVsYXkgYW5kIGxvc3MgZXN0aW1hdGVzLAogICBhbHRob3VnaCB0aGUgc3BlY2lmaWMg bWVjaGFuaXNtcyBmb3IgZG9pbmcgc28gYXJlIGJleW9uZCB0aGUgc2NvcGUKICAgb2YgdGhp cyBkb2N1bWVudC4KCiAgIFRoZSBvYmplY3RpdmUgb2YgdGhpcyBpbnRlcmZhY2UgaXMgdG8g c2V0IHVwIHNoYXJpbmcgb2YgZ3JvdXBzIG5vdAogICBzaGFyaW5nIHBvbGljeSBvZiByZWxh dGl2ZSB3ZWlnaHRzIG9mIHN0cmVhbXMgaW4gYSBtYWNyb2Zsb3cuICBUaGUKICAgbGF0dGVy IHJlcXVpcmVzIHRoZSBzY2hlZHVsZXIgdG8gcHJvdmlkZSBhbiBpbnRlcmZhY2UgdG8gc2V0 CiAgIHNoYXJpbmcgcG9saWN5LiAgSG93ZXZlciwgYmVjYXVzZSB3ZSB3YW50IHRvIHN1cHBv cnQgbWFueSBkaWZmZXJlbnQKICAgc2NoZWR1bGVycyAoZWFjaCBvZiB3aGljaCBtYXkgbmVl ZCBkaWZmZXJlbnQgaW5mb3JtYXRpb24gdG8gc2V0CiAgIHBvbGljeSksIHdlIGRvIG5vdCBz cGVjaWZ5IGEgY29tcGxldGUgQVBJIHRvIHRoZSBzY2hlZHVsZXIgKGJ1dCBzZWUKICAgU2Vj dGlvbiA1LjIpLiAgQSBsYXRlciBndWlkZWxpbmUgZG9jdW1lbnQgaXMgZXhwZWN0ZWQgdG8g ZGVzY3JpYmUgYQogICBmZXcgc2ltcGxlIHNjaGVkdWxlcnMgKGUuZy4sIHdlaWdodGVkIHJv dW5kLXJvYmluLCBoaWVyYXJjaGljYWwKICAgc2NoZWR1bGluZykgYW5kIHRoZSBBUEkgdGhl eSBleHBvcnQgdG8gcHJvdmlkZSByZWxhdGl2ZQogICBwcmlvcml0aXphdGlvbi4KCgo1LiAg ICAgIENNIGludGVybmFscwoKICAgVGhpcyBzZWN0aW9uIGRlc2NyaWJlcyB0aGUgaW50ZXJu YWwgY29tcG9uZW50cyBvZiB0aGUgQ00uICBJdAogICBpbmNsdWRlcyBhIENvbmdlc3Rpb24g Q29udHJvbGxlciBhbmQgYSBTY2hlZHVsZXIsIHdpdGgKICAgd2VsbC1kZWZpbmVkLCBhYnN0 cmFjdCBpbnRlcmZhY2VzIGV4cG9ydGVkIGJ5IHRoZW0uCgogICA1LjEgQ29uZ2VzdGlvbiBj b250cm9sbGVyCgogICBBc3NvY2lhdGVkIHdpdGggZWFjaCBtYWNyb2Zsb3cgaXMgYSBjb25n ZXN0aW9uIGNvbnRyb2wgYWxnb3JpdGhtOwogICB0aGUgY29sbGVjdGlvbiBvZiBhbGwgdGhl c2UgYWxnb3JpdGhtcyBjb21wcmlzZXMgdGhlIGNvbmdlc3Rpb24KICAgY29udHJvbGxlciBv ZiB0aGUgQ00uICBUaGUgY29udHJvbCBhbGdvcml0aG0gZGVjaWRlcyB3aGVuIGFuZCBob3cK ICAgbXVjaCBkYXRhIGNhbiBiZSB0cmFuc21pdHRlZCBieSBhIG1hY3JvZmxvdy4gIEl0IHVz ZXMgYXBwbGljYXRpb24KICAgbm90aWZpY2F0aW9ucyAoU2VjdGlvbiA0LjMpIGZyb20gY29u Y3VycmVudCBzdHJlYW1zIG9uIHRoZSBzYW1lCiAgIG1hY3JvZmxvdyB0byBidWlsZCB1cCBp bmZvcm1hdGlvbiBhYm91dCB0aGUgY29uZ2VzdGlvbiBzdGF0ZSBvZiB0aGUKICAgbmV0d29y ayBwYXRoIHVzZWQgYnkgdGhlIG1hY3JvZmxvdy4KCiAgIFRoZSBjb25nZXN0aW9uIGNvbnRy b2xsZXIgTVVTVCBpbXBsZW1lbnQgYSAiVENQLWZyaWVuZGx5IgogICBbTWFoZGF2aTk4XSBj b25nZXN0aW9uIGNvbnRyb2wgYWxnb3JpdGhtLiAgU2V2ZXJhbCBtYWNyb2Zsb3dzIE1BWQog ICAoYW5kIGluZGVlZCwgb2Z0ZW4gd2lsbCkgdXNlIHRoZSBzYW1lIGNvbmdlc3Rpb24gY29u dHJvbCBhbGdvcml0aG0KICAgYnV0IGVhY2ggbWFjcm9mbG93IG1haW50YWlucyBzdGF0ZSBh Ym91dCB0aGUgbmV0d29yayB1c2VkIGJ5IGl0cwogICBzdHJlYW1zLgoKICAgVGhlIGNvbmdl c3Rpb24gY29udHJvbCBtb2R1bGUgTVVTVCBpbXBsZW1lbnQgdGhlIGZvbGxvd2luZyBhYnN0 cmFjdAogICBpbnRlcmZhY2VzLiAgV2UgZW1waGFzaXplIHRoYXQgdGhlc2UgYXJlIG5vdCBk aXJlY3RseSB2aXNpYmxlIHRvCiAgIGFwcGxpY2F0aW9uczsgdGhleSBhcmUgd2l0aGluIHRo ZSBjb250ZXh0IG9mIGEgbWFjcm9mbG93LCBhbmQgYXJlCiAgIGRpZmZlcmVudCBmcm9tIHRo ZSBDTSBBUEkgZnVuY3Rpb25zIG9mIFNlY3Rpb24gNC4KCiAgIC0gdm9pZCBxdWVyeSh1NjQg KnJhdGUsIHUzMiAqc3J0dCwgdTMyICpydHRkZXYpOiBUaGlzIGZ1bmN0aW9uCiAgICAgcmV0 dXJucyB0aGUgZXN0aW1hdGVkIHJhdGUgKGluIGJpdHMgcGVyIHNlY29uZCkgYW5kIHNtb290 aGVkCiAgICAgcm91bmQgdHJpcCB0aW1lIChpbiBtaWNyb3NlY29uZHMpIGZvciB0aGUgbWFj cm9mbG93LgoKICAgLSB2b2lkIG5vdGlmeSh1MzIgbnNlbnQpOiBUaGlzIGZ1bmN0aW9uIE1V U1QgYmUgdXNlZCB0byBub3RpZnkgdGhlCiAgICAgY29uZ2VzdGlvbiBjb250cm9sIG1vZHVs ZSB3aGVuZXZlciBkYXRhIGlzIHNlbnQgYnkgYW4KICAgICBhcHBsaWNhdGlvbi4gIFRoZSBu c2VudCBwYXJhbWV0ZXIgaW5kaWNhdGVzIHRoZSBudW1iZXIgb2YgYnl0ZXMKICAgICBqdXN0 IHNlbnQgYnkgdGhlIGFwcGxpY2F0aW9uLgoKICAgLSB2b2lkIHVwZGF0ZSh1MzIgbnNlbnQs IHUzMiBucmVjZCwgdTMyIHJ0dCwgdTMyIGxvc3Ntb2RlKTogVGhpcwogICAgIGZ1bmN0aW9u IGlzIGNhbGxlZCB3aGVuZXZlciBhbnkgb2YgdGhlIENNIHN0cmVhbXMgYXNzb2NpYXRlZCB3 aXRoCiAgICAgYSBtYWNyb2Zsb3cgaWRlbnRpZmllcyB0aGF0IGRhdGEgaGFzIHJlYWNoZWQg dGhlIHJlY2VpdmVyIG9yIGhhcwogICAgIGJlZW4gbG9zdCBlbiByb3V0ZS4gIFRoZSBucmVj ZCBwYXJhbWV0ZXIgaW5kaWNhdGVzIHRoZSBudW1iZXIgb2YKICAgICBieXRlcyB0aGF0IGhh dmUganVzdCBhcnJpdmVkIGF0IHRoZSByZWNlaXZlci4gIFRoZSBuc2VudAogICAgIHBhcmFt ZXRlciBpcyB0aGUgc3VtIG9mIHRoZSBudW1iZXIgb2YgYnl0ZXMganVzdCByZWNlaXZlZCBh bmQgdGhlCiAgICAgbnVtYmVyIG9mIGJ5dGVzIGlkZW50aWZpZWQgYXMgbG9zdCBlbiByb3V0 ZS4gVGhlIHJ0dCBwYXJhbWV0ZXIgaXMKICAgICB0aGUgZXN0aW1hdGVkIHJvdW5kIHRyaXAg dGltZSBpbiBtaWNyb3NlY29uZHMgZHVyaW5nIHRoZQogICAgIHRyYW5zZmVyLiAgVGhlIGxv c3Ntb2RlIHBhcmFtZXRlciBwcm92aWRlcyBhbiBpbmRpY2F0b3Igb2YgaG93IGEKICAgICBs b3NzIHdhcyBkZXRlY3RlZCAoc2VjdGlvbiA0LjMpLgoKICAgQWx0aG91Z2ggdGhlc2UgaW50 ZXJmYWNlcyBhcmUgbm90IHZpc2libGUgdG8gYXBwbGljYXRpb25zLCB0aGUKICAgY29uZ2Vz dGlvbiBjb250cm9sbGVyIE1VU1QgaW1wbGVtZW50IHRoZXNlIGFic3RyYWN0IGludGVyZmFj ZXMgdG8KICAgcHJvdmlkZSBmb3IgbW9kdWxhciBpbnRlci1vcGVyYWJpbGl0eSB3aXRoIGRp ZmZlcmVudAogICBzZXBhcmF0ZWx5LWRldmVsb3BlZCBzY2hlZHVsZXJzLgoKICAgVGhlIGNv bmdlc3Rpb24gY29udHJvbCBtb2R1bGUgTVVTVCBhbHNvIGNhbGwgdGhlIGFzc29jaWF0ZWQK ICAgc2NoZWR1bGVyJ3Mgc2NoZWR1bGUgZnVuY3Rpb24gKHNlY3Rpb24gNS4yKSB3aGVuIGl0 IGJlbGlldmVzIHRoYXQKICAgdGhlIGN1cnJlbnQgY29uZ2VzdGlvbiBzdGF0ZSBhbGxvd3Mg YW4gTVRVLXNpemVkIHBhY2tldCB0byBiZSBzZW50LgoKICAgNS4yIFNjaGVkdWxlcgoKICAg V2hpbGUgaXQgaXMgdGhlIHJlc3BvbnNpYmlsaXR5IG9mIHRoZSBjb25nZXN0aW9uIGNvbnRy b2wgbW9kdWxlIHRvCiAgIGRldGVybWluZSB3aGVuIGFuZCBob3cgbXVjaCBkYXRhIGNhbiBi ZSB0cmFuc21pdHRlZCwgaXQgaXMgdGhlCiAgIHJlc3BvbnNpYmlsaXR5IG9mIGEgbWFjcm9m bG93J3Mgc2NoZWR1bGVyIG1vZHVsZSB0byBkZXRlcm1pbmUgd2hpY2gKICAgb2YgdGhlIHN0 cmVhbXMgc2hvdWxkIGdldCB0aGUgb3Bwb3J0dW5pdHkgdG8gdHJhbnNtaXQgZGF0YS4KCiAg IFRoZSBTY2hlZHVsZXIgTVVTVCBpbXBsZW1lbnQgdGhlIGZvbGxvd2luZyBpbnRlcmZhY2Vz OgoKICAgLSB2b2lkIHNjaGVkdWxlKHUzMiBudW1fYnl0ZXMpOiBXaGVuIHRoZSBjb25nZXN0 aW9uIGNvbnRyb2wgbW9kdWxlCiAgICAgZGV0ZXJtaW5lcyB0aGF0IGRhdGEgY2FuIGJlIHNl bnQsIHRoZSBzY2hlZHVsZSgpIHJvdXRpbmUgTVVTVCBiZQogICAgIGNhbGxlZCB3aXRoIG5v IG1vcmUgdGhhbiB0aGUgbnVtYmVyIG9mIGJ5dGVzIHRoYXQgY2FuIGJlIHNlbnQuCiAgICAg SW4gdHVybiwgdGhlIHNjaGVkdWxlciBNQVkgY2FsbCB0aGUgY21hcHBfc2VuZCgpIGZ1bmN0 aW9uIHRoYXQgQ00KICAgICBhcHBsaWNhdGlvbnMgbXVzdCBwcm92aWRlLgoKICAgLSBmbG9h dCBxdWVyeV9zaGFyZShpMzIgY21fc3RyZWFtaWQpOiBUaGlzIGNhbGwgcmV0dXJucyB0aGUK ICAgICBkZXNjcmliZWQgc3RyZWFtJ3Mgc2hhcmUgb2YgdGhlIHRvdGFsIGJhbmR3aWR0aCBh dmFpbGFibGUgdG8gdGhlCiAgICAgbWFjcm9mbG93LiAgVGhpcyBjYWxsIGNvbWJpbmVkIHdp dGggdGhlIHF1ZXJ5IGNhbGwgb2YgdGhlCiAgICAgY29uZ2VzdGlvbiBjb250cm9sbGVyIHBy b3ZpZGVzIHRoZSBpbmZvcm1hdGlvbiB0byBzYXRpc2Z5IGFuCiAgICAgYXBwbGljYXRpb24n cyBjbV9xdWVyeSgpIHJlcXVlc3QuCgogICAtIHZvaWQgbm90aWZ5KGkzMiBjbV9zdHJlYW1p ZCwgdTMyIG5zZW50KTogVGhpcyBpbnRlcmZhY2UgaXMgdXNlZAogICAgIHRvIG5vdGlmeSB0 aGUgc2NoZWR1bGVyIG1vZHVsZSB3aGVuZXZlciBkYXRhIGlzIHNlbnQgYnkgYSBDTQogICAg IGFwcGxpY2F0aW9uLiAgVGhlIG5zZW50IHBhcmFtZXRlciBpbmRpY2F0ZXMgdGhlIG51bWJl ciBvZiBieXRlcwogICAgIGp1c3Qgc2VudCBieSB0aGUgYXBwbGljYXRpb24uCgogICAgIFRo ZSBTY2hlZHVsZXIgTUFZIGltcGxlbWVudCBtYW55IGFkZGl0aW9uYWwgaW50ZXJmYWNlcy4g IEFzCiAgICAgZXhwZXJpZW5jZSB3aXRoIENNIHNjaGVkdWxlcnMgaW5jcmVhc2VzLCBmdXR1 cmUgZG9jdW1lbnRzIG1heQogICAgIG1ha2UgYWRkaXRpb25zIGFuZC9vciBjaGFuZ2VzIHRv IHNvbWUgcGFydHMgb2YgdGhlIHNjaGVkdWxlcgogICAgIEFQSS4gCgo2LiAgICAgIEV4YW1w bGVzCgogICA2LjEgRXhhbXBsZSBhcHBsaWNhdGlvbnMKCiAgIFRoZSBmb2xsb3dpbmcgZGVz Y3JpYmVzIHRoZSBwb3NzaWJsZSB1c2Ugb2YgdGhlIENNIEFQSSBieSBhbgogICBhc3luY2hy b25vdXMgYXBwbGljYXRpb24gKGFuIGltcGxlbWVudGF0aW9uIG9mIGEgVENQIHNlbmRlcikg YW5kIGEKICAgc3luY2hyb25vdXMgYXBwbGljYXRpb24gKGFuIGF1ZGlvIHNlcnZlcikuICBN b3JlIGRldGFpbHMgb2YgdGhlc2UKICAgYXBwbGljYXRpb25zIGFuZCBDTSBpbXBsZW1lbnRh dGlvbiBvcHRpbWl6YXRpb25zIGZvciBlZmZpY2llbnQKICAgb3BlcmF0aW9uIGFyZSBkZXNj cmliZWQgaW4gW0FuZGVyc2VuMDBdLiAgV2UgZW1waGFzaXplIHRoYXQgdGhlCiAgIHByb3Rv Y29scyBpbiB0aGlzIHNlY3Rpb24gYXJlIGV4YW1wbGVzIGFuZCBzdWdnZXN0aW9ucyBmb3IK ICAgaW1wbGVtZW50YXRpb24sIHJhdGhlciB0aGFuIHJlcXVpcmVtZW50cyBvZiBhbnkgY29u Zm9ybWFudAogICBpbXBsZW1lbnRhdGlvbi4gCgogICA2LjEuMSBUQ1AKCiAgIEEgVENQIE1V U1QgdXNlIHRoZSBjbWFwcF9zZW5kKCkgY2FsbGJhY2sgQVBJLiBUQ1Agb25seSBpZGVudGlm aWVzCiAgIHdoaWNoIGRhdGEgaXQgc2hvdWxkIHNlbmQgdXBvbiB0aGUgYXJyaXZhbCBvZiBh biBhY2tub3dsZWRnZW1lbnQgb3IKICAgZXhwaXJhdGlvbiBvZiBhIHRpbWVyLiBBcyBhIHJl c3VsdCwgaXQgcmVxdWlyZXMgdGlnaHQgY29udHJvbCBvdmVyCiAgIHdoZW4gYW5kIGlmIG5l dyBkYXRhIG9yIHJldHJhbnNtaXNzaW9ucyBhcmUgc2VudC4KCiAgIFdoZW4gVENQIGVpdGhl ciBjb25uZWN0cyB0byBvciBhY2NlcHRzIGEgY29ubmVjdGlvbiBmcm9tIGFub3RoZXIKICAg aG9zdCwgaXQgcGVyZm9ybXMgYSBjbV9vcGVuKCkgY2FsbCB0byBhc3NvY2lhdGUgdGhlIFRD UCBjb25uZWN0aW9uCiAgIHdpdGggYSBjbV9zdHJlYW1pZC4KCiAgIE9uY2UgYSBjb25uZWN0 aW9uIGlzIGVzdGFibGlzaGVkLCB0aGUgQ00gaXMgdXNlZCB0byBjb250cm9sIHRoZQogICB0 cmFuc21pc3Npb24gb2Ygb3V0Z29pbmcgZGF0YS4gIFRoZSBDTSBlbGltaW5hdGVzIHRoZSBu ZWVkIGZvcgogICB0cmFja2luZyBhbmQgcmVhY3RpbmcgdG8gY29uZ2VzdGlvbiBpbiBUQ1As IGJlY2F1c2UgdGhlIENNIGFuZCBpdHMKICAgdHJhbnNtaXNzaW9uIEFQSSBlbnN1cmUgcHJv cGVyIGNvbmdlc3Rpb24gYmVoYXZpb3IuICBMb3NzIHJlY292ZXJ5CiAgIGlzIHN0aWxsIHBl cmZvcm1lZCBieSBUQ1AgYmFzZWQgb24gZmFzdCByZXRyYW5zbWlzc2lvbnMgYW5kCiAgIHJl Y292ZXJ5IGFzIHdlbGwgYXMgdGltZW91dHMuICBJbiBhZGRpdGlvbiwgVENQIGlzIGFsc28g bW9kaWZpZWQgdG8KICAgaGF2ZSBpdHMgb3duIG91dHN0YW5kaW5nIHdpbmRvdyAodGNwX293 bmQpIGVzdGltYXRlLiAgV2hlbmV2ZXIgZGF0YQogICBzZWdtZW50cyBhcmUgc2VudCBmcm9t IGl0cyBjbWFwcF9zZW5kKCkgY2FsbGJhY2ssIFRDUCB1cGRhdGVzIGl0cwogICB0Y3Bfb3du ZCB2YWx1ZS4gVGhlIG93bmQgdmFyaWFibGUgaXMgYWxzbyB1cGRhdGVkIGFmdGVyIGVhY2gK ICAgY21fdXBkYXRlKCkgY2FsbC4gVENQIGFsc28gbWFpbnRhaW5zIGEgY291bnQgb2YgdGhl IG51bWJlciBvZgogICBvdXRzdGFuZGluZyBzZWdtZW50cyAocGt0X2NudCkuICBBdCBhbnkg dGltZSwgVENQIGNhbiBjYWxjdWxhdGUgdGhlCiAgIGF2ZXJhZ2UgcGFja2V0IHNpemUgKGF2 Z19wa3Rfc2l6ZSkgYXMgdGNwX293bmQvcGt0X2NudC4gIFRoZQogICBhdmdfcGt0X3NpemUg aXMgdXNlZCBieSBUQ1AgdG8gaGVscCBlc3RpbWF0ZSB0aGUgYW1vdW50IG9mCiAgIG91dHN0 YW5kaW5nIGRhdGEuICBOb3RlIHRoYXQgdGhpcyBpcyBub3QgbmVlZGVkIGlmIHRoZSBTQUNL IG9wdGlvbgogICBpcyB1c2VkIG9uIHRoZSBjb25uZWN0aW9uLCBzaW5jZSB0aGlzIGluZm9y bWF0aW9uIGlzIGV4cGxpY2l0bHkKICAgYXZhaWxhYmxlLgoKICAgVGhlIFRDUCBvdXRwdXQg cm91dGluZXMgYXJlIG1vZGlmaWVkIGFzIGZvbGxvd3M6CgogICAgIDEuIEFsbCBjb25nZXN0 aW9uIHdpbmRvdyAoY3duZCkgY2hlY2tzIGFyZSByZW1vdmVkLgoKICAgICAyLiBXaGVuIGFw cGxpY2F0aW9uIGRhdGEgaXMgYXZhaWxhYmxlLiAgVGhlIFRDUCBvdXRwdXQgcm91dGluZXMK ICAgICBwZXJmb3JtIGFsbCBub24tY29uZ2VzdGlvbiBjaGVja3MgKE5hZ2xlIGFsZ29yaXRo bSwKICAgICByZWNlaXZlci1hZHZlcnRpc2VkIHdpbmRvdyBjaGVjaywgZXRjKS4gIElmIHRo ZXNlIGNoZWNrcyBwYXNzLAogICAgIHRoZSBvdXRwdXQgcm91dGluZSBxdWV1ZXMgdGhlIGRh dGEgYW5kIGNhbGxzIGNtX3JlcXVlc3QoKSBmb3IgdGhlCiAgICAgc3RyZWFtLgoKICAgICAz LiBJZiBpbmNvbWluZyBkYXRhIG9yIHRpbWVycyByZXN1bHQgaW4gYSBsb3NzIGJlaW5nIGRl dGVjdGVkLAogICAgIHRoZSByZXRyYW5zbWlzc2lvbiBpcyBhbHNvIHBsYWNlZCBpbiBhIHF1 ZXVlIGFuZCBjbV9yZXF1ZXN0KCkgaXMKICAgICBjYWxsZWQgZm9yIHRoZSBzdHJlYW0uCgog ICAgIDQuIFRoZSBjbWFwcF9zZW5kKCkgY2FsbGJhY2sgZm9yIFRDUCBpcyBzZXQgdG8gYW4g b3V0cHV0CiAgICAgcm91dGluZS4gSWYgYW55IHJldHJhbnNtaXNzaW9uIGlzIGVucXVldWVk LCB0aGUgcm91dGluZSBvdXRwdXRzCiAgICAgdGhlIHJldHJhbnNtaXNzaW9uLiAgT3RoZXJ3 aXNlLCB0aGUgcm91dGluZSBvdXRwdXRzIGFzIG11Y2ggbmV3CiAgICAgZGF0YSBhcyB0aGUg VENQIGNvbm5lY3Rpb24gc3RhdGUgYWxsb3dzLiAgSG93ZXZlciwgdGhlCiAgICAgY21hcHBf c2VuZCgpIG5ldmVyIHNlbmRzIG1vcmUgdGhhbiBhIHNpbmdsZSBzZWdtZW50IHBlciBjYWxs LgogICAgIFRoaXMgcm91dGluZSBhcnJhbmdlcyBmb3IgdGhlIG90aGVyIG91dHB1dCBjb21w dXRhdGlvbnMgdG8gYmUKICAgICBkb25lLCBzdWNoIGFzIGhlYWRlciBhbmQgb3B0aW9ucyBj b21wdXRhdGlvbnMuCgogICBUaGUgSVAgb3V0cHV0IHJvdXRpbmUgb24gdGhlIGhvc3QgY2Fs bHMgY21fbm90aWZ5KCkgd2hlbiB0aGUKICAgcGFja2V0cyBhcmUgYWN0dWFsbHkgc2VudCBv dXQuICBCZWNhdXNlIGl0IGRvZXMgbm90IGtub3cgd2hpY2gKICAgY21fc3RyZWFtaWQgaXMg cmVzcG9uc2libGUgZm9yIHRoZSBwYWNrZXQsIGNtX25vdGlmeSgpIHRha2VzIHRoZQogICBz dHJlYW1faW5mbyBhcyBhcmd1bWVudCAoc2VlIFNlY3Rpb24gNCBmb3Igd2hhdCB0aGUgc3Ry ZWFtX2luZm8KICAgc2hvdWxkIGNvbnRhaW4pLiAgQmVjYXVzZSBjbV9ub3RpZnkoKSByZXBv cnRzIHRoZSBJUCBwYXlsb2FkIHNpemUsCiAgIFRDUCBrZWVwcyB0cmFjayBvZiB0aGUgdG90 YWwgaGVhZGVyIHNpemUgYW5kIGluY29ycG9yYXRlcyB0aGVzZQogICB1cGRhdGVzLgoKICAg VGhlIFRDUCBpbnB1dCByb3V0aW5lcyBhcmUgbW9kaWZpZWQgYXMgZm9sbG93czoKCiAgICAg MS4gUlRUIGVzdGltYXRpb24gaXMgZG9uZSBhcyBub3JtYWwgdXNpbmcgZWl0aGVyIHRpbWVz dGFtcHMgb3IKICAgICBLYXJuJ3MgYWxnb3JpdGhtLiAgQW55IHJ0dCBlc3RpbWF0ZSB0aGF0 IGlzIGdlbmVyYXRlZCBpcyBwYXNzZWQKICAgICB0byBDTSB2aWEgdGhlIGNtX3VwZGF0ZSBj YWxsLgoKICAgICAyLiBBbGwgY3duZCBhbmQgc2xvdyBzdGFydCB0aHJlc2hvbGQgKHNzdGhy ZXNoKSB1cGRhdGVzIGFyZQogICAgIHJlbW92ZWQuCgogICAgIDMuIFVwb24gdGhlIGFycml2 YWwgb2YgYW4gYWNrIGZvciBuZXcgZGF0YSwgVENQIGNvbXB1dGVzIHRoZQogICAgIHZhbHVl IG9mIGluX2ZsaWdodCAodGhlIGFtb3VudCBvZiBkYXRhIGluIGZsaWdodCkgYXMKICAgICBz bmRfbWF4LWFjay0xIChpLmUuIE1BWCBTZXF1ZW5jZSBTZW50IC0gQ3VycmVudCBBY2sgLSAx KS4gVENQCiAgICAgdGhlbiBjYWxscyBjbV91cGRhdGUoc3RyZWFtaWQsIHRjcF9vd25kIC0g aW5fZmxpZ2h0LCAwLAogICAgIENNX05PX0NPTkdFU1RJT04sIHJ0dCkuCgogICAgIDQuIFVw b24gdGhlIGFycml2YWwgb2YgYSBkdXBsaWNhdGUgYWNrbm93bGVkZ2VtZW50LCBUQ1AgbXVz dAogICAgIGNoZWNrIGl0cyBkdXBhY2sgY291bnQgKGR1cF9hY2tzKSB0byBkZXRlcm1pbmUg aXRzIGFjdGlvbi4gSWYKICAgICBkdXBfYWNrcyA8IDMsIHRoZSBUQ1AgZG9lcyBub3RoaW5n LiAgSWYgZHVwX2Fja3MgPT0gMywgVENQCiAgICAgYXNzdW1lcyB0aGF0IGEgcGFja2V0IHdh cyBsb3N0IGFuZCB0aGF0IGF0IGxlYXN0IDMgcGFja2V0cwogICAgIGFycml2ZWQgdG8gZ2Vu ZXJhdGUgdGhlc2UgZHVwbGljYXRlIGFja3MuIFRoZXJlZm9yZSwgaXQgY2FsbHMKICAgICBj bV91cGRhdGUoc3RyZWFtaWQsIDQgKiBhdmdfcGt0X3NpemUsIDMgKiBhdmdfcGt0X3NpemUs CiAgICAgQ01fTE9TU19GRUVEQkFDSywgcnR0KS4gIFRoZSBhdmVyYWdlIHBhY2tldCBzaXpl IGlzIHVzZWQgc2luY2UgdGhlCiAgICAgYWNrbm93bGVkZ2VtZW50cyBkbyBub3QgaW5kaWNh dGUgZXhhY3RseSBob3cgbXVjaCBkYXRhIGhhcwogICAgIHJlYWNoZWQgdGhlIG90aGVyIGVu ZC4gIE1vc3QgVENQIGltcGxlbWVudGF0aW9ucyBpbnRlcnByZXQgYQogICAgIGR1cGxpY2F0 ZSBBQ0sgYXMgYW4gaW5kaWNhdGlvbiB0aGF0IGEgZnVsbCBNU1MgaGFzIHJlYWNoZWQgaXRz CiAgICAgZGVzdGluYXRpb24uICBPbmNlIGEgbmV3IEFDSyBpcyByZWNlaXZlZCwgdGhlc2Ug VENQIHNlbmRlcgogICAgIGltcGxlbWVudGF0aW9ucyBtYXkgcmVzeW5jaHJvbml6ZSB3aXRo IFRDUCByZWNlaXZlci4gIFRoZSBDTSBBUEkKICAgICBkb2VzIG5vdCBwcm92aWRlIGEgbWVj aGFuaXNtIGZvciBUQ1AgdG8gcGFzcyBpbmZvcm1hdGlvbiBmcm9tCiAgICAgdGhpcyByZXN5 bmNocm9uaXphdGlvbi4gIFRoZXJlZm9yZSwgVENQIGNhbiBvbmx5IGluZmVyIHRoZQogICAg IGFycml2YWwgb2YgYW4gYXZnX3BrdF9zaXplIGFtb3VudCBvZiBkYXRhIGZyb20gZWFjaCBk dXBsaWNhdGUKICAgICBhY2suIFRDUCBhbHNvIGVucXVldWVzIGEgcmV0cmFuc21pc3Npb24g b2YgdGhlIGxvc3Qgc2VnbWVudCBhbmQKICAgICBjYWxscyBjbV9yZXF1ZXN0KCkuICBJZiBk dXBfYWNrcyA+IDMsIFRDUCBhc3N1bWVzIHRoYXQgYSBwYWNrZXQKICAgICBoYXMgcmVhY2hl ZCB0aGUgb3RoZXIgZW5kIGFuZCBjYXVzZWQgdGhpcyBhY2sgdG8gYmUgc2VudC4gIEFzIGEK ICAgICByZXN1bHQsIGl0IGNhbGxzIGNtX3VwZGF0ZShzdHJlYW1pZCwgYXZnX3BrdF9zaXpl LCBhdmdfcGt0X3NpemUsCiAgICAgQ01fTk9fQ09OR0VTVElPTiwgcnR0KS4KCiAgICAgNS4g VXBvbiB0aGUgYXJyaXZhbCBvZiBhIHBhcnRpYWwgYWNrbm93bGVkZ21lbnQgKG9uZSB0aGF0 IGRvZXMKICAgICBub3QgZXhjZWVkIHRoZSBoaWdoZXN0IHNlZ21lbnQgdHJhbnNtaXR0ZWQg YXQgdGhlIHRpbWUgdGhlIGxvc3MKICAgICBvY2N1cnJlZCwgYXMgZGVmaW5lZCBpbiBbRmxv eWQ5OWJdKSwgVENQIGFzc3VtZXMgdGhhdCBhIHBhY2tldAogICAgIHdhcyBsb3N0IGFuZCB0 aGF0IHRoZSByZXRyYW5zbWl0dGVkIHBhY2tldCBoYXMgcmVhY2hlZCB0aGUKICAgICByZWNp cGllbnQuICBUaGVyZWZvcmUsIGl0IGNhbGxzIGNtX3VwZGF0ZShzdHJlYW1pZCwgMiAqCiAg ICAgYXZnX3BrdF9zaXplLCBhdmdfcGt0X3NpemUsIENNX05PX0NPTkdFU1RJT04sCiAgICAg cnR0KS4gIENNX05PX0NPTkdFU1RJT04gaXMgdXNlZCBzaW5jZSB0aGUgbG9zcyBwZXJpb2Qg aGFzIGFscmVhZHkKICAgICBiZWVuIHJlcG9ydGVkLiBUQ1AgYWxzbyBlbnF1ZXVlcyBhIHJl dHJhbnNtaXNzaW9uIG9mIHRoZSBsb3N0CiAgICAgc2VnbWVudCBhbmQgY2FsbHMgY21fcmVx dWVzdCgpLgoKICAgV2hlbiB0aGUgVENQIHJldHJhbnNtaXNzaW9uIHRpbWVyIGV4cGlyZXMs IHRoZSBzZW5kZXIgaWRlbnRpZmllcwogICB0aGF0IGEgc2VnbWVudCBoYXMgYmVlbiBsb3N0 IGFuZCBjYWxscyBjbV91cGRhdGUoc3RyZWFtaWQsCiAgIGF2Z19wa3Rfc2l6ZSwgMCwgQ01f Tk9fRkVFREJBQ0ssIDApIHRvIHNpZ25pZnkgdGhlIG9jY3VycmVuY2Ugb2YKICAgcGVyc2lz dGVudCBjb25nZXN0aW9uIHRvIHRoZSBDTS4gIFRDUCBhbHNvIGVucXVldWVzIGEKICAgcmV0 cmFuc21pc3Npb24gb2YgdGhlIGxvc3Qgc2VnbWVudCBhbmQgY2FsbHMgY21fcmVxdWVzdCgp LgoKICAgNi4xLjIgQ29uZ2VzdGlvbi1jb250cm9sbGVkIFVEUAoKICAgQ29uZ2VzdGlvbi1j b250cm9sbGVkIFVEUCBpcyBhIHVzZWZ1bCBDTSBhcHBsaWNhdGlvbiwgd2hpY2ggd2UKICAg ZGVzY3JpYmUgaW4gdGhlIGNvbnRleHQgb2YgQmVya2VsZXkgc29ja2V0cyBbU3RldmVuczk0 XS4gIFRoZXkKICAgcHJvdmlkZSB0aGUgc2FtZSBmdW5jdGlvbmFsaXR5IGFzIHN0YW5kYXJk IEJlcmtlbGV5IFVEUCBzb2NrZXRzLAogICBidXQgaW5zdGVhZCBvZiBpbW1lZGlhdGVseSBz ZW5kaW5nIHRoZSBkYXRhIGZyb20gdGhlIGtlcm5lbCBwYWNrZXQKICAgcXVldWUgdG8gbG93 ZXIgbGF5ZXJzIGZvciB0cmFuc21pc3Npb24sIHRoZSBidWZmZXJlZCBzb2NrZXQKICAgaW1w bGVtZW50YXRpb24gbWFrZXMgY2FsbHMgdG8gdGhlIEFQSSBleHBvcnRlZCBieSB0aGUgQ00g aW5zaWRlIHRoZQogICBrZXJuZWwgYW5kIGdldHMgY2FsbGJhY2tzIGZyb20gdGhlIENNLiAg V2hlbiBhIENNIFVEUCBzb2NrZXQgaXMKICAgY3JlYXRlZCwgaXQgaXMgYm91bmQgdG8gYSBw YXJ0aWN1bGFyIHN0cmVhbS4gIExhdGVyLCB3aGVuIGRhdGEgaXMKICAgYWRkZWQgdG8gdGhl IHBhY2tldCBxdWV1ZSwgY21fcmVxdWVzdCgpIGlzIGNhbGxlZCBvbiB0aGUgc3RyZWFtCiAg IGFzc29jaWF0ZWQgd2l0aCB0aGUgc29ja2V0LiAgV2hlbiB0aGUgQ00gc2NoZWR1bGVzIHRo aXMgc3RyZWFtIGZvcgogICB0cmFuc21pc3Npb24sIGl0IGNhbGxzIHVkcF9jY2FwcHNlbmQo KSBpbiB0aGUgVURQIG1vZHVsZS4gIFRoaXMKICAgZnVuY3Rpb24gdHJhbnNtaXRzIG9uZSBN VFUgZnJvbSB0aGUgcGFja2V0IHF1ZXVlLCBhbmQgc2NoZWR1bGVzIHRoZQogICB0cmFuc21p c3Npb24gb2YgYW55IHJlbWFpbmluZyBwYWNrZXRzLiAgVGhlIGluLWtlcm5lbAogICBpbXBs ZW1lbnRhdGlvbiBvZiB0aGUgQ00gVURQIEFQSSBTSE9VTEQgTk9UIHJlcXVpcmUgYW55IGFk ZGl0aW9uYWwKICAgZGF0YSBjb3BpZXMgYW5kIFNIT1VMRCBzdXBwb3J0IGFsbCBzdGFuZGFy ZCBVRFAgb3B0aW9ucy4gIE1vZGlmeWluZwogICBleGlzdGluZyBhcHBsaWNhdGlvbnMgdG8g dXNlIGNvbmdlc3Rpb24tY29udHJvbGxlZCBVRFAgcmVxdWlyZXMgdGhlCiAgIGltcGxlbWVu dGF0aW9uIG9mIGEgbmV3IHNvY2tldCBvcHRpb24gb24gdGhlIHNvY2tldC4gIFRvIHdvcmsK ICAgY29ycmVjdGx5LCB0aGUgc2VuZGVyIE1VU1Qgb2J0YWluIGZlZWRiYWNrIGFib3V0IGNv bmdlc3Rpb24uICBUaGlzCiAgIGNhbiBiZSBkb25lIGluIGF0IGxlYXN0IHR3byB3YXlzOiAo aSkgdGhlIFVEUCByZWNlaXZlciBhcHBsaWNhdGlvbgogICBjYW4gcHJvdmlkZSBmZWVkYmFj ayB0byB0aGUgc2VuZGVyIGFwcGxpY2F0aW9uLCB3aGljaCB3aWxsIGluZm9ybQogICB0aGUg Q00gb2YgbmV0d29yayBjb25kaXRpb25zIHVzaW5nIGNtX3VwZGF0ZSgpOyAoaWkpIHRoZSBV RFAKICAgcmVjZWl2ZXIgaW1wbGVtZW50YXRpb24gY2FuIHByb3ZpZGUgZmVlZGJhY2sgdG8g dGhlIHNlbmRpbmcgVURQLgogICBOb3RlIHRoYXQgdGhpcyBsYXR0ZXIgYWx0ZXJuYXRpdmUg cmVxdWlyZXMgY2hhbmdlcyB0byB0aGUKICAgcmVjZWl2ZXIncyBuZXR3b3JrIHN0YWNrIGFu ZCB0aGUgc2VuZGVyIFVEUCBjYW5ub3QgYXNzdW1lIHRoYXQgYWxsCiAgIHJlY2VpdmVycyBz dXBwb3J0IHRoaXMgb3B0aW9uIHdpdGhvdXQgZXhwbGljaXQgbmVnb3RpYXRpb24uCgogICA2 LjEuMyBBdWRpbyBzZXJ2ZXIKCiAgIEEgdHlwaWNhbCBhdWRpbyBhcHBsaWNhdGlvbiBvZnRl biBoYXMgYWNjZXNzIHRvIHRoZSBzYW1wbGUgaW4gYQogICBtdWx0aXR1ZGUgb2YgZGF0YSBy YXRlcyBhbmQgcXVhbGl0aWVzLiBUaGUgb2JqZWN0aXZlIG9mIHRoZQogICBhcHBsaWNhdGlv biBpcyB0aGVuIHRvIGRlbGl2ZXIgdGhlIGhpZ2hlc3QgcG9zc2libGUgcXVhbGl0eSBvZgog ICBhdWRpbyAodHlwaWNhbGx5IHRoZSBoaWdoZXN0IGRhdGEgcmF0ZSkgaXRzIGNsaWVudHMu IFRoZSBzZWxlY3Rpb24KICAgb2Ygd2hpY2ggdmVyc2lvbiBvZiBhdWRpbyB0byB0cmFuc21p dCBzaG91bGQgYmUgYmFzZWQgb24gdGhlCiAgIGN1cnJlbnQgY29uZ2VzdGlvbiBzdGF0ZSBv ZiB0aGUgbmV0d29yay4gIEluIGFkZGl0aW9uLCB0aGUgc291cmNlCiAgIHdpbGwgd2FudCBh dWRpbyBkZWxpdmVyZWQgdG8gaXRzIHVzZXJzIGF0IGEgY29uc2lzdGVudCBzYW1wbGluZwog ICByYXRlLiAgQXMgYSByZXN1bHQsIGl0IG11c3Qgc2VuZCBkYXRhIGEgcmVndWxhciByYXRl LCBtaW5pbWl6aW5nCiAgIGRlbGF5aW5nIHRyYW5zbWlzc2lvbnMgYW5kIHJlZHVjaW5nIGJ1 ZmZlcmluZyBiZWZvcmUgcGxheWJhY2suIFRvCiAgIG1lZXQgdGhlc2UgcmVxdWlyZW1lbnRz LCB0aGlzIGFwcGxpY2F0aW9uIGNhbiB1c2UgdGhlIHN5bmNocm9ub3VzCiAgIHNlbmRlciBB UEkgKFNlY3Rpb24gNC4yKS4KCiAgIFdoZW4gdGhlIHNvdXJjZSBmaXJzdCBzdGFydHMsIGl0 IHVzZXMgdGhlIGNtX3F1ZXJ5KCkgY2FsbCB0byBnZXQgYW4KICAgaW5pdGlhbCBlc3RpbWF0 ZSBvZiBuZXR3b3JrIGJhbmR3aWR0aCBhbmQgZGVsYXkuICBJZiBzb21lIG90aGVyCiAgIHN0 cmVhbXMgb24gdGhhdCBtYWNyb2Zsb3cgaGF2ZSBhbHJlYWR5IGJlZW4gYWN0aXZlLCB0aGVu IGl0IGdldHMgYW4KICAgaW5pdGlhbCBlc3RpbWF0ZSB0aGF0IGlzIHZhbGlkOyBvdGhlcndp c2UsIGl0IGdldHMgbmVnYXRpdmUgdmFsdWVzLAogICB3aGljaCBpdCBpZ25vcmVzLiAgSXQg dGhlbiBjaG9vc2VzIGFuIGVuY29kaW5nIHRoYXQgZG9lcyBub3QgZXhjZWVkCiAgIHRoZXNl IGVzdGltYXRlcyAob3IsIGluIHRoZSBjYXNlIG9mIGFuIGludmFsaWQgZXN0aW1hdGUsIHVz ZXMKICAgYXBwbGljYXRpb24tc3BlY2lmaWMgaW5pdGlhbCB2YWx1ZXMpIGFuZCBiZWdpbnMg dHJhbnNtaXR0aW5nCiAgIGRhdGEuIFRoZSBhcHBsaWNhdGlvbiBhbHNvIGltcGxlbWVudHMg dGhlIGNtYXBwX3VwZGF0ZSgpIGNhbGxiYWNrLgogICBXaGVuIHRoZSBDTSBkZXRlcm1pbmVz IHRoYXQgbmV0d29yayBjaGFyYWN0ZXJpc3RpY3MgaGF2ZSBjaGFuZ2VkLAogICBpdCBjYWxs cyB0aGUgYXBwbGljYXRpb24ncyBjbWFwcF91cGRhdGUoKSBmdW5jdGlvbiBhbmQgcGFzc2Vz IGl0IGEKICAgbmV3IHJhdGUgYW5kIHJvdW5kLXRyaXAgdGltZSBlc3RpbWF0ZS4gVGhlIGFw cGxpY2F0aW9uIE1VU1QgY2hhbmdlCiAgIGl0cyBjaG9pY2Ugb2YgYXVkaW8gZW5jb2Rpbmcg dG8gZW5zdXJlIHRoYXQgaXQgZG9lcyBub3QgZXhjZWVkCiAgIHRoZXNlIG5ldyBlc3RpbWF0 ZXMuCgogICBUbyB1c2UgdGhlIENNLCB0aGUgYXBwbGljYXRpb24gTVVTVCBpbmNvcnBvcmF0 ZSBmZWVkYmFjayBmcm9tIHRoZQogICByZWNlaXZlci4gSW4gdGhpcyBleGFtcGxlLCBpdCBt dXN0IHBlcmlvZGljYWxseSAodHlwaWNhbGx5IG9uY2Ugb3IKICAgdHdpY2UgcGVyIHJvdW5k IHRyaXAgdGltZSkgZGV0ZXJtaW5lIGhvdyBtYW55IG9mIGl0cyBwYWNrZXRzCiAgIGFycml2 ZWQgYXQgdGhlIHJlY2VpdmVyLiBXaGVuIHRoZSBzb3VyY2UgZ2V0cyB0aGlzIGZlZWRiYWNr LCBpdAogICBNVVNUIHVzZSBjbV91cGRhdGUoKSB0byBpbmZvcm0gdGhlIENNIG9mIHRoaXMg bmV3IGluZm9ybWF0aW9uLgogICBUaGlzIHJlc3VsdHMgaW4gdGhlIENNIHVwZGF0aW5nIG93 bmQgYW5kIG1heSByZXN1bHQgaW4gQ00gY2hhbmdpbmcKICAgaXRzIGVzdGltYXRlcyBhbmQg Y2FsbGluZyBjbWFwcF91cGRhdGUoKSBvZiB0aGUgc3RyZWFtcyBvZiB0aGUKICAgbWFjcm9m bG93LgoKICAgNi4zIEV4YW1wbGUgY29uZ2VzdGlvbiBjb250cm9sIG1vZHVsZQoKICAgVG8g aWxsdXN0cmF0ZSB0aGUgcmVzcG9uc2liaWxpdGllcyBvZiBhIGNvbmdlc3Rpb24gY29udHJv bCBtb2R1bGUsCiAgIHRoZSBmb2xsb3dpbmcgZGVzY3JpYmVzIHNvbWUgb2YgdGhlIGFjdGlv bnMgb2YgYSBzaW1wbGUgVENQLWxpa2UKICAgY29uZ2VzdGlvbiBjb250cm9sIG1vZHVsZSB0 aGF0IGltcGxlbWVudHMgQWRkaXRpdmUgSW5jcmVhc2UKICAgTXVsdGlwbGljYXRpdmUgRGVj cmVhc2UgY29uZ2VzdGlvbiBjb250cm9sIChBSU1EX0NDKToKCiAgIC0gcXVlcnkoKTogQUlN RF9DQyByZXR1cm5zIHRoZSBjdXJyZW50IGNvbmdlc3Rpb24gd2luZG93IChjd25kKQogICAg IGRpdmlkZWQgYnkgdGhlIHNtb290aGVkIHJ0dCAoc3J0dCkgYXMgaXRzIGJhbmR3aWR0aCBl c3RpbWF0ZS4gSXQKICAgICByZXR1cm5zIHRoZSBzbW9vdGhlZCBydHQgZXN0aW1hdGUgYXMg c3J0dC4KCiAgIC0gbm90aWZ5KCk6IEFJTURfQ0MgYWRkcyB0aGUgbnVtYmVyIG9mIGJ5dGVz IHNlbnQgdG8gaXRzCiAgICAgb3V0c3RhbmRpbmcgZGF0YSB3aW5kb3cgKG93bmQpLgoKICAg LSB1cGRhdGUoKTogQUlNRF9DQyBzdWJ0cmFjdHMgbnNlbnQgZnJvbSBvd25kLiBJZiB0aGUg dmFsdWUgb2YgcnR0CiAgICAgaXMgbm9uLXplcm8sIEFJTURfQ0MgdXBkYXRlcyBzcnR0IHVz aW5nIHRoZSBUQ1Agc3J0dCBjYWxjdWxhdGlvbi4KICAgICBJZiB0aGUgdXBkYXRlIGluZGlj YXRlcyB0aGF0IGRhdGEgaGFzIGJlZW4gbG9zdCwgQUlNRF9DQyBzZXRzCiAgICAgY3duZCB0 byAxIE1UVSBpZiB0aGUgbG9zc19tb2RlIGlzIENNX05PX0ZFRURCQUNLIGFuZCB0byBjd25k LzIKICAgICAod2l0aCBhIG1pbmltdW0gb2YgMSBNVFUpIGlmIHRoZSBsb3NzX21vZGUgaXMg Q01fTE9TU19GRUVEQkFDSyBvcgogICAgIENNX0VYUExJQ0lUX0NPTkdFU1RJT04uICBBSU1E X0NDIGFsc28gc2V0cyBpdHMgaW50ZXJuYWwgc3N0aHJlc2ggdmFyaWFibGUgdG8KICAgICBj d25kLzIuIElmIG5vIGxvc3MgaGFkIG9jY3VycmVkLCBBSU1EX0NDIG1pbWljcyBUQ1Agc2xv dyBzdGFydAogICAgIGFuZCBsaW5lYXIgZ3Jvd3RoIG1vZGVzLiAgSXQgaW5jcmVtZW50cyBj d25kIGJ5IG5zZW50IHdoZW4gY3duZCA8CiAgICAgc3N0aHJlc2ggKGJvdW5kZWQgYnkgYSBt YXhpbXVtIG9mIHNzdGhyZXNoLWN3bmQpIGFuZCBieSBuc2VudCAqCiAgICAgTVRVL2N3bmQg d2hlbiBjd25kID4gc3N0aHJlc2guCgogICAtIFdoZW4gY3duZCBvciBvd25kIGFyZSB1cGRh dGVkIGFuZCBpbmRpY2F0ZSB0aGF0IGF0IGxlYXN0IG9uZSBNVFUKICAgICBtYXkgYmUgdHJh bnNtaXR0ZWQsIEFJTURfQ0MgY2FsbHMgdGhlIENNIHRvIHNjaGVkdWxlIGEKICAgICB0cmFu c21pc3Npb24uCgogICA2LjQgRXhhbXBsZSBTY2hlZHVsZXIgTW9kdWxlCgogICBUbyBjbGFy aWZ5IHRoZSByZXNwb25zaWJpbGl0aWVzIG9mIGEgc2NoZWR1bGVyIG1vZHVsZSwgdGhlCiAg IGZvbGxvd2luZyBkZXNjcmliZXMgc29tZSBvZiB0aGUgYWN0aW9ucyBvZiBhIHNpbXBsZSBy b3VuZCByb2JpbgogICBzY2hlZHVsZXIgbW9kdWxlIChSUl9zY2hlZCk6CgogICAtIHNjaGVk dWxlKCk6IFJSX3NjaGVkIHNjaGVkdWxlcyBhcyBtYW55IHN0cmVhbXMgYXMgcG9zc2libGUg aW4gcm91bmQKICAgICByb2JpbiBmYXNoaW9uLgoKICAgLSBxdWVyeV9zaGFyZSgpOiBSUl9z Y2hlZCByZXR1cm5zIDEvKG51bWJlciBvZiBzdHJlYW1zIGluIG1hY3JvZmxvdykuCgogICAt IG5vdGlmeSgpOiBSUl9zY2hlZCBkb2VzIG5vdGhpbmcuIFJvdW5kIHJvYmluIHNjaGVkdWxp bmcgaXMgbm90CiAgICAgYWZmZWN0ZWQgYnkgdGhlIGFtb3VudCBvZiBkYXRhIHNlbnQuCgo3 LiAgICAgIFNlY3VyaXR5IGNvbnNpZGVyYXRpb25zCgogICBUaGUgQ00gcHJvdmlkZXMgbWFu eSBvZiB0aGUgc2FtZSBzZXJ2aWNlcyB0aGF0IHRoZSBjb25nZXN0aW9uCiAgIGNvbnRyb2wg aW4gVENQIHByb3ZpZGVzLiAgQXMgc3VjaCwgaXQgaXMgdnVsbmVyYWJsZSB0byBtYW55IG9m IHRoZQogICBzYW1lIHNlY3VyaXR5IHByb2JsZW1zLiAgRm9yIGV4YW1wbGUsIGluY29ycmVj dCByZXBvcnRzIG9mIGxvc3NlcwogICBhbmQgdHJhbnNtaXNzaW9ucyB3aWxsIGdpdmUgdGhl IENNIGFuIGluYWNjdXJhdGUgcGljdHVyZSBvZiB0aGUKICAgbmV0d29yaydzIGNvbmdlc3Rp b24gc3RhdGUuICBCeSBnaXZpbmcgQ00gYSBoaWdoIGVzdGltYXRlIG9mCiAgIGNvbmdlc3Rp b24sIGFuIGF0dGFja2VyIGNhbiBkZWdyYWRlIHRoZSBwZXJmb3JtYW5jZSBvYnNlcnZlZCBi eQogICBhcHBsaWNhdGlvbnMuICBUaGUgbW9yZSBkYW5nZXJvdXMgZm9ybSBvZiBhdHRhY2sg aXMgZ2l2aW5nIENNIGEgbG93CiAgIGVzdGltYXRlIG9mIGNvbmdlc3Rpb24uICBUaGlzIHdv dWxkIGNhdXNlIENNIHRvIGJlIG92ZXJseQogICBhZ2dyZXNzaXZlIGFuZCBhbGxvdyBkYXRh IHRvIGJlIHNlbnQgbXVjaCBtb3JlIHF1aWNrbHkgdGhhbiBzb3VuZAogICBjb25nZXN0aW9u IGNvbnRyb2wgcG9saWNpZXMgd291bGQgYWxsb3cuICBbVG91Y2g5N10gZGVzY3JpYmVzIHRo ZQogICBzZWN1cml0eSBwcm9ibGVtcyB0aGF0IGFyaXNlIHdpdGggY29uZ2VzdGlvbiBpbmZv cm1hdGlvbiBzaGFyaW5nIGluCiAgIG1vcmUgZGV0YWlsLgoKOC4gICAgICBSZWZlcmVuY2Vz CgogICBbQWxsbWFuOTldIEFsbG1hbiwgTS4gYW5kIFBheHNvbiwgVi4sICBUQ1AgQ29uZ2Vz dGlvbiBDb250cm9sLAogICBSRkMtMjU4MSwgQXByaWwgMTk5OS4KCiAgIFtBbmRlcnNlbjAw XSBBbmRlcnNlbiwgRC4sIEJhbnNhbCwgRC4sIEN1cnRpcywgRC4sIFNlc2hhbiwgUy4sIGFu ZAogICAgICBCYWxha3Jpc2huYW4sIEguLCBTeXN0ZW0gU3VwcG9ydCBmb3IgQmFuZHdpZHRo IE1hbmFnZW1lbnQgYW5kCiAgICAgIENvbnRlbnQgQWRhcHRhdGlvbiBpbiBJbnRlcm5ldCBB cHBsaWNhdGlvbnMsIFByb2MuIDR0aCBTeW1wLiBvbgogICAgICBPcGVyYXRpbmcgU3lzdGVt cyBEZXNpZ24gYW5kIEltcGxlbWVudGF0aW9uLCBTYW4gRGllZ28sIENBLAogICAgICBPY3Rv YmVyIDIwMDAuICBBdmFpbGFibGUgZnJvbQogICAgICBodHRwOi8vbm1zLmxjcy5taXQuZWR1 L3BhcGVycy9jbS1vc2RpMjAwMC5odG1sCgogICBbQmFsYWtyaXNobmFuOThdIEJhbGFrcmlz aG5hbiwgSC4sIFBhZG1hbmFiaGFuLCBWLiwgU2VzaGFuLCBTLiwKICAgICAgU3RlbW0sIE0u LCBhbmQgS2F0eiwgUi4sICJUQ1AgQmVoYXZpb3Igb2YgYSBCdXN5IFdlYiBTZXJ2ZXI6CiAg ICAgIEFuYWx5c2lzIGFuZCBJbXByb3ZlbWVudHMsIiBQcm9jLiBJRUVFIElORk9DT00sIFNh biBGcmFuY2lzY28sCiAgICAgIENBLCBNYXJjaCAxOTk4LgoKICAgW0JhbGFrcmlzaG5hbjk5 XSBCYWxha3Jpc2huYW4sIEguLCBSYWh1bCwgSC4sIGFuZCBTZXNoYW4sIFMuLCAiQW4KICAg ICAgSW50ZWdyYXRlZCBDb25nZXN0aW9uIE1hbmFnZW1lbnQgQXJjaGl0ZWN0dXJlIGZvciBJ bnRlcm5ldAogICAgICBIb3N0cywiIFByb2MuIEFDTSBTSUdDT01NLCBDYW1icmlkZ2UsIE1B LCBTZXB0ZW1iZXIgMTk5OS4KCiAgIFtCcmFkbmVyOTZdIEJyYWRuZXIsIFMuLCAiVGhlIElu dGVybmV0IFN0YW5kYXJkcyBQcm9jZXNzIC0tLQogICAgICBSZXZpc2lvbiAzIiwgQkNQIDks IFJGQy0yMDI2LCBPY3RvYmVyIDE5OTYuCgogICBbQnJhZG5lcjk3XSBCcmFkbmVyLCBTLiwg IktleSB3b3JkcyBmb3IgdXNlIGluIFJGQ3MgdG8gSW5kaWNhdGUKICAgICAgUmVxdWlyZW1l bnQgTGV2ZWxzIiwgQkNQIDE0LCBSRkMtMjExOSwgTWFyY2ggMTk5Ny4KCiAgIFtDbGFyazkw XSBDbGFyaywgRC4gYW5kIFRlbm5lbmhvdXNlLCBELiwgIkFyY2hpdGVjdHVyYWwKICAgICAg Q29uc2lkZXJhdGlvbiBmb3IgYSBOZXcgR2VuZXJhdGlvbiBvZiBQcm90b2NvbHMiLCBQcm9j LiBBQ00KICAgICAgU0lHQ09NTSwgUGhpbGFkZWxwaGlhLCBQQSwgU2VwdGVtYmVyIDE5OTAu CgogICBbRWdnZXJ0MDBdIEVnZ2VydCwgTC4sIEhlaWRlbWFubiwgSi4sIGFuZCBUb3VjaCwg Si4sICJFZmZlY3RzIG9mCiAgICAgIEVuc2VtYmxlIFRDUCwiIEFDTSBDb21wdXRlciBDb21t LiBSZXZpZXcsIEphbnVhcnkgMjAwMC4KCiAgIFtGbG95ZDk5YV0gRmxveWQsIFMuIGFuZCBG YWxsLCBLLiwiIFByb21vdGluZyB0aGUgVXNlIG9mIEVuZC10by1FbmQKICAgICAgIENvbmdl c3Rpb24gQ29udHJvbCBpbiB0aGUgSW50ZXJuZXQsIiBJRUVFL0FDTSBUcmFucy4gb24KICAg ICAgIE5ldHdvcmtpbmcsIDcoNCksIEF1Z3VzdCAxOTk5LCBwcC4gNDU4LTQ3Mi4KCiAgIFtG bG95ZDk5Yl0gRmxveWQsIFMuIGFuZCBIZW5kZXJzb24sIFQuLCAiVGhlIE5ld1Jlbm8gTW9k aWZpY2F0aW9uCiAgICAgIHRvIFRDUCdzIEZhc3QgUmVjb3ZlcnkgQWxnb3JpdGhtLCIgUkZD LTI1ODIsIEFwcmlsCiAgICAgIDE5OTkuIChFeHBlcmltZW50YWwuKQoKICAgW0phY29ic29u ODhdIEphY29ic29uLCBWLiwgIkNvbmdlc3Rpb24gQXZvaWRhbmNlIGFuZCBDb250cm9sLCIK ICAgICAgUHJvYy4gQUNNIFNJR0NPTU0sIFN0YW5mb3JkLCBDQSwgQXVndXN0IDE5ODguCgog ICBbTWFoZGF2aTk4XSBNYWhkYXZpLCBKLiBhbmQgRmxveWQsIFMuLCAiVGhlIFRDUCBGcmll bmRseSBXZWJzaXRlLCIKICAgICAgaHR0cDovL3d3dy5wc2MuZWR1L25ldHdvcmtpbmcvdGNw X2ZyaWVuZGx5Lmh0bWwKCiAgIFtNb2d1bDkwXSBNb2d1bCwgSi4gYW5kIERlZXJpbmcsIFMu LCAiUGF0aCBNVFUgRGlzY292ZXJ5LCIKICAgICAgUkZDLTExOTEsIE5vdmVtYmVyIDE5OTAu CgogICBbUGFkbWFuYWJoYW45OF0gUGFkbWFuYWJoYW4sIFYuLCAiQWRkcmVzc2luZyB0aGUg Q2hhbGxlbmdlcyBvZiBXZWIKICAgICAgRGF0YSBUcmFuc3BvcnQsIiBQaEQgdGhlc2lzLCBV bml2LiBvZiBDYWxpZm9ybmlhLCBCZXJrZWxleSwKICAgICAgRGVjZW1iZXIgMTk5OC4KCiAg IFtQYXhzb24wMF0gUGF4c29uLiBWLiBhbmQgQWxsbWFuLCBNLiwgIkNvbXB1dGluZyBUQ1An cwogICAgICBSZXRyYW5zbWlzc2lvbiBUaW1lciwiIEludGVybmV0IERyYWZ0CiAgICAgIGRy YWZ0LXBheHNvbi10Y3AtcnRvLTAxLnR4dCwgQXByaWwgMjAwMC4gIChFeHBpcmVzIE9jdG9i ZXIKICAgICAgMjAwMC4pCgogICBbUG9zdGVsODFdIFBvc3RlbCwgSi4gKGVkLiksICJUcmFu c21pc3Npb24gQ29udHJvbCBQcm90b2NvbCwiCiAgICAgIFJGQy03OTMsIFNlcHRlbWJlciAx OTgxLgoKICAgW1JhbWFrcmlzaG5hbjk4XSBSYW1ha3Jpc2huYW4sIEsuIGFuZCBGbG95ZCwg Uy4sICJBIFByb3Bvc2FsIHRvIEFkZAogICAgICBFeHBsaWNpdCBDb25nZXN0aW9uIE5vdGlm aWNhdGlvbiAoRUNOKSB0byBJUCwiIFJGQy0yNDgxLgogICAgICAoRXhwZXJpbWVudGFsLikK CiAgIFtTdGV2ZW5zOTRdIFN0ZXZlbnMsIFcuLCBUQ1AvSVAgSWxsdXN0cmF0ZWQsIFZvbHVt ZSAxLgogICAgICBBZGRpc29uLVdlc2xleSwgUmVhZGluZywgTUEsIDE5OTQuIAogICAgICAK ICAgW1RvdWNoOTddIFRvdWNoLCBKLiwgIlRDUCBDb250cm9sIEJsb2NrIEludGVyZGVwZW5k ZW5jZSwiIFJGQy0yMTQwLAogICAgICBBcHJpbCAxOTk3LiAoSW5mb3JtYXRpb25hbC4pCgo5 LiAgICAgIEFja25vd2xlZGdtZW50cwoKICAgV2UgdGhhbmsgRGF2aWQgQW5kZXJzZW4sIERl ZXBhayBCYW5zYWwsIGFuZCBEb3JvdGh5IEN1cnRpcyBmb3IKICAgdGhlaXIgd29yayBvbiB0 aGUgQ00gZGVzaWduIGFuZCBpbXBsZW1lbnRhdGlvbi4gIFdlIHRoYW5rIFZlcm4KICAgUGF4 c29uIGZvciBoaXMgZGV0YWlsZWQgY29tbWVudHMgYW5kIHBhdGllbmNlLCBhbmQgU2FsbHkg RmxveWQsCiAgIE1hcmsgSGFuZGxleSwgYW5kIFN0ZXZlbiBNY0Nhbm5lIGZvciB1c2VmdWwg ZmVlZGJhY2sgb24gdGhlIENNCiAgIGFyY2hpdGVjdHVyZS4KCjEwLiAgICAgQXV0aG9ycycg YWRkcmVzc2VzCgogICBIYXJpIEJhbGFrcmlzaG5hbgogICBMYWJvcmF0b3J5IGZvciBDb21w dXRlciBTY2llbmNlCiAgIDU0NSBUZWNobm9sb2d5IFNxdWFyZQogICBNYXNzYWNodXNldHRz IEluc3RpdHV0ZSBvZiBUZWNobm9sb2d5CiAgIENhbWJyaWRnZSwgTUEgMDIxMzkKICAgRW1h aWw6IGhhcmlAbGNzLm1pdC5lZHUKICAgV2ViOiBodHRwOi8vbm1zLmxjcy5taXQuZWR1L35o YXJpLwoKICAgU3Jpbml2YXNhbiBTZXNoYW4KICAgU2Nob29sIG9mIENvbXB1dGVyIFNjaWVu Y2UKICAgQ2FybmVnaWUgTWVsbG9uIFVuaXZlcnNpdHkKICAgNTAwMCBGb3JiZXMgQXZlLgog ICBQaXR0c2J1cmdoLCBQQSAxNTIxMwogICBFbWFpbDogc3JpbmlAY211LmVkdQogICBXZWI6 IGh0dHA6Ly93d3cuY3MuY211LmVkdS9+c3JpbmkvCgoKRnVsbCBDb3B5cmlnaHQgU3RhdGVt ZW50CgogICAiQ29weXJpZ2h0IChDKSBUaGUgSW50ZXJuZXQgU29jaWV0eSAoZGF0ZSkuIEFs bCBSaWdodHMgUmVzZXJ2ZWQuCiAgIFRoaXMgZG9jdW1lbnQgYW5kIHRyYW5zbGF0aW9ucyBv ZiBpdCBtYXkgYmUgY29waWVkIGFuZCBmdXJuaXNoZWQgdG8KICAgb3RoZXJzLCBhbmQgZGVy aXZhdGl2ZSB3b3JrcyB0aGF0IGNvbW1lbnQgb24gb3Igb3RoZXJ3aXNlIGV4cGxhaW4KICAg aXQgb3IgYXNzaXN0IGluIGl0cyBpbXBsZW1lbnRhdGlvbiBtYXkgYmUgcHJlcGFyZWQsIGNv cGllZCwKICAgcHVibGlzaGVkIGFuZCBkaXN0cmlidXRlZCwgaW4gd2hvbGUgb3IgaW4gcGFy dCwgd2l0aG91dCByZXN0cmljdGlvbgogICBvZiBhbnkga2luZCwgcHJvdmlkZWQgdGhhdCB0 aGUgYWJvdmUgY29weXJpZ2h0IG5vdGljZSBhbmQgdGhpcwogICBwYXJhZ3JhcGggYXJlIGlu Y2x1ZGVkIG9uIGFsbCBzdWNoIGNvcGllcyBhbmQgZGVyaXZhdGl2ZSB3b3Jrcy4KICAgSG93 ZXZlciwgdGhpcyBkb2N1bWVudCBpdHNlbGYgbWF5IG5vdCBiZSBtb2RpZmllZCBpbiBhbnkg d2F5LCBzdWNoCiAgIGFzIGJ5IHJlbW92aW5nIHRoZSBjb3B5cmlnaHQgbm90aWNlIG9yIHJl ZmVyZW5jZXMgdG8gdGhlIEludGVybmV0CiAgIFNvY2lldHkgb3Igb3RoZXIgSW50ZXJuZXQg b3JnYW5pemF0aW9ucywgZXhjZXB0IGFzIG5lZWRlZCBmb3IgdGhlCiAgIHB1cnBvc2Ugb2Yg ZGV2ZWxvcGluZyBJbnRlcm5ldCBzdGFuZGFyZHMgaW4gd2hpY2ggY2FzZSB0aGUKICAgcHJv Y2VkdXJlcyBmb3IgY29weXJpZ2h0cyBkZWZpbmVkIGluIHRoZSBJbnRlcm5ldCBTdGFuZGFy ZHMgcHJvY2VzcwogICBtdXN0IGJlIGZvbGxvd2VkLCBvciBhcyByZXF1aXJlZCB0byB0cmFu c2xhdGUgaXQgaW50byB0aGUgZmluYWwKICAgZHJhZnQgb3V0cHV0Lgo= --==_Exmh_-20288952360-- From owner-ecm@wyvern.aciri.org Mon Oct 16 01:23:57 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id BAA16963 for ; Mon, 16 Oct 2000 01:14:12 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id WAA76426 for ecm-outgoing; Sun, 15 Oct 2000 22:11:51 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id WAA76420 for ; Sun, 15 Oct 2000 22:11:50 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: (from hari@localhost) by breeze.lcs.mit.edu (8.11.0/8.9.3) id e9G4ZlW23425; Mon, 16 Oct 2000 00:35:47 -0400 Date: Mon, 16 Oct 2000 00:35:47 -0400 From: Hari Balakrishnan Message-Id: <200010160435.e9G4ZlW23425@breeze.lcs.mit.edu> To: ecm@aciri.org, end2end-interest@isi.edu Subject: Congestion Manager code release (alpha code for Linux) Cc: cm@wind.lcs.mit.edu Sender: owner-ecm@aciri.org Precedence: bulk This note is to announce the alpha release of Version 1 of the Congestion Manager (CM) software for Linux. This code is available under a GPL license (and an MIT copyright) from http://nms.lcs.mit.edu/software/CM/ Q: What is the CM? A: The CM is an end-system module that: (i) Enables an ensemble of multiple concurrent streams from a sender destined to the same receiver and sharing the same congestion properties to perform proper congestion avoidance and control. It is independent of particular transport or application protocols, and unifies congestion control for an ensemble of flows. (ii) Allows applications to easily adapt to network congestion by providing a platform-independent adaptation API. Q: Where can I learn more about it? A: http://nms.lcs.mit.edu/projects/cm/ has more information, including a paper or two, including an evolving Internet draft of the IETF ECM working group (http://www.ietf.org/html.charters/ecm-charter.html). http://nms.lcs.mit.edu/papers/draft-ietf-ecm-cm-02.txt has the current version of this draft. ecm@aciri.org is the mailing list of the ECM WG (to subscribe, send a request to the majordomo at ecm-request@aciri.org). It is perhaps best if discussions about this software and ECM happen on that list, rather than on both ecm and end2end-interest. Q: What does this code release contain? A: A distribution for Linux (2.2.9), an adaptation API library, and a few sample applications. The kernel includes a TCP that exclusively uses the CM for congestion control (for a per-destination connection ensemble). Several simple user-level applications, including some ALF-based ones, are included. Yes, somewhat surprisingly, it also includes some reasonable documentation, which we hope (perhaps wrongly) will be useful! Please send any bugs reports/fixes, feedback, comments, suggestions, code contributions, etc. on the CM software to cm@nms.lcs.mit.edu Thanks, Hari (for the CM dev team) From owner-ecm@wyvern.aciri.org Mon Oct 16 07:44:48 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA02694 for ; Mon, 16 Oct 2000 07:44:48 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id EAA78349 for ecm-outgoing; Mon, 16 Oct 2000 04:44:03 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id EAA78344 for ; Mon, 16 Oct 2000 04:44:02 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: from breeze.lcs.mit.edu (IDENT:hari@localhost.localdomain [127.0.0.1]) by breeze.lcs.mit.edu (8.11.0/8.9.3) with ESMTP id e9GBi0v25413; Mon, 16 Oct 2000 07:44:00 -0400 Message-Id: <200010161144.e9GBi0v25413@breeze.lcs.mit.edu> X-Mailer: exmh version 2.1.1 10/15/1999 To: Scott Bradner cc: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document In-Reply-To: Message from Scott Bradner of "Mon, 16 Oct 2000 07:21:22 EDT." <200010161121.HAA00404@newdev.harvard.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 16 Oct 2000 07:44:00 -0400 From: Hari Balakrishnan Sender: owner-ecm@aciri.org Precedence: bulk > > It hasn't been > > submitted to the IETF yet; the intention is to get some discussion going and > > obtain comments well in advance of the deadline of the next IETF. > > comments should be done on the basis of published Internet Drafts - please > submit this one ASAP - if you are in a position to update it before > the deadline then please do but get this one into the normal IETF > document process > > Scott Scott, Sorry for the faux pas. It (draft-ietf-ecm-cm-02.txt) has now been submitted. Thanks, Hari From owner-ecm@wyvern.aciri.org Tue Oct 17 06:50:07 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id GAA07194 for ; Tue, 17 Oct 2000 06:50:05 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id DAA86084 for ecm-outgoing; Tue, 17 Oct 2000 03:46:11 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from ietf.org (odin.ietf.org [132.151.1.176]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id DAA86079 for ; Tue, 17 Oct 2000 03:46:10 -0700 (PDT) (envelope-from nsyracus@cnri.reston.va.us) Received: from CNRI.Reston.VA.US (localhost [127.0.0.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA06973; Tue, 17 Oct 2000 06:46:08 -0400 (EDT) Message-Id: <200010171046.GAA06973@ietf.org> Mime-Version: 1.0 Content-Type: Multipart/Mixed; Boundary="NextPart" To: IETF-Announce: ; Cc: ecm@aciri.org From: Internet-Drafts@ietf.org Reply-to: Internet-Drafts@ietf.org Subject: I-D ACTION:draft-ietf-ecm-cm-02.txt Date: Tue, 17 Oct 2000 06:46:08 -0400 Sender: owner-ecm@aciri.org Precedence: bulk --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Endpoint Congestion Management Working Group of the IETF. Title : The Congestion Manager Author(s) : H. Balakrishnan, S. Seshan Filename : draft-ietf-ecm-cm-02.txt Pages : 17 Date : 16-Oct-00 This document describes the Congestion Manager (CM), an end-system module that: (i) Enables an ensemble of multiple concurrent streams from a sender destined to the same receiver and sharing the same congestion properties to perform proper congestion avoidance and control, and (ii) Allows applications to easily adapt to network congestion. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-ietf-ecm-cm-02.txt Internet-Drafts are also available by anonymous FTP. Login with the username "anonymous" and a password of your e-mail address. After logging in, type "cd internet-drafts" and then "get draft-ietf-ecm-cm-02.txt". A list of Internet-Drafts directories can be found in http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt Internet-Drafts can also be obtained by e-mail. Send a message to: mailserv@ietf.org. In the body type: "FILE /internet-drafts/draft-ietf-ecm-cm-02.txt". NOTE: The mail server at ietf.org can return the document in MIME-encoded form by using the "mpack" utility. To use this feature, insert the command "ENCODING mime" before the "FILE" command. To decode the response(s), you will need "munpack" or a MIME-compliant mail reader. Different MIME-compliant mail readers exhibit different behavior, especially when dealing with "multipart" MIME messages (i.e. documents which have been split up into multiple messages), so check your local documentation on how to manipulate these messages. Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Multipart/Alternative; Boundary="OtherAccess" --OtherAccess Content-Type: Message/External-body; access-type="mail-server"; server="mailserv@ietf.org" Content-Type: text/plain Content-ID: <20001016145459.I-D@ietf.org> ENCODING mime FILE /internet-drafts/draft-ietf-ecm-cm-02.txt --OtherAccess Content-Type: Message/External-body; name="draft-ietf-ecm-cm-02.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <20001016145459.I-D@ietf.org> --OtherAccess-- --NextPart-- From owner-ecm@wyvern.aciri.org Tue Oct 17 10:26:22 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA12765 for ; Tue, 17 Oct 2000 10:26:21 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id HAA87019 for ecm-outgoing; Tue, 17 Oct 2000 07:24:19 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id HAA87014 for ; Tue, 17 Oct 2000 07:24:18 -0700 (PDT) (envelope-from touch@ISI.EDU) Received: from isi.edu (ras01.isi.edu [128.9.176.101]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id HAA08560; Tue, 17 Oct 2000 07:24:15 -0700 (PDT) Message-ID: <39EC5DD2.D1AA8870@isi.edu> Date: Tue, 17 Oct 2000 07:10:26 -0700 From: Joe Touch X-Mailer: Mozilla 4.74 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: Hari Balakrishnan CC: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document References: <200010160159.e9G1xBv20056@breeze.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Hari Balakrishnan wrote: > > On Thu, 03 Aug 2000 00:33:20 PDT, Vern Paxson wrote: > > > The chair then made the following proposal for moving forward with ecm-cm-01: > > > > 5. Resolve the issue Joe raised regarding sending multiple packets. > > The issue here is to make the cm_request(id) call, which currently takes no > other arguments, richer. In particular, the suggestion is: > cm_request(id, npkts, size) > where npkts is the number of packets to be sent and size the size of each > packet. The apparent reason for this is the smaller number of cm_request() > calls. > > I tried to articulate (at the last IETF) why this may not be a good idea. > Here's the reasoning: > > 1. The current API is not as inefficient as one might believe. It may be > tempting to think that the current API causes a user-kernel crossing for each > packet. That's not necessarily true. A user-level library that implements the > API can package multiple such calls into one system call. For in-kernel apps > like TCP, cm_request(id) is only a function call. And if you believe that a > function call is expensive, inlining it (and inlining the in-kernel function > callback) is pretty straightforward. It may be difficult to inline functions in loadable kernel modules. Further, beyond the function call overhead, there is the per-call processing overhead. Taking this overhead (or any overhead) on a per-packet basis is asking for substantial performance penalty. > 2. If we add a "MAY" clause for cm_request(id, npkts, size), it MUST require > that applications negotiate the specific supported mode in the API. The proposed extension to the API contains two parameters: size: this is consistent with the byte-counting proposal for TCP. npkts: this allows per-API-call processing to occur on timescales coarser than per-packet. The issue is being able to amortize the processing on the API; inlining isn't sufficient to provide amortization. An application that desires to use the current interface can use size=MTU and npkts=1. > I am not sure at all that complicating the API > to accommodate this, in the absence of significant experience that dictates > that it's necessary, is a good idea. That experience is in abundance; per-packet overhead should be avoided, especially where not otherwise necessary. If you want a particular system not to support it (i.e., to require npkts to be 1), that can be decided on a per-system basis. Assuming how many bytes are in a packet or how many packets should be sent between API calls is the assumption to be avoided. > Note that several non-ALF applications may not care to receive a callback for > each transmission. ALF applications are more likely to require the kind of performance that cannot take additional per-packet processing hits. Joe From owner-ecm@wyvern.aciri.org Tue Oct 17 14:44:07 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA19106 for ; Tue, 17 Oct 2000 14:44:07 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id LAA89684 for ecm-outgoing; Tue, 17 Oct 2000 11:43:09 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from ux10.sp.cs.cmu.edu (UX10.SP.CS.CMU.EDU [128.2.209.197]) by wyvern.aciri.org (8.9.3/8.9.3) with SMTP id LAA89679 for ; Tue, 17 Oct 2000 11:43:08 -0700 (PDT) (envelope-from Srinivasan_Seshan@ux10.sp.cs.cmu.edu) Message-Id: <200010171843.LAA89679@wyvern.aciri.org> Received: from ux10.sp.cs.cmu.edu by ux10.sp.cs.cmu.edu id aa03878; 17 Oct 2000 14:42 EDT X-Mailer: exmh version 2.0.1 12/23/97 From: Srinivasan Seshan Reply-to: Srinivasan Seshan X-Face: %nh^b4o^oekj}G4Wz=R.U%-Fj?^fD=Lh+|r@*>xYfNf}h5,9LQe~KzAa\03UoV> mpi!Ac`& t@[@V2Hk_(y+8_l:On:W$Mb4U]h_VHT^obq%'.4X5?k:5:iI@*`_42UZ6vw>x$DbBci;+iuw_v"\Tk NmZo{fUZ*B9+I"}{p=zi$c+YPRzU+g)yiQ?YE$UXVPw^Tnj8OB?MK,LOt>GnQFOoHX},TwNL*]]q:7 A#/4Q_[[.HK?Ofc#?&3u$%prg&(HU1kmbn@XmKA8c'cufs0cE%3*WZbF4;[P;la1^}wy`vlg%e To: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 17 Oct 2000 14:42:49 -0400 Sender: owner-ecm@aciri.org Precedence: bulk > > Joe Touch wrote: > > > > On Thu, 03 Aug 2000 00:33:20 PDT, Vern Paxson wrote: > > > > > > The issue here is to make the cm_request(id) call, which currently takes no > > other arguments, richer. In particular, the suggestion is: > > cm_request(id, npkts, size) > > > > I tried to articulate (at the last IETF) why this may not be a good idea. > > Here's the reasoning: > > > > 1. The current API is not as inefficient as one might believe. It may be > > tempting to think that the current API causes a user-kernel crossing for ea > > packet. That's not necessarily true. A user-level library that implements > > API can package multiple such calls into one system call. For in-kernel ap > > like TCP, cm_request(id) is only a function call. And if you believe that > > function call is expensive, inlining it (and inlining the in-kernel functio > > callback) is pretty straightforward. > > It may be difficult to inline functions in loadable kernel modules. > Agreed.. > Further, beyond the function call overhead, there is the per-call > processing overhead. Taking this overhead (or any overhead) on > a per-packet basis is asking for substantial performance penalty. > However, the amount of per packet processing done within the CM is minimal compared to the other per packet processing done by a host (e.g. it is significantly less than the per packet processing done by TCP currently). As a result, it should not be a significant penalty. > > 2. If we add a "MAY" clause for cm_request(id, npkts, size), it MUST requir > > that applications negotiate the specific supported mode in the API. > > The proposed extension to the API contains two parameters: > > size: this is consistent with the byte-counting proposal > for TCP. > > npkts: this allows per-API-call processing to occur on > timescales coarser than per-packet. The issue is > being able to amortize the processing on the API; > inlining isn't sufficient to provide amortization. > > An application that desires to use the current interface can use > size=MTU and npkts=1. > The concern here is about the "MAY". This is bad wording to have in an API document. The issue is not that your API can support simpler interactions but that the MAY forces all applications to support both versions of API. E.g. I write an application that has cm_request( id, 10 pkts, 10KB)... Because of the MAY, I must now write a version of the application that uses the cm_request(id) only and a set of configure scripts that chooses between the two. If the wording should use MUST, I believe that the we should (at least for now) stick with the simpler interface. > > I am not sure at all that complicating the API > > to accommodate this, in the absence of significant experience that dictates > > that it's necessary, is a good idea. > > That experience is in abundance; per-packet overhead should be avoided, > especially where not otherwise necessary. ... and where you do not give up any other benefits as a result. One important aspect of the CM, is that it is supposed to effectively shape/control the transmission of data. By allowing application to send bursts of data it gives up much of this control. In addition, scheduling such variable sized bursts may also prove to be a non-trivial problem. This seems to have much greater future implications than the per-packet processing overheads. > > If you want a particular system not to support it (i.e., to require > npkts to be 1), that can be decided on a per-system basis. Assuming how > many bytes are in a packet or how many packets should be sent between > API calls is the assumption to be avoided. > Srini and Hari From owner-ecm@wyvern.aciri.org Tue Oct 17 15:31:16 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA19751 for ; Tue, 17 Oct 2000 15:31:15 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id MAA89910 for ecm-outgoing; Tue, 17 Oct 2000 12:30:54 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id MAA89905 for ; Tue, 17 Oct 2000 12:30:54 -0700 (PDT) (envelope-from touch@ISI.EDU) Received: from isi.edu (ras01.isi.edu [128.9.176.101]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id MAA26123; Tue, 17 Oct 2000 12:30:50 -0700 (PDT) Message-ID: <39ECA890.87A5A7D9@isi.edu> Date: Tue, 17 Oct 2000 12:29:20 -0700 From: Joe Touch X-Mailer: Mozilla 4.74 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: Srinivasan Seshan CC: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document References: <200010171843.LAA89679@wyvern.aciri.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Srinivasan Seshan wrote: > > > > > Joe Touch wrote: > > > > > > On Thu, 03 Aug 2000 00:33:20 PDT, Vern Paxson wrote: > > > > > > > > > The issue here is to make the cm_request(id) call, which currently takes no > > > other arguments, richer. In particular, the suggestion is: > > > cm_request(id, npkts, size) > > > > > > I tried to articulate (at the last IETF) why this may not be a good idea. > > > Here's the reasoning: > > > > > > 1. The current API is not as inefficient as one might believe. It may be > > > tempting to think that the current API causes a user-kernel crossing for ea > > > packet. That's not necessarily true. A user-level library that implements > > > API can package multiple such calls into one system call. For in-kernel ap > > > like TCP, cm_request(id) is only a function call. And if you believe that > > > function call is expensive, inlining it (and inlining the in-kernel functio > > > callback) is pretty straightforward. > > > > It may be difficult to inline functions in loadable kernel modules. > > > > Agreed.. > > > Further, beyond the function call overhead, there is the per-call > > processing overhead. Taking this overhead (or any overhead) on > > a per-packet basis is asking for substantial performance penalty. > > > > However, the amount of per packet processing done within the CM is minimal > compared to the other per packet processing done by a host (e.g. it is > significantly less than the per packet processing done by TCP currently). As a > result, it should not be a significant penalty. I've heard these arguments before. In the absence of evidence that CM is not a significant penalty (i.e., emperical evidence of the performance of running code), it is wise to err on the side of caution here. > > > > 2. If we add a "MAY" clause for cm_request(id, npkts, size), it MUST requir > > > that applications negotiate the specific supported mode in the API. > > > > The proposed extension to the API contains two parameters: > > > > size: this is consistent with the byte-counting proposal > > for TCP. > > > > npkts: this allows per-API-call processing to occur on > > timescales coarser than per-packet. The issue is > > being able to amortize the processing on the API; > > inlining isn't sufficient to provide amortization. > > > > An application that desires to use the current interface can use > > size=MTU and npkts=1. > > > > The concern here is about the "MAY". This is bad wording to have in an API > document. The issue is not that your API can support simpler interactions but > that the MAY forces all applications to support both versions of API. E.g. I > write an application that has cm_request( id, 10 pkts, 10KB)... Because of the > MAY, I must now write a version of the application that uses the > cm_request(id) only and a set of configure scripts that chooses between the > two. If the wording should use MUST, I believe that the we should (at least > for now) stick with the simpler interface. The solution is to not provide the cm_request(id) format; as you point out, any provision for two interfaces is asking for trouble. The better solution is the more general interface. Applications that can make due with the simpler version can call it as I indicated, or use a macro in the code of the API. I.e., then the cm_request(id, npkts, size) should be the MUST; the simpler version should be the MAY. > > > I am not sure at all that complicating the API > > > to accommodate this, in the absence of significant experience that dictates > > > that it's necessary, is a good idea. > > > > That experience is in abundance; per-packet overhead should be avoided, > > especially where not otherwise necessary. > > ... and where you do not give up any other benefits as a result. One important > aspect of the CM, is that it is supposed to effectively shape/control the > transmission of data. By allowing application to send bursts of data it gives > up much of this control. In addition, scheduling such variable sized bursts > may also prove to be a non-trivial problem. This seems to have much greater > future implications than the per-packet processing overheads. There are two issues here: 1. size assuming MTU is imprecise anyway 2. npkts making this larger does increase granularity of the API whether an implementation of the API supports this should be up to the API it is overrestrictive to assume that CM must be tickled for every packet going out the door, regardless of the size there are numerous cases where such bursts are desirable, and might be impeded by the current interface, e.g., bursts in gigabit ethernet, and the small-scale burstiness of TCP (sending multiple packets per ACK). Joe From owner-ecm@wyvern.aciri.org Tue Oct 17 19:25:59 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22538 for ; Tue, 17 Oct 2000 19:25:57 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id QAA91221 for ecm-outgoing; Tue, 17 Oct 2000 16:25:10 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id QAA91216 for ; Tue, 17 Oct 2000 16:25:09 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: from breeze.lcs.mit.edu (IDENT:hari@localhost.localdomain [127.0.0.1]) by breeze.lcs.mit.edu (8.11.0/8.9.3) with ESMTP id e9HNP6v00896; Tue, 17 Oct 2000 19:25:07 -0400 Message-Id: <200010172325.e9HNP6v00896@breeze.lcs.mit.edu> X-Mailer: exmh version 2.1.1 10/15/1999 X-url: http://nms.lcs.mit.edu/~hari/ From: Hari Balakrishnan Reply-to: Hari Balakrishnan To: touch@isi.edu cc: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document (fwd) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 17 Oct 2000 19:25:06 -0400 Sender: owner-ecm@aciri.org Precedence: bulk Joe, > I've heard these arguments before. In the absence of evidence that CM > is not a significant penalty (i.e., emperical evidence of the > performance > of running code), it is wise to err on the side of caution here. There are some running code measurements in the OSDI 2000 paper. In addition there is now a code release. Hopefully, these should reduce the concerns of overhead from cm_request(). > The solution is to not provide the cm_request(id) format; > as you point out, any provision for two interfaces is asking for > trouble. > The better solution is the more general interface. Applications that > can make due with the simpler version can call it as I indicated, or > use a macro in the code of the API. Yep, your interface is more general. However, purely being more general does not necessarily make it more appropriate. The implication of this design choice on implementation and on future design choices must be considered. For example, it will always be easier to move application code from the simpler interface to the more general interface through: #define cm_request(id) cm_request(id, 1, MTU) > > > > I am not sure at all that complicating the API > > > > to accommodate this, in the absence of significant experience that dictates > > > > that it's necessary, is a good idea. > > > > > > That experience is in abundance; per-packet overhead should be avoided, > > > especially where not otherwise necessary. > > > > ... and where you do not give up any other benefits as a result. One important > > aspect of the CM, is that it is supposed to effectively shape/control the > > transmission of data. By allowing application to send bursts of data it gives > > up much of this control. In addition, scheduling such variable sized bursts > > may also prove to be a non-trivial problem. This seems to have much greater > > future implications than the per-packet processing overheads. > > There are two issues here: > > 1. size > assuming MTU is imprecise anyway This was chosen as the balance between making it hard on the application to know a priori the exact amount of information being sent and CM needing to control the transmission precisely. This was based on a similar choice by existing protocols like TCP, and by the preferred mode for ALF (as described in Clark's 1990 paper). However, do note that cm_notify & cm_updates are on the exact number of bytes sent. Therefore, the CM is able to keep a precise account of what's going on, within a short time delay (the time between callback and transmission). > > 2. npkts > making this larger does increase granularity of the API > whether an implementation of the API supports this > should be up to the API > > it is overrestrictive to assume that CM must be tickled > for every packet going out the door, regardless of the size > > there are numerous cases where such bursts are desirable, > and might be impeded by the current interface, e.g., > bursts in gigabit ethernet, and the small-scale burstiness of > TCP (sending multiple packets per ACK). > I assume that even though the application requests npkts, the callback can return anywhere between 1 and npkts as sendable now (or does a callback indicate that all npkts are sendable?). If so, then this would rather complicate both the CM and the application. E.g. how does the CM decide when it should do the callback---especially when there are multiple applications all with differently-sized requests? E.g., if the CM has 2 applications, 1 asking for 5 pkts and the other asking for 1. When a 2 pkts become available in the congestion window, should it wait for 5 to become available or just satisfy the smaller request. Also, suppose an adaptive application wanted to send 10 pkts and got a callback for 2 pkts - what should its adaptation decision be? Does it need revoke its request 8 more pkts (i.e., is there a "revoke" call)? Historical note: In fact, we had npkts in the API for about 3-4 months inthe early stages of design, until we were convinced otherwise (my recollection is that we were going back and forth on it, and that Steve McCanne and Mark Handley persuasively convinced us to make the simpler decision). In addition, with the npkts added we also found it necessary to add min_bytes (actually our original interface was cm_request(id, nbytes, min_bytes)). This was in order to help the scheduler decide when to do callbacks. A callback had to indicate a value between min_bytes to nbytes that could be transmitted. We were convinced that this provided little gain to applications and would complicate the internals significantly. We chose the path of lower implementation complexity. This approach is in fact not new by any means; UNIX did this routinely in their system. See, for example, Richard P. Gabriel, Worse is better. From LISP: good news, bad news, how to win BIG, AI Expert 6, 6 (June, 1991) pages 33-35 for an interesting description of this in the context of UNIX and LISP. Hari & Srini From owner-ecm@wyvern.aciri.org Tue Oct 17 19:41:29 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA22749 for ; Tue, 17 Oct 2000 19:41:27 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id QAA91319 for ecm-outgoing; Tue, 17 Oct 2000 16:41:10 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from wind.lcs.mit.edu (wind.lcs.mit.edu [18.31.0.82]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id QAA91314 for ; Tue, 17 Oct 2000 16:41:09 -0700 (PDT) (envelope-from dga@wind.lcs.mit.edu) Received: (from dga@localhost) by wind.lcs.mit.edu (8.10.2/8.10.2) id e9HNf5J06761; Tue, 17 Oct 2000 19:41:05 -0400 Message-Id: <200010172341.e9HNf5J06761@wind.lcs.mit.edu> Subject: Re: Next draft of ECM WG congestion manager document (fwd) To: hari@lcs.mit.edu Date: Tue, 17 Oct 2000 19:41:05 -0400 (EDT) Cc: touch@isi.edu, ecm@aciri.org In-Reply-To: <200010172325.e9HNP6v00896@breeze.lcs.mit.edu> from "Hari Balakrishnan" at Oct 17, 2000 07:25:06 PM From: "David G. Andersen" X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Hari Balakrishnan just mooed: > > > Joe, > > > I've heard these arguments before. In the absence of evidence that CM > > is not a significant penalty (i.e., emperical evidence of the > > performance > > of running code), it is wise to err on the side of caution here. > > There are some running code measurements in the OSDI 2000 paper. In addition > there is now a code release. Hopefully, these should reduce the concerns of > overhead from cm_request(). To expand upon these slightly: Using the congestion manager from userland imposes a measurable overhead on its clients, primarily because it's now necessary to handle ACKs in userland. This results in an additional read() system call and memory copy (acks are typically too small to be handled via a zero-copy scheme like memory mapped buffers). Unfortunately, this is something that can't be avoided if you want to handle your protocol entirely in userland, which is an extremely desirable property for protocol designers - witness the proliferation of UDP based protocols. Once you've paid this penalty, the extra overhead of a request system call is slight. If you want a maximally responsive protocol, you'll be calling cm_update() every time you receive an ACK from your clients anyway, so as to not withhold additional information from the CM. Considered in this framework, the typical process for packet transmission looks something like: request transmit notify update Our implementation optimizes the 'notify' stage by having it called directly by ip_output, so from userland, it looks like: request transmit update The request operation is actually one of the cheapest operations performed by our implementation of the CM. It's not zero-cost; we were able to get a measurable benefit by making it completely implicit in the case of the buffered UDP sockets, but doing so also eliminated several other sources of overhead -- additional sockets to select on and test, the need to perform any checking before enqueuing data into the kernel buffer, and by giving the kernel the ability to immediately transmit the data from its buffer at interrupt time, instead of waiting for the process to be scheduled to call send(). The marginal benefits from JUST eliminating the call to request were very small. If you'd like more details on the numbers, I've got gobs of 'em that we looked at when writing the OSDI paper, and I'm more than happy to share. -Dave Andersen -- work: dga@lcs.mit.edu me: dga@pobox.com MIT Laboratory for Computer Science http://www.angio.net/ From owner-ecm@wyvern.aciri.org Wed Oct 18 09:56:49 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19956 for ; Wed, 18 Oct 2000 09:56:48 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id GAA95434 for ecm-outgoing; Wed, 18 Oct 2000 06:55:01 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id GAA95429 for ; Wed, 18 Oct 2000 06:55:00 -0700 (PDT) (envelope-from touch@ISI.EDU) Received: from isi.edu (ras02.isi.edu [128.9.176.102]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id GAA25578; Wed, 18 Oct 2000 06:54:56 -0700 (PDT) Message-ID: <39EDA8B0.D8914FDC@isi.edu> Date: Wed, 18 Oct 2000 06:42:08 -0700 From: Joe Touch X-Mailer: Mozilla 4.74 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: Hari Balakrishnan CC: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document (fwd) References: <200010172325.e9HNP6v00896@breeze.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Hari Balakrishnan wrote: > > > > > > I am not sure at all that complicating the API > > > > > to accommodate this, in the absence of significant experience that dictates > > > > > that it's necessary, is a good idea. > > > > > > > > That experience is in abundance; per-packet overhead should be avoided, > > > > especially where not otherwise necessary. > > > > > > ... and where you do not give up any other benefits as a result. One important > > > aspect of the CM, is that it is supposed to effectively shape/control the > > > transmission of data. By allowing application to send bursts of data it gives > > > up much of this control. In addition, scheduling such variable sized bursts > > > may also prove to be a non-trivial problem. This seems to have much greater > > > future implications than the per-packet processing overheads. > > > > There are two issues here: > > > > 1. size > > assuming MTU is imprecise anyway > > This was chosen as the balance between making it hard on the application to > know a priori the exact amount of information being sent and CM needing to > control the transmission precisely. This was based on a similar choice by > existing protocols like TCP, and by the preferred mode for ALF (as described in > Clark's 1990 paper). TCP is already being extended to handle byte counting, to be more precise. The difference between MTUs and bytes can range from trivial to a factor of around 10 (e.g., ACKs), on average, given other overheads. Since TCP is already being extended to account for this concern, it makes sense to consider it in the CM now. > However, do note that cm_notify & cm_updates are on the > exact number of bytes sent. Therefore, the CM is able to keep a precise account > of what's going on, within a short time delay (the time between callback and > transmission). It's precise only if, given the grant to send X bytes, X bytes are sent. If TCP's window allows a full MTU, it must request that, because that much data MIGHT appear by the time the callback occurs. However, if new data does not appear, that TCP connection will have waited too long to receive its share of the connection. The CM should be told this, or the utilization of the channel will suffer significantly. > > 2. npkts > > making this larger does increase granularity of the API > > whether an implementation of the API supports this > > should be up to the API > > > > it is overrestrictive to assume that CM must be tickled > > for every packet going out the door, regardless of the size > > > > there are numerous cases where such bursts are desirable, > > and might be impeded by the current interface, e.g., > > bursts in gigabit ethernet, and the small-scale burstiness of > > TCP (sending multiple packets per ACK). > > > > I assume that even though the application requests npkts, the callback can > return anywhere between 1 and npkts as sendable now (or does a callback > indicate that all npkts are sendable?). > > If so, then this would rather complicate both the CM and the application. E.g. > how does the CM decide when it should do the callback---especially when there > are multiple applications all with differently-sized requests? Good question. Some applications have no utility in sending one packet, e.g., transaction systems. Each request needs a min (don't call me back until there are this many bytes available to send) and a max (tell me, up to this number, how many I can send on this callback). If a particular implementation of the CM cannot support it, then so be it. But it should not be excluded from the specification. > Historical note: > > In fact, we had npkts in the API for about 3-4 months inthe early stages of > design, until we were convinced otherwise (my recollection is that we were > going back and forth on it, and that Steve McCanne and Mark Handley > persuasively convinced us to make the simpler decision). In addition, with the > npkts added we also found it necessary to add min_bytes (actually our original > interface was cm_request(id, nbytes, min_bytes)). This was in order to help > the scheduler decide when to do callbacks. A callback had to indicate a value > between min_bytes to nbytes that could be transmitted. > > We were convinced that this provided little gain to applications and would > complicate the internals significantly. We chose the path of lower > implementation complexity. This approach is in fact not new by any means; UNIX If this were a demonstration prototype, by all means. But this is a standarization process, and a reasonable set of expected uses (and performance capabilities) must be supported at the outset. Joe From owner-ecm@wyvern.aciri.org Wed Oct 18 09:56:50 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA19957 for ; Wed, 18 Oct 2000 09:56:48 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id GAA95446 for ecm-outgoing; Wed, 18 Oct 2000 06:55:05 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id GAA95437 for ; Wed, 18 Oct 2000 06:55:04 -0700 (PDT) (envelope-from touch@ISI.EDU) Received: from isi.edu (ras02.isi.edu [128.9.176.102]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id GAA25583; Wed, 18 Oct 2000 06:55:01 -0700 (PDT) Message-ID: <39EDAA22.BAB4C51B@isi.edu> Date: Wed, 18 Oct 2000 06:48:18 -0700 From: Joe Touch X-Mailer: Mozilla 4.74 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: "David G. Andersen" CC: hari@lcs.mit.edu, ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document (fwd) References: <200010172341.e9HNf5J06761@wind.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit "David G. Andersen" wrote: > > Hari Balakrishnan just mooed: > > > > > > Joe, > > > > > I've heard these arguments before. In the absence of evidence that CM > > > is not a significant penalty (i.e., emperical evidence of the > > > performance > > > of running code), it is wise to err on the side of caution here. > > > > There are some running code measurements in the OSDI 2000 paper. In addition > > there is now a code release. Hopefully, these should reduce the concerns of > > overhead from cm_request(). > > To expand upon these slightly: > > Using the congestion manager from userland imposes a measurable overhead > on its clients, primarily because it's now necessary to handle ACKs in > userland. This results in an additional read() system call and memory > copy (acks are typically too small to be handled via a zero-copy scheme > like memory mapped buffers) ... > Unfortunately, this is something that can't be avoided if you want > to handle your protocol entirely in userland, which is an > extremely desirable property for protocol designers - witness > the proliferation of UDP based protocols. The CM is for operational systems, not just as a testbed for protocol designers. While measurement of userland protocol performance may suffice for an OSDI paper, it does not address the issue of performance degradation of kernel protocols, notably TCP, which concern me. > If you want a maximally responsive > protocol, you'll be calling cm_update() every time you receive > an ACK from your clients anyway, so as to not withhold > additional information from the CM. Calling the CM per ACK isn't the issue - it's calling the CM per send. TCP sends multiple packets per ACK. Joe From owner-ecm@wyvern.aciri.org Wed Oct 18 12:08:54 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA22962 for ; Wed, 18 Oct 2000 12:08:53 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id HAA95699 for ecm-outgoing; Wed, 18 Oct 2000 07:40:30 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id HAA95694 for ; Wed, 18 Oct 2000 07:40:30 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: from breeze.lcs.mit.edu (IDENT:hari@localhost.localdomain [127.0.0.1]) by breeze.lcs.mit.edu (8.11.0/8.9.3) with ESMTP id e9IEeSu02019; Wed, 18 Oct 2000 10:40:28 -0400 Message-Id: <200010181440.e9IEeSu02019@breeze.lcs.mit.edu> X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Hari Balakrishnan X-url: http://nms.lcs.mit.edu/~hari/ To: Joe Touch cc: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document (fwd) In-reply-to: Your message of "Wed, 18 Oct 2000 06:48:18 PDT" <39EDAA22.BAB4C51B@isi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 18 Oct 2000 10:40:28 -0400 From: Hari Balakrishnan Sender: owner-ecm@aciri.org Precedence: bulk Joe, On Wed, 18 Oct 2000 06:48:18 PDT, you wrote: > The CM is for operational systems, > not just as a testbed for protocol designers. It *is* for operational systems, including those running new protocols. E.g., SCTP, storage-area network protocols, streaming media,... > While measurement of userland protocol performance may suffice for > an OSDI paper, it does not address the issue of performance > degradation of kernel protocols, notably TCP, which concern me. Our measurements show that in fact an in-kernel protocol like TCP does not suffer any penalties observable ourside of experimental error. Function calls not even inlined) are blazingly fast on even the standard PCs these days. > > If you want a maximally responsive > > protocol, you'll be calling cm_update() every time you receive > > an ACK from your clients anyway, so as to not withhold > > additional information from the CM. > > Calling the CM per ACK isn't the issue - it's calling the > CM per send. TCP sends multiple packets per ACK. Yes, so it calls cm_request() for each MSS and gets a permission when deemed appropriate. No big deal. If there's anything to be concerned about, it's userland apps. As it turns out, our measurements show that it isn't too awful. Hari From owner-ecm@wyvern.aciri.org Wed Oct 18 12:08:55 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA22971 for ; Wed, 18 Oct 2000 12:08:54 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id HAA95665 for ecm-outgoing; Wed, 18 Oct 2000 07:35:43 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id HAA95660 for ; Wed, 18 Oct 2000 07:35:42 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: from breeze.lcs.mit.edu (IDENT:hari@localhost.localdomain [127.0.0.1]) by breeze.lcs.mit.edu (8.11.0/8.9.3) with ESMTP id e9IEZZu01995; Wed, 18 Oct 2000 10:35:35 -0400 Message-Id: <200010181435.e9IEZZu01995@breeze.lcs.mit.edu> X-Mailer: exmh version 2.1.1 10/15/1999 To: Joe Touch cc: ecm@aciri.org Subject: Re: Next draft of ECM WG congestion manager document (fwd) In-Reply-To: Message from Joe Touch of "Wed, 18 Oct 2000 06:42:08 PDT." <39EDA8B0.D8914FDC@isi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 18 Oct 2000 10:35:35 -0400 From: Hari Balakrishnan Sender: owner-ecm@aciri.org Precedence: bulk Joe, > TCP is already being extended to handle byte counting, to be more > precise. > The difference between MTUs and bytes can range from trivial to > a factor of around 10 (e.g., ACKs), on average, given other overheads. > > Since TCP is already being extended to account for this concern, it > makes sense to consider it in the CM now. By saying this, you're giving the impression that the CM precludes byte counting or somehow makes it hard. Neither of this is true. In fact, our implementation does do a form of byte counting, where the cm_update() call from the TCP sender upon receiving an ACK tells the CM exactly how many bytes cleared the pipe (esp. with TCP/SACK). Whether or not byte counting is done in adjusting the window size is orthogonal to the interface provided by cm_request(), as mentioned immediately below. > > However, do note that cm_notify & cm_updates are on the > > exact number of bytes sent. Therefore, the CM is able to keep a precise account > > of what's going on, within a short time delay (the time between callback and > > transmission). > > It's precise only if, given the grant to send X bytes, X bytes are sent. > If TCP's window allows a full MTU, it must request that, because > that much data MIGHT appear by the time the callback occurs. > > However, if new data does not appear, that TCP connection will > have waited too long to receive its share of the connection. The > CM should be told this, or the utilization of the channel will > suffer significantly. It is a simple matter now for the TCP sender to now make a number of cm_request() calls, and receive function callbacks. I don't agree with you that the performance penalty for a good in-kernel implementation in doing this is significant. Furthermore, I claim that it is almost always a good idea to limit bursts to a reasonably small number. You're probably going to say that that's not the case for a 1Gbps/2.5Gpbs (name your number) fast TCP. Perhaps so, but I don't think the additional inlineable function call for the request callback will be _the_ limiting factor here. > > In fact, we had npkts in the API for about 3-4 months inthe early stages of > > design, until we were convinced otherwise (my recollection is that we were > > going back and forth on it, and that Steve McCanne and Mark Handley > > persuasively convinced us to make the simpler decision). In addition, with the > > npkts added we also found it necessary to add min_bytes (actually our original > > interface was cm_request(id, nbytes, min_bytes)). This was in order to help > > the scheduler decide when to do callbacks. A callback had to indicate a value > > between min_bytes to nbytes that could be transmitted. > > > > We were convinced that this provided little gain to applications and would > > complicate the internals significantly. We chose the path of lower > > implementation complexity. This approach is in fact not new by any means; UNIX > > If this were a demonstration prototype, by all means. > But this is a standarization process, and a reasonable > set of expected uses (and performance capabilities) > must be supported at the outset. I don't know where the comment about "demonstration prototype" came from! Are you arguing that UNIX is a "demonstration prototype"?! I was in fact using UNIX, and the "Worse is better" paper on system architecture philosophy to argue that when confronted with design choices that are seemingly hard to resolve, it is often best to take the path of simpler implementation. History seems to suggest that this is almost always the correct choice. I don't believe that any expected use (or performance target) is precluded at all. In fact, that's one of the big advantages of a simple interface and implementation structure. Hari From owner-ecm@wyvern.aciri.org Wed Oct 18 14:17:06 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA10207 for ; Wed, 18 Oct 2000 14:17:06 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id LAA98264 for ecm-outgoing; Wed, 18 Oct 2000 11:15:54 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from wind.lcs.mit.edu (wind.lcs.mit.edu [18.31.0.82]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id LAA98259 for ; Wed, 18 Oct 2000 11:15:53 -0700 (PDT) (envelope-from dga@wind.lcs.mit.edu) Received: (from dga@localhost) by wind.lcs.mit.edu (8.10.2/8.10.2) id e9IIFoG13437; Wed, 18 Oct 2000 14:15:50 -0400 Message-Id: <200010181815.e9IIFoG13437@wind.lcs.mit.edu> Subject: Re: Next draft of ECM WG congestion manager document (fwd) To: touch@isi.edu (Joe Touch) Date: Wed, 18 Oct 2000 14:15:50 -0400 (EDT) Cc: dga@lcs.mit.edu (David G. Andersen), hari@lcs.mit.edu, ecm@aciri.org In-Reply-To: <39EDAA22.BAB4C51B@isi.edu> from "Joe Touch" at Oct 18, 2000 06:48:18 AM From: "David G. Andersen" X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Joe Touch just mooed: > > The CM is for operational systems, > not just as a testbed for protocol designers. > > While measurement of userland protocol performance may suffice for > an OSDI paper, it does not address the issue of performance > degradation of kernel protocols, notably TCP, which concern me. Oh! Sorry about that - I assumed that we were talking about the cases where the performance really goes downhill, and those are all in userland. Our current implementation is really quite unoptimized: We do a lot of pointer chasing to find schedulers and congestion controllers, things that in a production system you could aggressively optimize away or specialize for your common cases. With that in mind, both microbenchmarks and throughput degredation benchmarks indicate that we're currently causing a 1-2% performance hit. Much of cm_request can be expressed as a macro; in particular, a clever implementation could actually treat a call to cm_request as a two-part request: a) I'd like to send a packet b) While you're at it, check how many more I can send if I keep begging resulting in a cm_request call that looks something like: (Sorry if the terminology isn't quite up to that in the draft; my brain is stuck in the code): #define cm_request(macroflow *mflow) lock_mflow(mflow); if (mflow->extra_send == 0) { real_cm_request(mflow); } else { mflow->extra_send--; mflow->unregistered_pkts++; } real_cm_request would do the cleanup job of coping with the extra_send and unregistered packets. This can optimize the heck out of the same case you're worried about, and it has the same set of attendant complications in scheduling multiple packets. If a vendor wants to tackle these complications, they can do so in a way that's API transparent even without taking a number-of-bytes argument. There's lots of room for a really clever person to get in and reduce the API overhead. However, consider what happens in standard TCP during normal transmission: Receipt of an ACK gives permission to send 1 or 2 more packets worth. Not 10. Therefore, if you're trying to transmit as rapidly as possible, cm_request(foo, 10, bar) is likely to fail and tell you you've only got window space for two packets _anyway_. This means that there's one additional function call -- that can perhaps be optimized if you can cope with the agony of basing your scheduler around a multiple request paradigm anyway -- for every two packets you transmit. Also note that the cm_request gives you permission to transmit up to 1 PMTU. If you're sending tiny packets and worried about the overhead, then: a) You shouldn't be sending tiny packets, because they DO have higher overhead in all regards and b) You can send PMTU / packetsize packets per request call already. > Calling the CM per ACK isn't the issue - it's calling the > CM per send. TCP sends multiple packets per ACK. When sending bulk data, TCP sends ~2 packets per ACK. -Dave Andersen -- work: dga@lcs.mit.edu me: dga@pobox.com MIT Laboratory for Computer Science http://www.angio.net/ From owner-ecm@wyvern.aciri.org Tue Oct 24 18:14:42 2000 Received: from wyvern.aciri.org ([192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA19791 for ; Tue, 24 Oct 2000 18:14:41 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id PAA32274 for ecm-outgoing; Tue, 24 Oct 2000 15:11:01 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from daffy.ee.lbl.gov (daffy.ee.lbl.gov [131.243.1.31]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id PAA32269 for ; Tue, 24 Oct 2000 15:11:01 -0700 (PDT) (envelope-from vern@daffy.ee.lbl.gov) Received: (from vern@localhost) by daffy.ee.lbl.gov (8.10.0/8.10.0) id e9OMB0s17454; Tue, 24 Oct 2000 15:11:00 -0700 (PDT) Message-Id: <200010242211.e9OMB0s17454@daffy.ee.lbl.gov> To: ecm@aciri.org Cc: mankin@east.isi.edu Subject: WG last call for draft-ietf-ecm-cm-02.txt as Proposed Standard Date: Tue, 24 Oct 2000 15:11:00 PDT From: Vern Paxson Sender: owner-ecm@aciri.org Precedence: bulk I'd like to begin a 2-week last call for publishing draft-ietf-ecm-cm-02.txt as Proposed. It looks to me that for the most part version -02 of the document addresses the points framed during the last IETF as the necessary work before we could last call it (again). Two of the points haven't been addressed. The first deals with whether to include opaque data with the call to cm_open(). Hari outlined in his note of 15Oct00 why he and Srini think this should not be included. No one has commented further. (Personally, I buy their argument.) The second deals with the interface for sending multiple packets. Joe still disagrees with the arguments put forth that the interface is unneeded, so this issue remains alive. My own leaning is towards the simpler interface, with a view that (1) particular CM implementations can also provide the richer interface, and (2) if experience with these implementations proves that the richer interface is a significant win, then there is still time to change the interface when the document advances to Draft. In any case, let's see if we can achieve rough consensus for this issue during the last call period. The last call will expire November 7. Vern From owner-ecm@wyvern.aciri.org Wed Oct 25 15:07:04 2000 Received: from wyvern.aciri.org ([192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id PAA27524 for ; Wed, 25 Oct 2000 15:07:03 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id MAA39746 for ecm-outgoing; Wed, 25 Oct 2000 12:03:49 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from daffy.ee.lbl.gov (daffy.ee.lbl.gov [131.243.1.31]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id MAA39741 for ; Wed, 25 Oct 2000 12:03:49 -0700 (PDT) (envelope-from vern@daffy.ee.lbl.gov) Received: (from vern@localhost) by daffy.ee.lbl.gov (8.10.0/8.10.0) id e9PJ3mo20892; Wed, 25 Oct 2000 12:03:48 -0700 (PDT) Message-Id: <200010251903.e9PJ3mo20892@daffy.ee.lbl.gov> To: ecm@aciri.org Cc: mankin@east.isi.edu Subject: ECM planning for the San Diego IETF and the future Date: Wed, 25 Oct 2000 12:03:48 PDT From: Vern Paxson Sender: owner-ecm@aciri.org Precedence: bulk I'm currently undecided as to whether ECM should meet at the upcoming San Diego IETF. There are two reasons that would justify meeting. The first is if the last call for draft-ietf-ecm-cm-02.txt results in new issues that we can only resolve during a face-to-face meeting. I think this is unlikely, given that we already had a meeting at the last IETF dedicated to ironing out the final issues. The second is if someone gets rolling on the ECM scheduler document. I asked for volunteers at the last meeting, but none were forthcoming, and no one has contacted me subsequently. This is a significant problem, and my thinking is that if no one picks it up during the next month or so, then I will discuss with the ADs terminating the working group, as we won't be able to produce our remaining work item in a timely fashion. I will also be inclined at this point to argue against rechartering the WG, due to the apparent lack of sufficient community interest for moving it forward. Vern From owner-ecm@wyvern.aciri.org Thu Oct 26 02:54:17 2000 Received: from wyvern.aciri.org ([192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA08071 for ; Thu, 26 Oct 2000 02:54:17 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id XAA43093 for ecm-outgoing; Wed, 25 Oct 2000 23:53:33 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from flashmail.com ([203.8.68.101]) by wyvern.aciri.org (8.9.3/8.9.3) with SMTP id XAA43088 for ecm@aciri.org; Wed, 25 Oct 2000 23:53:29 -0700 (PDT) (envelope-from fast_ride2001@flashmail.com) Message-Id: X-Sender: fast_ride2001@flashmail.com X-Mailer: Mozilla 4.71 [en] (Win98; I) Date: Wed, 25 Oct 2000 23:52:54 -0700 To: carlist From: fast_ride2001 Subject: FREE! Get A Great Price On A New Car! (64a37529) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-ecm@aciri.org Precedence: bulk X-MIME-Autoconverted: from 8bit to quoted-printable by wyvern.aciri.org id XAB43093 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by ietf.org id CAA08071 Get A Great Price On A New Car! Absolutely Free! Want to save time and money? Want to have quick access to car quotes? Want to have all makes and models available to you? If you answered yes to any of these, then take advantage of this free, no-hassle service. Simply click on the link below to get low prices on all makes and models of new and used cars, without having to negotiate with a dealer. It’s painless and stress-free! CLICK-HERE--> http://3627528622/ <--CLICK-HERE ************************************************** If you wish to unsubscribe from this list, simply go to http://3627528622/remove.html and follow the instructions. You will be removed immediately. PLEASE NOTE: replying to this message will not remove you, you MUST follow the link above. Thank you. ************************************************** From owner-ecm@wyvern.aciri.org Thu Oct 26 13:27:28 2000 Received: from wyvern.aciri.org ([192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA26666 for ; Thu, 26 Oct 2000 13:27:27 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id KAA47643 for ecm-outgoing; Thu, 26 Oct 2000 10:26:10 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id KAA47638 for ; Thu, 26 Oct 2000 10:26:10 -0700 (PDT) (envelope-from touch@ISI.EDU) Received: from isi.edu (sci.isi.edu [128.9.160.93]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id KAA22270; Thu, 26 Oct 2000 10:26:05 -0700 (PDT) Message-ID: <39F8690F.DD5C2F3E@isi.edu> Date: Thu, 26 Oct 2000 10:25:35 -0700 From: Joe Touch X-Mailer: Mozilla 4.74 [en] (Win98; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: Vern Paxson CC: ecm@aciri.org, mankin@isi.edu Subject: Re: ECM planning for the San Diego IETF and the future References: <200010251903.e9PJ3mo20892@daffy.ee.lbl.gov> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Vern Paxson wrote: > > I'm currently undecided as to whether ECM should meet at the upcoming > San Diego IETF. There are two reasons that would justify meeting. > The first is if the last call for draft-ietf-ecm-cm-02.txt results in new > issues that we can only resolve during a face-to-face meeting. I think > this is unlikely, given that we already had a meeting at the last IETF > dedicated to ironing out the final issues. > > The second is if someone gets rolling on the ECM scheduler document. > I asked for volunteers at the last meeting, but none were forthcoming, and > no one has contacted me subsequently. This is a significant problem, and > my thinking is that if no one picks it up during the next month or so, then > I will discuss with the ADs terminating the working group, as we won't be > able to produce our remaining work item in a timely fashion. I will also > be inclined at this point to argue against rechartering the WG, due to > the apparent lack of sufficient community interest for moving it forward. > > Vern In that case I would like to suggest that ECM-CM be 'Experimental', rather than 'Proposed Standard'. A framework without the scheduler may not be useful as a standard. Joe From owner-ecm@wyvern.aciri.org Thu Oct 26 13:32:58 2000 Received: from wyvern.aciri.org ([192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA27348 for ; Thu, 26 Oct 2000 13:32:57 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id KAA47692 for ecm-outgoing; Thu, 26 Oct 2000 10:32:54 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from breeze.lcs.mit.edu (breeze.lcs.mit.edu [18.31.0.83]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id KAA47687 for ; Thu, 26 Oct 2000 10:32:53 -0700 (PDT) (envelope-from hari@breeze.lcs.mit.edu) Received: from breeze.lcs.mit.edu (IDENT:hari@localhost.localdomain [127.0.0.1]) by breeze.lcs.mit.edu (8.11.0/8.9.3) with ESMTP id e9QHWou09183; Thu, 26 Oct 2000 13:32:50 -0400 Message-Id: <200010261732.e9QHWou09183@breeze.lcs.mit.edu> X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Hari Balakrishnan X-url: http://nms.lcs.mit.edu/~hari/ To: Joe Touch cc: ecm@aciri.org Subject: Re: ECM planning for the San Diego IETF and the future In-reply-to: Your message of "Thu, 26 Oct 2000 10:25:35 PDT" <39F8690F.DD5C2F3E@isi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 26 Oct 2000 13:32:50 -0400 From: Hari Balakrishnan Sender: owner-ecm@aciri.org Precedence: bulk On Thu, 26 Oct 2000 10:25:35 PDT, you wrote: > In that case I would like to suggest that ECM-CM be 'Experimental', > rather than 'Proposed Standard'. A framework without the scheduler > may not be useful as a standard. Independent of whether it be "experimental" or "proposed standard," my response is about the comment you made about the CM's usefulness. I disagree that the framework is not useful without a detailed scheduler. A round-robin default does not need any documentation. Vendors may choose to implement their own schedulers, however sophisticated, as a means to differentiate themselves. The core API and congestion controller themselves can accommodate a wide variety of schedulers, which need not be standardized---or even publicly documented---in my opinion. Hari From owner-ecm@wyvern.aciri.org Thu Oct 26 13:58:35 2000 Received: from wyvern.aciri.org ([192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA00391 for ; Thu, 26 Oct 2000 13:58:35 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id KAA47855 for ecm-outgoing; Thu, 26 Oct 2000 10:58:24 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id KAA47850 for ; Thu, 26 Oct 2000 10:58:24 -0700 (PDT) (envelope-from touch@ISI.EDU) Received: from isi.edu (sci.isi.edu [128.9.160.93]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id KAA25436; Thu, 26 Oct 2000 10:58:22 -0700 (PDT) Message-ID: <39F870A0.917FC159@isi.edu> Date: Thu, 26 Oct 2000 10:57:52 -0700 From: Joe Touch X-Mailer: Mozilla 4.74 [en] (Win98; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: Hari Balakrishnan CC: ecm@aciri.org Subject: Re: ECM planning for the San Diego IETF and the future References: <200010261732.e9QHWou09183@breeze.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Hari Balakrishnan wrote: > > On Thu, 26 Oct 2000 10:25:35 PDT, you wrote: > > > In that case I would like to suggest that ECM-CM be 'Experimental', > > rather than 'Proposed Standard'. A framework without the scheduler > > may not be useful as a standard. > > Independent of whether it be "experimental" or "proposed standard," my response > is about the comment you made about the CM's usefulness. > > I disagree that the framework is not useful without a detailed scheduler. A > round-robin default does not need any documentation. Vendors may choose to > implement their own schedulers, however sophisticated, as a means to > differentiate themselves. > > The core API and congestion controller themselves can accommodate a wide > variety of schedulers, which need not be standardized---or even publicly > documented---in my opinion. There still needs to be a specification of the scheduler; the particular algorithm may not be important, but the architecture is. Also, 'experimental' is driven perhaps better by Vern's observation about community interest. Joe From owner-ecm@wyvern.aciri.org Thu Oct 26 14:05:05 2000 Received: from wyvern.aciri.org ([192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA01219 for ; Thu, 26 Oct 2000 14:05:04 -0400 (EDT) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id LAA47905 for ecm-outgoing; Thu, 26 Oct 2000 11:04:57 -0700 (PDT) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from daffy.ee.lbl.gov (daffy.ee.lbl.gov [131.243.1.31]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id LAA47900 for ; Thu, 26 Oct 2000 11:04:56 -0700 (PDT) (envelope-from vern@daffy.ee.lbl.gov) Received: (from vern@localhost) by daffy.ee.lbl.gov (8.10.0/8.10.0) id e9QI4sA24004; Thu, 26 Oct 2000 11:04:54 -0700 (PDT) Message-Id: <200010261804.e9QI4sA24004@daffy.ee.lbl.gov> To: Joe Touch Cc: Hari Balakrishnan , ecm@aciri.org Subject: Re: ECM planning for the San Diego IETF and the future In-reply-to: Your message of Thu, 26 Oct 2000 10:57:52 PDT. Date: Thu, 26 Oct 2000 11:04:53 PDT From: Vern Paxson Sender: owner-ecm@aciri.org Precedence: bulk > Also, 'experimental' is driven perhaps better > by Vern's observation about community interest. My comment was conditional on *if* we can't get the scheduler work going in the next month or two. I haven't given up on community interest, but am certainly underwhelmed at this point. But putting that aside, the last call remains for Proposed. Assuming the WG approves the document during last call, then I'll note to the ADs that there was some dissension about the document's status. My current reading of the WG's view based on past discussions is that Proposed remains the correct category. Vern From owner-ecm@wyvern.aciri.org Mon Oct 30 18:21:16 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA13753 for ; Mon, 30 Oct 2000 18:21:15 -0500 (EST) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id PAA83787 for ecm-outgoing; Mon, 30 Oct 2000 15:18:04 -0800 (PST) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id PAA83782 for ; Mon, 30 Oct 2000 15:18:04 -0800 (PST) (envelope-from touch@ISI.EDU) Received: from isi.edu (sci.isi.edu [128.9.160.93]) by boreas.isi.edu (8.9.3/8.9.3) with ESMTP id PAA16074; Mon, 30 Oct 2000 15:18:01 -0800 (PST) Message-ID: <39FE00F7.2EBC3D15@isi.edu> Date: Mon, 30 Oct 2000 15:15:03 -0800 From: Joe Touch X-Mailer: Mozilla 4.74 [en] (Win98; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: "David G. Andersen" CC: hari@lcs.mit.edu, ecm@aciri.org, touch@isi.edu Subject: Re: Next draft of ECM WG congestion manager document (fwd) References: <200010181815.e9IIFoG13437@wind.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit "David G. Andersen" wrote: > > Joe Touch just mooed: > > > > The CM is for operational systems, > > not just as a testbed for protocol designers. > > > > While measurement of userland protocol performance may suffice for > > an OSDI paper, it does not address the issue of performance > > degradation of kernel protocols, notably TCP, which concern me. > > Oh! Sorry about that - I assumed that we were talking about the cases > where the performance really goes downhill, and those are all in userland. > Our current implementation is really quite unoptimized: We do a lot of > pointer chasing to find schedulers and congestion controllers, things that > in a production system you could aggressively optimize away or specialize > for your common cases. With that in mind, both microbenchmarks and > throughput degredation benchmarks indicate that we're currently causing a > 1-2% performance hit. > > Much of cm_request can be expressed as a macro; in particular, a clever > implementation could actually treat a call to cm_request as a two-part > request: > > a) I'd like to send a packet > b) While you're at it, check how many more I can send if I keep > begging > > resulting in a cm_request call that looks something like: > > (Sorry if the terminology isn't quite up to that in the draft; > my brain is stuck in the code): > > #define cm_request(macroflow *mflow) > lock_mflow(mflow); > if (mflow->extra_send == 0) { > real_cm_request(mflow); > } else { > mflow->extra_send--; > mflow->unregistered_pkts++; > } > > real_cm_request would do the cleanup job of coping with > the extra_send and unregistered packets. The lock is probably the most worrisome of the components, computationally as well as in latency of the function call. > There's lots of room for a really clever person to get in and > reduce the API overhead. However, consider what happens > in standard TCP during normal transmission: Receipt of an > ACK gives permission to send 1 or 2 more packets worth. > Not 10. Therefore, if you're trying to transmit as rapidly as > possible, cm_request(foo, 10, bar) is likely to fail and tell you > you've only got window space for two packets _anyway_. But if there are 4 ACKs waiting to be processed, the CM API must be called 8 times, rather than 4 (or even better, 1). > Also note that the cm_request gives you permission to transmit > up to 1 PMTU. If you're sending tiny packets and worried > about the overhead, then: > > a) You shouldn't be sending tiny packets, because they DO > have higher overhead in all regards I might have to, e.g., to use gigabit ethernet in non-jumbogram mode. > and > b) You can send PMTU / packetsize packets per request > call already. This conflicts with a). If tiny != sum of the data, then the CM should differentiate between sending 10 packets of 50 bytes and one of 500. > > Calling the CM per ACK isn't the issue - it's calling the > > CM per send. TCP sends multiple packets per ACK. > > When sending bulk data, TCP sends ~2 packets per ACK. That too depends on the ACK. If ACKs are lost (or compressed, as with some proposals), the receipt of a single ACK can generate up to a windows' worth of new outgoing packets. Joe From owner-ecm@wyvern.aciri.org Mon Oct 30 21:15:48 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA28237 for ; Mon, 30 Oct 2000 21:15:47 -0500 (EST) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id SAA85067 for ecm-outgoing; Mon, 30 Oct 2000 18:15:12 -0800 (PST) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from wind.lcs.mit.edu (wind.lcs.mit.edu [18.31.0.82]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id SAA85062 for ; Mon, 30 Oct 2000 18:15:12 -0800 (PST) (envelope-from dga@wind.lcs.mit.edu) Received: (from dga@localhost) by wind.lcs.mit.edu (8.10.2/8.10.2) id e9V2F9d08577; Mon, 30 Oct 2000 21:15:09 -0500 Message-Id: <200010310215.e9V2F9d08577@wind.lcs.mit.edu> Subject: Re: Next draft of ECM WG congestion manager document (fwd) To: touch@isi.edu (Joe Touch) Date: Mon, 30 Oct 2000 21:15:09 -0500 (EST) Cc: dga@lcs.mit.edu (David G. Andersen), hari@lcs.mit.edu, ecm@aciri.org, touch@isi.edu In-Reply-To: <39FE00F7.2EBC3D15@isi.edu> from "Joe Touch" at Oct 30, 2000 03:15:03 PM From: "David G. Andersen" X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit Joe Touch just mooed: > > > > #define cm_request(macroflow *mflow) > > lock_mflow(mflow); ... > > The lock is probably the most worrisome of the components, > computationally as well as in latency of the function call. In a uniprocessor system, these operations are NOPs. In a multiprocessor system, these are macros that, in the uncontested case, throw in a memory write barrier and an atomic test/set of the lock. I agree completely with the point you're about to make - that in the contested case, this will slow down access to the control structures. Inserting a small dose of reality, however, the time spent with these structures locked is puny compared to the time taken to actually transmit a packet -- since you're calling these once per packet request, the extra time wasted in the case of a contested multiprocessor system is minimal. ... and in the common case of a uniprocessor system, or a multiprocessor system without a multithreaded network stack, or an uncontested multithreaded / multiprocessor machine, the lock is a small cost, especially considering that in the same circumstances, the act of packet transmission requires grabbing at least as many of the same kinds of locks. > > There's lots of room for a really clever person to get in and > > reduce the API overhead. However, consider what happens > > in standard TCP during normal transmission: Receipt of an > > ACK gives permission to send 1 or 2 more packets worth. > > Not 10. Therefore, if you're trying to transmit as rapidly as > > possible, cm_request(foo, 10, bar) is likely to fail and tell you > > you've only got window space for two packets _anyway_. > > But if there are 4 ACKs waiting to be processed, > the CM API must be called 8 times, rather than 4 > (or even better, 1). I completely agree. That's why I suggested that the actual call to the CM API can be optimized up one side and down the other, _provided_ the task of writing the "multiple request" API can be done easily. If it can, you can make the single request case very fast, as I noted above. If you can't, well, it doesn't matter anyway. > > Also note that the cm_request gives you permission to transmit > > up to 1 PMTU. If you're sending tiny packets and worried > > about the overhead, then: > > > > a) You shouldn't be sending tiny packets, because they DO > > have higher overhead in all regards > > I might have to, e.g., to use gigabit ethernet in non-jumbogram mode. Um. I think I'm missing something in intuiting the circumstances under which you'd want to artificially limit your MTU for the same (sender, receiver) pair (e.g. the same macroflow) for one application, and NOT do so with another. Either your jumbogram mode is better (mtu==big, and use it) or it's not (mtu==1500). Which is a longwinded way of saying: If you don't want to use your ethernet in jumbogram mode, then your MTU is 1500. > > and > > b) You can send PMTU / packetsize packets per request > > call already. > > This conflicts with a). If tiny != sum of the data, then > the CM should differentiate between sending 10 packets of 50 bytes > and one of 500. I disagree. My point in (a) was about CPU and interrupt utilization, but this actually brings up an interesting issue of how much attention the CM should pay to link layer costs: With a network like ethernet, packets really are expensive per-packet because of the back-off, but with a full-duplex point to point link, you only pay the cost of the extra headers. I think that this goes way off the deep end of what the CM should do, however -- if you take this to the extreme, you have to then consider every possible network technology over which you pass. Best to simply count bytes passed to IP, and let the other layers do their job as they see fit. > That too depends on the ACK. If ACKs are lost (or > compressed, as with some proposals), the receipt of > a single ACK can generate up to a windows' worth of > new outgoing packets. ... which can still be well optimized as above. -Dave From owner-ecm@wyvern.aciri.org Tue Oct 31 23:47:55 2000 Received: from wyvern.aciri.org (wyvern.aciri.org [192.150.187.14]) by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA29707 for ; Tue, 31 Oct 2000 23:47:54 -0500 (EST) Received: (from majordomo@localhost) by wyvern.aciri.org (8.9.3/8.9.3) id UAA95303 for ecm-outgoing; Tue, 31 Oct 2000 20:44:11 -0800 (PST) (envelope-from owner-ecm@smtp.aciri.org) X-Authentication-Warning: wyvern.aciri.org: majordomo set sender to owner-ecm@smtp.aciri.org using -f Received: from wind.lcs.mit.edu (wind.lcs.mit.edu [18.31.0.82]) by wyvern.aciri.org (8.9.3/8.9.3) with ESMTP id UAA95298 for ; Tue, 31 Oct 2000 20:44:10 -0800 (PST) (envelope-from dga@wind.lcs.mit.edu) Received: (from dga@localhost) by wind.lcs.mit.edu (8.10.2/8.10.2) id eA14i9j16609 for ecm@aciri.org; Tue, 31 Oct 2000 23:44:09 -0500 Message-Id: <200011010444.eA14i9j16609@wind.lcs.mit.edu> Subject: Multiple transmission requests in cm_request To: ecm@aciri.org Date: Tue, 31 Oct 2000 23:44:09 -0500 (EST) From: "David G. Andersen" X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-ecm@aciri.org Precedence: bulk Content-Transfer-Encoding: 7bit A few (last) thoughts about the cm_request issue: There seem to be two competing issues here: a) Can the congestion controller and scheduler be implemented as efficiently / as well if the API is changed to allow multiple packet transmissions to be requested? Would mandating this hobble vendors trying to implement a working CM? b) Would constraining cm_request to a single packet at a time impose unreasonable overhead on performance-sensitive clients? It's fairly obvious that we can continue this discussion well into this century through lakes of thought experiments about the effects of request caching and locking, but it seems like this is a terribly obvious place where some experimentation would settle the issue. I think that a good answer to (a) is necessary before any consideration of (b) can be realistically considered. My gut feeling is that implementing (a) will be a non-trivial project. That said, it seems reasonable and expedient to follow Vern's suggestion to move on with the interface as-is. And for now, there are plenty of opportunities to demonstrate the benefits of a batched cm_reqest -- our source code is out there for hacking upon, and obviously, the more independent implementations of the CM, the better! -Dave