USOII743123B2 «2 United States Patent (10) Patent No: US 11,743,123 B2 Casado et al. (45) Date of Patent: *Aug. 29, 2023 (St) MANAGED SWITCIL ARCHITECTURES: 66) References Cited SOFTWARE MANAGED SWITCHES, HARDWARE MANAGED SWITCHES, AND US. PATENT DOCUMENTS HETEROGENEOUS MANAGED SWITCHES 5224100 A 611993 Loe tal (71) Applicant: Nieira, Ine, Palo Alto, CA (US) ree eee a (Continved) (72) Inventors: Martin Casado, Portola Valley, CA (US); Teemu Koponen, San Francisco, FOREIGN PATENT DOCUMENTS CA WS); Pankal Thakkar, Cupertino, ry Lesn6es AL $2006 cAUS) 2 ao02iaions A $2002 (73) Assignee: NICIRA, INC, Palo Alto, CA (US) (Continued) (4) Notice: Subject to any disclaimer, the term ofthis (OTHER PUBLICATIONS patent is extended or adjusted under 35 TSC. 1545) by 0 days. AbEares, Mohammad, ea. “A Scalable, Commodity Data Center Nawork Architet,” SIGCOMME ‘08, Aug. 17-22, 2008. 12 pages, Satie, Washington, USA, (Continved) ‘This patent is subject to a terminal dis- claimer, 21) Appl. No. 16902,281 ee Primary Examiner — Apu tisu (22) Filed: Jun, 15, 2020 (74) Attorney, Agent, o” Firm — ADELL LLP 6s) Prior Publication Data on ABSTRACT US 202010396130.A1 Dee. 17, 2020 Some embodiments of the invention provide a a method of meester processing packets associated with a logical switching ele- ‘ent implemented by multiple physical switching elements (63) Continuation of application No. 16/084,035, filed on executing on multiple host computers on which multiple Jul. 12, 2018, now Pat. No. 10,686,663, which is a machines execute. Ata first physial switching clement ofa (Continued) first host computer, the method receives a packet from a frst ‘machine associsted with the logical switching element, Por GI) Ince, the packet, the method identities a logical ingress port ofthe How. 1224 (2006.01) logical switch that is associated with the packet. For the Host, 12726 (2005.01) packet, the method also uses the logical ingress por 10 (Continued ‘dentify a logical egress port of the logical switch that is associated withthe packet, For the packet, the method also (52) Us. CPC... OL. 41/893 (2013.01); GOOF 1817312 (2013.01), HOdd. 12/4633 (2013.01): ‘ses the logical egress port to identify a physical egress port ofthe fist host eompater to use to send the packet along 10 4 second machine associated with the logical egress por. (Contino rom te seed phystal eres por the meted Bt (58) Field of Classification Search ede pce ites capa Reader Hats cre SG 1/07, Gost 15/7312; nou, pum racst» : iiaess, Hoa. 10816: Hos. 410856 (Contino 20 Claims, 57 Drawing Sets . Cohen ao a Lt Cos tan [ate] ee |US 11,743,123 B2 Page 2 Related U.S. Application Data (58) Fleld of Classification Search CPC "HOSE 31/0893; HOT 41/0896; HOST 45/00; ‘continuation of application No. 151076,634, fled on uitiastoee ar 47/1 TioaL? ao Mar. 21, 2016, now Pat, No, 10,038,597, which is 0 oat. 49/1586: Hod. 49/3003; E08. ‘continuation of application No. 13/218,497, fled on 440/70, HL 61/2007; HOAL. 61/602 Aug. 26, 2011, now Pat. No. 9,306,875, which is a ‘See application file foe compete search history. ‘continiation-in-part of application No. 13/177,536, filed on Jul. 6, 2011, now Pat. No. 8,959,215, anda (56) References Cited ‘continuation of application No, 13/177,535, fled on Jul. 6, 2011, now Pa No, 8,750,164 and a continu Ueto eel sald ‘ion:in-prt of aplication No, 1177.53 ied 5268002 A W188 Selomy a Jul. 6, 2011, now Pat. No. 8.830.823 SSotoa1 A ‘41999 Dow ea SSS0KIG A R190 Hardwick ct Simi § 31998 Chan (60). Provisional application No. 61/505,103, filed on Ju sre s S201 protons sprion No-61/505100, ied «SONG A008 fon Jl.” 6, 2011, provisional application No, Soca a ‘T1999 61/805,102, filed ‘on Jul. 6, 2011, provisional soon A 99 ‘pplisation No, 61/501.788, fled on Jun. 28, 201 Gassa8 A 2000 Vial ta provisional pplication No, 61/501.745, fil on un S104 $2000 lender a 21, 2011, provisional application No. 61/482,615, Gites A dom) Abeer fled on May 4, 2011, provisional application No, Ghanzss A 102000 Mme a 61/482,616, fled on May” 4, 2011 provisional SISLE9" BL "12001. Noreen aplication No. 61/882,205, filed on Nay 3, 2011, 8219009 BI 42001 MeCToghi provisional pplication No 61/4665, ed on Mat ales 22, 2011, provisional application No. 61/429,783, eae fled on Jan. 4, 2011, provisional application No ‘£2002 Nid oa 661/429,754, filed on. Jan, 4, 2011, provisional application No. 61/361.913, filed on Tul. 6, 2010, ‘42002 Not ta 5/2002 estes et eevee aplcaion No SUMGLI2 Ml aS Ble TAM ime... ML 06 6, 2010, 6512945 BL 12003 abe tat : Utes Reve st) mc SamA0 BL, 3208 Tague Io. 1246 (200601) SSPE Me C38 ma 90 ‘oat 1s (202201) 7 370895 Mose 12713, (201301), Sours20 Bie R20 Phong Host 457142 ‘osl 2911 (201301) erhaoea ‘oat 2931 (201301) 665438 BL 112003 Tolan eta udt 12933201301) Secon Bl 12200) Rosa ‘Ho 2935201301) 0 Toot Can oar 938 Goi8o1, SoroM BL 42001 yuan Conk 107 (2006.01) Som BL 42001 teeta Gar 1816 (Go0601) Sea BL 93001 Kee ta Guor 19173 (200001) S70 BL $200) Slo wal fina timss—aom301) Baa Se a ‘oat 491545 (202201) cen ee ‘at 45586 (202201) 208 Patan ta ‘oat 1900 (202201) % $08 Sad ca How. 4783 (202201) eomoe m2 200 Meo ca fost soe (aon3y sotidiy at 3s di a Mout 410896 (202201) Sotoaoe bi 93008 ‘oa ors? (302201) osnoaa Be 102008 ‘oa. 1802 (202201) Sees RE oul tinsi6 (202201 : 2 feats = Gren pana he Ee ‘oa. 10622 (2022.01) ae S000 (32) US.CL 2006 £2000, Bao ea $2006 ‘Theis e 1012006. Krskiran ta 102005. Yang et 123008 Cheriton CPC. MOsL 41/0816 (2013.01); Host 41/0883 (2013.01); HOME. 41/0896 (2013.01); HO. 45700 (2013.01); HOsL 452 (2013.01), Hol 48/886 (2013.01), HOM. 477783 201301, most wae coisony rom; Fisaars be aa Ma 49/546 20101), Hod 492068 (20180 3 22a Tahal ost 4970 201S01y Hos sisooy IRS BL 4287 oat (202205), GaP 11/07 (013.01), HoaL Bd eer 210/622 (202205) 7oo0'430 8242007 Rawlins eaUS 11,743,123 B2 Page 3 (66) References Cited wo Nowes BL ti USS. PATENT DOCUMENTS, 7898412 Bi Kerr etal oo027 Cosignoi ea 7268200 8282007 Fontin ta. Toons Bo hanno a Wane elena 7912986 Br Macha et a Foes B2 102007 Amit 791678 Be Trt 10.007 Saleh ta rate. th007 Wiehe Miler ea Novaes Gh ea. Zeige Mer ea Bas eta. Tue Merle 7o61882 Be ‘wash 3008 Mince a Fomor? Nakao tl 738745 Bo 4/3008 Pare eta Sslomen 7309971 B2 43008 Jorgensen Sons eta F300852 B2 $2008 Newbury etal Miler 730.787 BI 62008 Hames ca, Bayar ot a T3917 BO 62008 Orava ct al ‘AwoodSaith et Fatsa6s B2 2008 Tes imon ea 728.220 BL 9/2008 Caron ot a Sanat SojeMoloy ot a. 7430782 B2 102008 Neo ea Soxstes rho Yhut03 B2_1/3008. Chen eal sors Forlani ta Tas.508 B21 208 Chen Sa S310 Mamon ea: Zed as? B2 122008 Pike Sa31633 Ben et a 703599 B2 122008 Lapa a Si 99 ‘Ache ea Hare BI ‘T2009 Daleo 8136.23 Baoctse Hoa, 4.384 788370 BL 1/2009 Dayal etal. sm0397 Jaa B2 1009 Weiss ta S048.580 52 102011 LaVigne ta 796597 B ising Silease BL 102011 Miler ea 7512744 Be Bangs ta Sos4s32 BL 12011 Shukla ca 75235595 B2 43009 BensDvor ta 808879 B2 11011 Richaroon et 7520:199 BL $009 Winds ees. $080.560 82102011 Cahiama es F5s8002 BE 2009 Amc a 060.875 BL 112011 Lambeth 7587492 B2 9/009 Dyck cea Site4336 82 11/2011 Sahat ea 759066) B29 Yipaat SO90871 B2 12012 Hhgtu ot a Foes? B2 10 2009 Zeng 8099 904 21/2012 Balasubramaniam ea. 7106239 Bi 10/2009 Foschao ct. 8099.96) B? 2012 Melman etal 706260 B2 10009 Oguchi eta 809961 8 12012 Loa Zst6309 B2 11/2009 Jabbar a Slo 781 B2 12012 Smith 719960 B2 11/2009 Lee 812104 B2 23012. Sani ca 7683961 B2 123009. Ratt a S130.688 83 —32012 Kwan ea fas BE 112010. Khanna eal S107 B2 42012 Watanabe et a Fede 74 B2 12010 Woah SIS1882 BL 32012 Miler eal, 78881 B2 1/2010 Takashige ta Slao.a8 B? 32012 Cheriton 7160025) B2— 23010 Monet ta ett 00 ta HOSOT BE 4010 Soja Molloy a 8140737 B2 42012 Mea et 798.158 Bi 42010 Corie oat 6v10 8158000 B? 42012 Vasseur 70409 S188.098 B2 4012 Abrflamdeh ea. 702610 B2 4/2010 Basso eta £2012 enbaom et a T1072 B2 $2010 Ver 42012 Opis et ZIIO874 B2—5$/010 Balsa et 42013 Pater ea 7718309 B2 S010. Scholl et a Suess B2 42012 Mohan 7364999 B2 72010 Dovet 1e6.201 B2 42012 Richanson ca. 757%259 Bi 82010 Mabalingiah 42013 Faninace TENS B2 $2010. Hashing oa $3003 Farkas ea 739203 B2_ 92010 Faldeal 62012 Schulte eta 7592046 82° 9/2010. Ruchareryk HOH 49/284 72012 Jabba ta aoa 73013 Atta e a 1992009 B2 912010. Yasaka a. 72012 Brandi ta 382987 BI 9.010 Yom et a 826071 BL 72012 Miler eta 7356.03 B2 9.2010, Hamada et 823008 BL 72012. Banding ot al 7796594 B2 9010 Melman et 82.489 Bo 72019 Wel eca TROLLS? B2 92010. Vasseur ea S2:3.732 B2 82012 Chen ta. 7sM2251 B2 94010 Kitamere 92012 Ranjan 7808407 Bi 9010. Verbeke ea. 972012 Panky TRIRSID BE 102010 Nadeau ea 102012 Sian et F08929 B2 102010 Wong ea 402012. Venkatraman a 78089 B2 102010 Joly 12012 Reber TS18340 B2 102010. Novaes 2012 Miler et a 7817637 B2 102010 Kitna 2012. Moisand ea. ZRINAS? B2 102010 Mathews et a 122012 Gnanaskara ea 7822032 Bi 102010. Palker ea. 122012 Betiak ea Faxes? BL 112010 Mint ea $348,680 B2 12019 Foxworthy ot a TRMLSK? B2* 112010 Johnson SSSLAI8 B2 12013 Zhao etal. S386 B? 22013 Euan etal 780547 B2 11/2010. Nadeaw eta S391185 B2 32019 Winds a. 7888980 62122010 Zwiebe eal 8422389 B2 42013 Nakajima T8860 B2 122010 Whee Sairo19 B2 $2013 Vaswur