This project aims to be a load benchmarking suite, no more, no less
Measuring response times (routing times) for each framework (middleware).
Results are not production-ready yet
- Helping decide between languages, depending on use case
- Learning languages, best practices, devops culture ...
- Having fun ❤️
- Crystal as
built-intools are made in this language - Docker as frameworks are
isolatedinto containers - wrk as benchmarking tool,
>= 4.1.0 - postgresql to store data,
>= 10
ℹ️ℹ️ℹ️ℹ️ℹ️
OSX you need docker-machine to use docker containerization
brew install docker-machine
docker-machine create default
eval $(docker-machine env default)
ℹ️ℹ️ℹ️ℹ️ℹ️
- Install all dependencies
shards install- Build internal tools
shards build- Create and initialize the database
createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sqlDocker can be used to set up the database:
docker run -it --rm -d \
-p 5432:5432 \
-e POSTGRES_DB=benchmark \
-e POSTGRES_HOST_AUTH_METHOD=trust \
-v /tmp/pg-data:/var/lib/postgresql/data \
--name pg postgres:12-alpineWait several seconds for the container to start, then inject the dump:
docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"After creating the database, export its URL:
export DATABASE_URL="postgresql://postgres@localhost/benchmark"- Make configuration
bin/make config- Build containers
jobs are either languages (example : crystal) or frameworks (example : router.cr)
bin/neph [job1] [job2] [job3] ...- Export all results readme
bin/db to_readmeℹ️ Updated on 2020-08-06 ℹ️
Benchmarking with wrk
- Threads : 8
- Timeout : 8
- Duration : 15s (seconds)
ℹ️ Sorted by max req/s on concurrency 64 ℹ️
| Language | Framework | Speed (64) | Speed (256) | Speed (512) | |
|---|---|---|---|---|---|
| 1 | php (7.4) | mark (1.1) | 167 687 | 171 413 | 172 108 |
| 2 | nim (1.2) | httpbeast (0.2) | 163 348 | 194 799 | 196 969 |
| 3 | go (1.14) | fiber (1.13) | 161 393 | 172 412 | 175 606 |
| 4 | go (1.14) | atreugo (11.5) | 157 177 | 171 116 | 175 690 |
| 5 | go (1.14) | fasthttp (1.15) | 155 891 | 168 589 | 172 120 |
| 6 | go (1.14) | fasthttprouter (0.1) | 154 165 | 169 320 | 174 094 |
| 7 | javascript (14.7) | sifrr (0.0) | 154 028 | 196 301 | 200 982 |
| 8 | nim (1.2) | whip (0.2) | 152 326 | 179 652 | 181 107 |
| 9 | go (1.14) | gorouter-fasthttp (4.4) | 149 822 | 161 272 | 164 973 |
| 10 | go (1.14) | router (1.2) | 146 966 | 159 894 | 164 312 |
| 11 | java (11) | rapidoid (5.5) | 142 883 | 170 582 | 170 382 |
| 12 | java (11) | jooby (2.8) | 141 464 | 185 609 | 190 176 |
| 13 | kotlin (1.3) | kooby (2.8) | 139 571 | 181 294 | 186 057 |
| 14 | crystal (0.35) | toro (0.4) | 139 415 | 167 287 | 166 543 |
| 15 | crystal (0.35) | router.cr (0.2) | 137 983 | 165 408 | 163 673 |
| 16 | java (11) | light-4j (2.0) | 137 889 | 173 832 | 178 946 |
| 17 | crystal (0.35) | spider-gazelle (3.3) | 136 456 | 163 188 | 162 035 |
| 18 | php (7.4) | workerman (4.0) | 135 687 | 165 076 | 167 225 |
| 19 | crystal (0.35) | grip (0.28) | 130 188 | 149 298 | 153 046 |
| 20 | crystal (0.35) | kemal (0.26) | 130 074 | 151 565 | 150 099 |
| 21 | nim (1.2) | jester (0.4) | 127 553 | 148 481 | 150 413 |
| 22 | rust (1.45) | actix (2.0) | 125 052 | 134 080 | 133 326 |
| 23 | crystal (0.35) | amber (0.35) | 124 130 | 143 971 | 141 380 |
| 24 | c (11) | agoo-c (0.7) | 112 272 | 155 541 | 152 611 |
| 25 | go (1.14) | rte (0.0) | 108 698 | 111 169 | 116 843 |
| 26 | go (1.14) | clevergo (0.3) | 107 992 | 110 563 | 115 685 |
| 27 | php (7.4) | simps (1.0) | 105 529 | 158 539 | 174 480 |
| 28 | c (99) | kore (3.3) | 103 639 | 109 017 | 137 654 |
| 29 | go (1.14) | gearbox (1.1) | 103 471 | 152 999 | 166 295 |
| 30 | go (1.14) | httprouter (1.3) | 103 219 | 104 640 | 109 085 |
| 31 | go (1.14) | echo (4.1) | 102 670 | 104 395 | 108 246 |
| 32 | java (11) | act (1.9) | 101 599 | 130 515 | 132 140 |
| 33 | go (1.14) | gin (1.6) | 99 998 | 109 751 | 115 103 |
| 34 | go (1.14) | violetear (7.0) | 98 851 | 102 374 | 107 708 |
| 35 | go (1.14) | aero (1.3) | 97 601 | 98 556 | 102 327 |
| 36 | go (1.14) | gorouter (4.4) | 96 912 | 103 694 | 108 517 |
| 37 | go (1.14) | chi (4.1) | 96 824 | 93 468 | 94 702 |
| 38 | go (1.14) | kami (2.2) | 96 014 | 102 798 | 105 358 |
| 39 | go (1.14) | gorilla-mux (1.7) | 95 152 | 96 159 | 101 928 |
| 40 | go (1.14) | webgo (4.1) | 94 415 | 94 939 | 99 208 |
| 41 | fsharp (4.7) | frank (6.1) | 93 277 | 107 477 | 110 448 |
| 42 | fsharp (4.7) | falco (1.2) | 92 545 | 105 200 | 108 074 |
| 43 | csharp (8.0) | aspnetcore (3.1) | 92 288 | 103 057 | 106 059 |
| 44 | go (1.14) | beego (1.12) | 91 337 | 99 619 | 98 269 |
| 45 | go (1.14) | goroute (0.0) | 90 629 | 90 615 | 99 683 |
| 46 | javascript (14.7) | polkadot (1.0) | 88 580 | 100 055 | 101 405 |
| 47 | crystal (0.35) | athena (0.9) | 85 462 | 104 694 | 106 107 |
| 48 | javascript (14.7) | polka (0.5) | 80 400 | 86 943 | 85 842 |
| 49 | javascript (14.7) | 0http (2.5) | 79 808 | 95 537 | 92 069 |
| 50 | go (1.14) | air (0.19) | 78 961 | 80 203 | 83 725 |
| 51 | cpp (14/17) | drogon (1.0) | 78 680 | 85 199 | 87 951 |
| 52 | php (7.4) | one (2.0) | 76 656 | 93 036 | 98 657 |
| 53 | csharp (8.0) | carter (5.1) | 75 652 | 76 685 | 68 673 |
| 54 | elixir (1.1) | cowboy_stream (2.8) | 75 406 | 77 525 | 74 847 |
| 55 | javascript (14.7) | rayo (1.3) | 72 841 | 83 563 | 81 045 |
| 56 | scala (2.13) | akkahttp (10.1) | 72 387 | 83 842 | 81 697 |
| 57 | java (11) | javalin (3.9) | 71 991 | 78 140 | 77 932 |
| 58 | swift (5.2) | perfect (3.1) | 71 835 | 81 898 | 87 430 |
| 59 | ruby (2.7) | agoo (2.13) | 71 685 | 103 305 | 113 051 |
| 60 | php (7.4) | hyperf (2.0) | 71 520 | 89 980 | 95 132 |
| 61 | go (1.14) | gf (1.13) | 71 263 | 78 113 | 80 266 |
| 62 | javascript (14.7) | fastify (3.1) | 69 627 | 75 186 | 72 184 |
| 63 | javascript (14.7) | muneem (2.4) | 67 871 | 74 978 | 73 748 |
| 64 | javascript (14.7) | restana (4.7) | 67 368 | 79 662 | 78 915 |
| 65 | python (3.8) | falcon (2.0) | 66 478 | 75 165 | 75 566 |
| 66 | kotlin (1.3) | ktor (1.2) | 65 912 | 86 291 | 89 913 |
| 67 | fsharp (4.7) | saturn (0.14) | 65 604 | 62 032 | 55 089 |
| 68 | java (11) | spring-boot (2.3) | 64 618 | 71 898 | 71 675 |
| 69 | php (7.4) | comet (0.8) | 62 182 | 67 595 | 67 400 |
| 70 | javascript (14.7) | foxify (0.1) | 62 035 | 67 154 | 67 142 |
| 71 | go (1.14) | mars (1.0) | 61 557 | 63 325 | 66 455 |
| 72 | javascript (14.7) | koa (2.13) | 60 361 | 65 341 | 64 025 |
| 73 | haskell (8.8) | scotty (0.12) | 57 266 | 62 268 | 64 922 |
| 74 | java (11) | micronaut (1.2) | 57 105 | 66 008 | 65 265 |
| 75 | fsharp (4.7) | websharper (4.6) | 56 216 | 57 279 | 54 791 |
| 76 | javascript (14.7) | nestjs-fastify (7.4) | 54 510 | 58 066 | 58 673 |
| 77 | python (3.8) | bottle (0.12) | 54 102 | 58 688 | 56 820 |
| 78 | clojure (1.1) | coast (1.0) | 53 486 | 55 248 | 55 393 |
| 79 | javascript (14.7) | iotjs-express (0.0) | 53 069 | 57 486 | 55 791 |
| 80 | java (11) | spring-framework (5.2) | 51 271 | 58 787 | 58 487 |
| 81 | rust (1.45) | nickel (0.11) | 49 654 | 48 297 | 47 748 |
| 82 | swift (5.2) | kitura (2.9) | 49 346 | 50 133 | 50 097 |
| 83 | swift (5.2) | kitura-nio (2.9) | 49 114 | 50 035 | 49 767 |
| 84 | python (3.8) | apidaora (0.27) | 48 161 | 55 525 | 55 294 |
| 85 | python (3.8) | asgineer (0.7) | 47 228 | 51 188 | 51 818 |
| 86 | elixir (1.1) | cowboy (2.8) | 47 062 | 48 013 | 48 323 |
| 87 | rust (1.45) | gotham (0.4) | 46 724 | 53 252 | 55 457 |
| 88 | python (3.8) | pyramid (1.1) | 46 457 | 49 978 | 49 606 |
| 89 | swift (5.2) | vapor (4.27) | 46 299 | 48 477 | 48 352 |
| 90 | php (7.4) | siler-swoole (1.7) | 45 957 | 67 943 | 72 658 |
| 91 | cpp (11) | evhtp (1.2) | 44 061 | 45 027 | 45 232 |
| 92 | python (3.8) | hug (2.6) | 43 553 | 46 229 | 46 389 |
| 93 | scala (2.13) | http4s (0.21) | 42 716 | 48 797 | 47 827 |
| 94 | python (3.8) | blacksheep (0.2) | 42 046 | 48 111 | 47 823 |
| 95 | python (3.8) | sanic (20.6) | 41 203 | 46 894 | 46 759 |
| 96 | php (7.4) | imi (1.2) | 41 149 | 46 231 | 47 917 |
| 97 | javascript (14.7) | moleculer (0.14) | 40 160 | 41 906 | 41 807 |
| 98 | elixir (1.1) | plug (1.10) | 39 948 | 40 488 | 40 910 |
| 99 | javascript (14.7) | restify (8.5) | 39 594 | 40 540 | 42 807 |
| 100 | python (3.8) | starlette (0.13) | 39 381 | 44 130 | 43 838 |
| 101 | python (3.8) | emmett (2.0) | 36 005 | 39 497 | 39 264 |
| 102 | csharp (8.0) | simplify.web (4.1) | 35 879 | 36 331 | 35 319 |
| 103 | javascript (14.7) | hapi (19.2) | 35 261 | 37 581 | 37 959 |
| 104 | crystal (0.35) | orion (3.0) | 35 119 | 33 157 | 29 685 |
| 105 | scala (2.13) | play (2.8) | 33 978 | 36 458 | 36 422 |
| 106 | elixir (1.1) | phoenix (1.5) | 32 895 | 33 806 | 33 902 |
| 107 | clojure (1.1) | luminus (1.0) | 31 588 | 32 641 | 32 281 |
| 108 | crystal (0.35) | shivneri (0.15) | 30 648 | 29 095 | 26 566 |
| 109 | haskell (8.8) | servant (0.17) | 29 952 | 29 327 | 28 164 |
| 110 | fsharp (4.7) | giraffe (4.1) | 28 473 | 27 953 | 27 837 |
| 111 | fsharp (4.7) | suave (2.5) | 28 285 | 29 277 | 31 285 |
| 112 | dart (2.9) | aqueduct (3.3) | 27 637 | 27 605 | 27 693 |
| 113 | php (7.4) | swoft (2.0) | 27 519 | 33 857 | 34 708 |
| 114 | nim (1.2) | rosencrantz (0.4) | 27 234 | 23 234 | 21 258 |
| 115 | nim (1.2) | akane (0.1) | 26 452 | 22 850 | 20 726 |
| 116 | php (7.4) | yii-swoole (2.0) | 26 106 | 31 698 | 32 537 |
| 117 | python (3.8) | fastapi (0.6) | 26 078 | 28 621 | 28 450 |
| 118 | ruby (2.7) | syro (3.2) | 25 872 | 29 482 | 28 604 |
| 119 | python (3.8) | responder (2.0) | 25 826 | 29 654 | 29 477 |
| 120 | javascript (14.7) | turbo_polka (0.3) | 25 762 | 24 974 | 23 662 |
| 121 | php (7.4) | sw-fw-less (preview) | 25 516 | 31 629 | 31 882 |
| 122 | php (7.4) | chubbyphp-swoole (3.1) | 24 491 | 29 157 | 29 391 |
| 123 | python (3.8) | molten (1.0) | 24 332 | 25 638 | 25 234 |
| 124 | rust (1.45) | iron (0.6) | 24 156 | 24 226 | 24 224 |
| 125 | ruby (2.7) | hanami-api (0.1) | 23 795 | 28 690 | 27 491 |
| 126 | javascript (14.7) | express (4.17) | 23 777 | 25 199 | 24 733 |
| 127 | javascript (14.7) | feathersjs (4.5) | 23 678 | 25 317 | 25 058 |
| 128 | python (3.8) | clastic (19.9) | 23 558 | 24 731 | 24 290 |
| 129 | php (7.4) | slim-swoole (4.5) | 23 491 | 27 761 | 28 484 |
| 130 | python (3.8) | aiohttp (3.6) | 23 467 | 26 145 | 26 133 |
| 131 | python (3.8) | flask (1.1) | 22 008 | 22 949 | 22 764 |
| 132 | python (3.8) | masonite (2.3) | 21 928 | 23 044 | 22 629 |
| 133 | ruby (2.7) | cuba (3.9) | 21 758 | 24 544 | 23 046 |
| 134 | ruby (2.7) | roda (3.34) | 21 501 | 26 936 | 26 241 |
| 135 | go (1.14) | macaron (1.3) | 20 414 | 23 026 | 23 193 |
| 136 | ruby (2.7) | rack-routing (0.0) | 19 517 | 20 584 | 20 083 |
| 137 | ruby (2.7) | rack_app (7.6) | 19 318 | 20 991 | 20 614 |
| 138 | javascript (14.7) | nestjs-express (7.4) | 18 421 | 19 860 | 19 488 |
| 139 | ruby (2.7) | camping (2.1) | 17 059 | 18 462 | 18 593 |
| 140 | java (11) | blade (2.0) | 14 806 | 16 857 | 16 619 |
| 141 | dart (2.9) | start (0.4) | 14 197 | 13 860 | 12 993 |
| 142 | go (1.14) | tango (0.6) | 14 126 | 14 645 | 14 637 |
| 143 | php (7.4) | spiral (2.5) | 12 797 | 13 066 | 13 088 |
| 144 | ruby (2.7) | sinatra (2.0) | 12 751 | 12 972 | 13 014 |
| 145 | crystal (0.35) | runcobo (0.2) | 12 623 | 12 463 | 11 872 |
| 146 | go (1.14) | gramework (1.7) | 12 596 | 12 900 | 12 884 |
| 147 | java (11) | struts2 (2.5) | 12 079 | 13 085 | 12 930 |
| 148 | ruby (2.7) | grape (1.4) | 11 544 | 11 850 | 11 834 |
| 149 | python (3.8) | quart (0.13) | 10 324 | 11 147 | 10 618 |
| 150 | php (7.4) | chubbyphp-roadrunner (3.1) | 9 908 | 10 647 | 10 698 |
| 151 | swift (5.2) | swifter (1.4) | 9 744 | 9 810 | 9 822 |
| 152 | php (7.4) | slim-roadrunner (4.5) | 9 729 | 10 389 | 10 510 |
| 153 | php (7.4) | sunrise-router-roadrunner (2.4) | 9 612 | 9 369 | 9 185 |
| 154 | python (3.8) | tornado (6.0) | 9 444 | 9 809 | 9 529 |
| 155 | pony (0.36) | jennet (0.1) | 8 825 | 17 361 | 15 826 |
| 156 | python (3.8) | cherrypy (18.6) | 8 766 | 8 736 | 8 628 |
| 157 | php (7.4) | basicphp (0.9) | 8 157 | 8 122 | 7 915 |
| 158 | python (3.8) | django (3.1) | 8 037 | 8 024 | 7 949 |
| 159 | php (7.4) | fastsitephp (1.3) | 7 874 | 7 764 | 7 631 |
| 160 | ruby (2.7) | flame (4.18) | 7 572 | 7 693 | 7 279 |
| 161 | ruby (2.7) | hanami (1.3) | 7 383 | 7 752 | 7 581 |
| 162 | php (7.4) | ubiquity (2.3) | 6 701 | 6 570 | 6 578 |
| 163 | python (3.8) | tonberry (0.2) | 6 642 | 6 523 | 6 147 |
| 164 | php (7.4) | one-fpm (2.0) | 6 007 | 5 923 | 5 905 |
| 165 | php (7.4) | hamlet (3.2) | 5 721 | 5 636 | 5 644 |
| 166 | php (7.4) | phalcon (4.0) | 5 698 | 5 652 | 5 659 |
| 167 | php (7.4) | siler (1.7) | 5 668 | 5 590 | 5 612 |
| 168 | javascript (14.7) | sails (1.2) | 5 512 | 5 591 | 5 477 |
| 169 | php (7.4) | bearframework (1.2) | 5 108 | 5 060 | 5 073 |
| 170 | php (7.4) | ice (1.5) | 4 977 | 4 943 | 4 982 |
| 171 | php (7.4) | sunrise-router (2.4) | 4 897 | 4 886 | 4 856 |
| 172 | php (7.4) | chubbyphp (3.1) | 4 529 | 4 508 | 4 552 |
| 173 | v (0.1) | vape (0.3) | 3 871 | 2 198 | 1 639 |
| 174 | php (7.4) | slim (4.5) | 3 846 | 3 859 | 3 920 |
| 175 | php (7.4) | nette (3.0) | 3 576 | 3 594 | 3 659 |
| 176 | php (7.4) | lumen (7.2) | 3 479 | 3 500 | 3 601 |
| 177 | php (7.4) | yii (2.0) | 3 406 | 3 426 | 3 498 |
| 178 | julia (1.5) | merly (0.2) | 3 351 | 8 196 | 6 254 |
| 179 | ruby (2.7) | rails (6.0) | 3 246 | 3 131 | 3 139 |
| 180 | php (7.4) | sunrise-router-annotations (2.4) | 3 091 | 3 139 | 3 186 |
| 181 | php (7.4) | symfony (5.1) | 2 730 | 2 769 | 2 815 |
| 182 | php (7.4) | mezzio (3.2) | 2 496 | 2 539 | 2 566 |
| 183 | python (3.8) | cyclone (1.3) | 2 404 | 2 413 | 2 372 |
| 184 | ruby (2.7) | pakyow (1.0) | 2 071 | 2 112 | 1 850 |
| 185 | python (3.8) | klein (20.6) | 1 591 | 1 616 | 1 592 |
| 186 | python (3.8) | nameko (2.12) | 1 535 | 1 502 | 1 478 |
| 187 | php (7.4) | laminas (3.1) | 1 332 | 1 355 | 1 368 |
| 188 | php (7.4) | codeigniter4 (4.0) | 1 024 | 1 046 | 1 122 |
| 189 | perl (5.32) | dancer2 (2.0) | 935 | 757 | 727 |
| 190 | php (7.4) | laravel (7.23) | 363 | 177 | 2 438 |
| 191 | crystal (0.35) | lucky (0.23) | 314 | 338 | 327 |
In any way you want ...
- Request a framework addition
- Report a bug (on any implementation)
- Suggest an idea
- ...
Any kind of idea is ❤️
- Taichiro Suzuki - Author | Maintainer
- OvermindDL1 - Maintainer
- Marwan Rabbâa - Maintainer