multinode: fix control plane not ready on restart #8698

sharifelgamal · 2020-07-10T22:41:46Z

Fixes #8600

k8s-ci-robot · 2020-07-10T22:42:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sharifelgamal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [sharifelgamal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

medyagh · 2020-07-10T22:45:22Z

/ok-to-test

minikube-pr-bot · 2020-07-10T22:51:45Z

kvm2 Driver
docker Driver

medyagh · 2020-07-11T01:22:27Z

Restart mutlinode test failed in gh action and jenkins

minikube-pr-bot · 2020-07-13T17:47:38Z

kvm2 Driver
Times for minikube: [63.395773223999996 62.948747802999996 62.95453990000001]
Average time for minikube: 63.099686975666664

Times for Minikube (PR 8698): [66.956771815 61.817401701 67.55278709000001]
Average time for Minikube (PR 8698): 65.442320202

Averages Time Per Log

Details

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8698) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0 on Debian   |  0.061149 |           0.058207 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.019940 |           0.020221 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.018578 |           0.011582 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 40.687434 |          39.753913 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.3 | 20.716186 |          23.716130 |
| on Docker 19.03.12 ...         |           |                    |
| * Verifying Kubernetes         |  1.290910 |           1.462192 |
| components...                  |           |                    |
| * Enabled addons:              |  0.216773 |           0.342546 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.084756 |           0.072508 |
| configured to use "minikube"   |           |                    |
|                                |  0.003961 |           0.005021 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [27.354459246000005 26.260922277000006 24.684449708999995]
Average time for minikube: 26.099943744

Times for Minikube (PR 8698): [26.068001815 26.878878755000002 25.971940163000003]
Average time for Minikube (PR 8698): 26.30627357766667

Averages Time Per Log

Details

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8698) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0 on Debian   |  0.073177 |           0.070752 |
|                           9.11 |           |                    |
| * Using the docker driver      |  0.002761 |           0.002433 |
| based on existing profile      |           |                    |
| * Starting control plane node  |  0.057940 |           0.058091 |
| minikube in cluster minikube   |           |                    |
| * Creating docker container    |  7.714459 |           7.489948 |
| (CPUs=2, Memory=3700MB) ...    |           |                    |
| * Preparing Kubernetes v1.18.3 | 17.316825 |          17.685894 |
| on Docker 19.03.2 ...          |           |                    |
| * Verifying Kubernetes         |  0.863298 |           0.910828 |
| components...                  |           |                    |
| * Enabled addons:              |  0.002935 |           0.019278 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.064496 |           0.064479 |
| configured to use "minikube"   |           |                    |
|                                |  0.004054 |           0.004570 |
+--------------------------------+-----------+--------------------+

minikube-pr-bot · 2020-07-14T02:13:00Z

kvm2 Driver
Times for minikube: [66.747257383 58.394459907000005 61.846409698]
Average time for minikube: 62.32937566266667

Times for Minikube (PR 8698): [64.426172565 63.275204669 63.86242006]
Average time for Minikube (PR 8698): 63.854599097999994

Averages Time Per Log

Details

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8698) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0 on Debian   |  0.060890 |           0.058120 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.020402 |           0.019504 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.009148 |           0.007484 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 38.847625 |          40.857418 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.3 | 21.462387 |          21.569621 |
| on Docker 19.03.12 ...         |           |                    |
| * Verifying Kubernetes         |  1.293836 |           1.208418 |
| components...                  |           |                    |
| * Enabled addons:              |  0.539397 |           0.060953 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.091944 |           0.069607 |
| configured to use "minikube"   |           |                    |
|                                |  0.003747 |           0.003473 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.366106334000005 26.736700506000002 25.027741804]
Average time for minikube: 26.04351621466667

Times for Minikube (PR 8698): [25.670679615999997 25.058768182 26.182394657]
Average time for Minikube (PR 8698): 25.637280818333334

Averages Time Per Log

Details

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8698) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0 on Debian   |  0.070244 |           0.072590 |
|                           9.11 |           |                    |
| * Using the docker driver      |  0.002919 |           0.002618 |
| based on existing profile      |           |                    |
| * Starting control plane node  |  0.055386 |           0.055818 |
| minikube in cluster minikube   |           |                    |
| * Creating docker container    |  7.552148 |           7.529301 |
| (CPUs=2, Memory=3700MB) ...    |           |                    |
| * Preparing Kubernetes v1.18.3 | 17.362146 |          16.977670 |
| on Docker 19.03.2 ...          |           |                    |
| * Verifying Kubernetes         |  0.901628 |           0.847624 |
| components...                  |           |                    |
| * Enabled addons:              |  0.031435 |           0.082176 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.062622 |           0.064098 |
| configured to use "minikube"   |           |                    |
|                                |  0.004990 |           0.005386 |
+--------------------------------+-----------+--------------------+

TravisBuddy · 2020-07-15T22:08:57Z

Travis tests have failed

Hey @sharifelgamal,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

make test

which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.14.4.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
which go-bindata || GO111MODULE=off GOBIN="/home/travis/gopath/bin" go get github.com/jteeuwen/go-bindata/...
/home/travis/gopath/bin/go-bindata
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.14.4.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
MINIKUBE_LDFLAGS="-X k8s.io/minikube/pkg/version.version=v1.12.0 -X k8s.io/minikube/pkg/version.isoVersion=v1.12.0 -X k8s.io/minikube/pkg/version.isoPath=minikube/iso -X k8s.io/minikube/pkg/version.gitCommitID="2f52099048ac7f70d9243aa549fb8dd1c0af4015"" ./test.sh
= make lint =============================================================
golangci/golangci-lint info checking GitHub for tag 'v1.26.0'
golangci/golangci-lint info found version: 1.26.0 for v1.26.0/linux/amd64
golangci/golangci-lint info installed out/linters/golangci-lint
test/integration/multinode_test.go:245:6: ineffectual assignment to `err` (ineffassign)
	rr, err = Run(t, exec.CommandContext(ctx, "kubectl", "get", "nodes"))
	    ^
test/integration/multinode_test.go:250:6: ineffectual assignment to `err` (ineffassign)
	rr, err = Run(t, exec.CommandContext(ctx, "kubectl", "get", "nodes", "-o", `go-template='{{range .items}}{{range .status.conditions}}{{if eq .type "Ready"}} {{.status}}{{"\n"}}{{end}}{{end}}{{end}}'`))
	    ^
Makefile:390: recipe for target 'lint-ci' failed
make[1]: *** [lint-ci] Error 1
= go mod ================================================================
ok
= boilerplate ===========================================================
ok
Makefile:274: recipe for target 'test' failed
make: *** [test] Error 4

TravisBuddy Request Identifier: c601e760-c6e7-11ea-aa06-e17841301e13

minikube-pr-bot · 2020-07-15T22:17:48Z

kvm2 Driver
Times for minikube: [63.560546136 65.823352181 67.455718995]
Average time for minikube: 65.61320577066665

Times for Minikube (PR 8698): [62.60711526800001 68.28837775599999 63.468774929]
Average time for Minikube (PR 8698): 64.78808931766666

Averages Time Per Log

Details

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8698) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0 on Debian   |  0.063931 |           0.065394 |
|                           9.11 |           |                    |
| * Using the kvm2 driver based  |  0.020236 |           0.022005 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.005029 |           0.003377 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 40.870193 |          41.192600 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.3 | 22.616917 |          21.811290 |
| on Docker 19.03.12 ...         |           |                    |
| * Verifying Kubernetes         |  1.393783 |           1.480006 |
| components...                  |           |                    |
| * Enabled addons:              |  0.561050 |           0.134758 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.078922 |           0.074132 |
| configured to use "minikube"   |           |                    |
|                                |  0.003144 |           0.004529 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [29.905466684 26.331447788 27.557172334999997]
Average time for minikube: 27.931362269

Times for Minikube (PR 8698): [26.868504083999998 27.713833857 26.800029612]
Average time for Minikube (PR 8698): 27.127455850999997

Averages Time Per Log

Details

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8698) |
+--------------------------------+-----------+--------------------+
| * minikube v1.12.0 on Debian   |  0.078444 |           0.082314 |
|                           9.11 |           |                    |
| * Using the docker driver      |  0.002686 |           0.002556 |
| based on existing profile      |           |                    |
| * Starting control plane node  |  0.063158 |           0.063104 |
| minikube in cluster minikube   |           |                    |
| * Creating docker container    |  7.964612 |           7.707685 |
| (CPUs=2, Memory=3700MB) ...    |           |                    |
| * Preparing Kubernetes v1.18.3 | 18.775553 |          17.991636 |
| on Docker 19.03.2 ...          |           |                    |
| * Verifying Kubernetes         |  0.965697 |           1.175617 |
| components...                  |           |                    |
| * Enabled addons:              |  0.012315 |           0.003120 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.064707 |           0.090518 |
| configured to use "minikube"   |           |                    |
|                                |  0.004190 |           0.010905 |
+--------------------------------+-----------+--------------------+

update control plane properly on multinode restart

aa4f7b3

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 10, 2020

k8s-ci-robot requested review from afbjorklund and prasadkatti July 10, 2020 22:42

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 10, 2020

medyagh approved these changes Jul 10, 2020

View reviewed changes

medyagh changed the title ~~update control plane properly on multinode restart~~ multinode: fix control plane not ready on restart Jul 10, 2020

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Jul 10, 2020

add wait to multinode restart test

970d954

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 13, 2020

sharifelgamal added 3 commits July 13, 2020 14:56

fix up multinode test

ad9e71d

Merge branch 'master' of github.com:kubernetes/minikube into master

bd721de

several fixes

9f2f604

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 14, 2020

sharifelgamal added 2 commits July 15, 2020 11:32

Merge branch 'master' of github.com:kubernetes/minikube into master

af99a13

sleeping for 30 seconds works for some reason?

58c501c

fix lint

b7445a7

sharifelgamal merged commit 97484a2 into kubernetes:master Jul 16, 2020

fmotrifork mentioned this pull request Aug 3, 2020

minikube 1.12.1 fishworks/fish-food#875

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

multinode: fix control plane not ready on restart #8698

multinode: fix control plane not ready on restart #8698

Uh oh!

sharifelgamal commented Jul 10, 2020

Uh oh!

k8s-ci-robot commented Jul 10, 2020

Uh oh!

medyagh commented Jul 10, 2020

Uh oh!

minikube-pr-bot commented Jul 10, 2020

Uh oh!

medyagh commented Jul 11, 2020

Uh oh!

minikube-pr-bot commented Jul 13, 2020

Uh oh!

minikube-pr-bot commented Jul 14, 2020

Uh oh!

TravisBuddy commented Jul 15, 2020

Uh oh!

minikube-pr-bot commented Jul 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

multinode: fix control plane not ready on restart #8698

multinode: fix control plane not ready on restart #8698

Uh oh!

Conversation

sharifelgamal commented Jul 10, 2020

Uh oh!

k8s-ci-robot commented Jul 10, 2020

Uh oh!

medyagh commented Jul 10, 2020

Uh oh!

minikube-pr-bot commented Jul 10, 2020

Uh oh!

medyagh commented Jul 11, 2020

Uh oh!

minikube-pr-bot commented Jul 13, 2020

Uh oh!

minikube-pr-bot commented Jul 14, 2020

Uh oh!

TravisBuddy commented Jul 15, 2020

Travis tests have failed

1st Build

TravisBuddy Request Identifier: c601e760-c6e7-11ea-aa06-e17841301e13

Uh oh!

minikube-pr-bot commented Jul 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants