Kubernetes HA leader control plane services restarting -

-

while following kubernetes-the-hard-way , porting local ubuntu 16.04 vms, bringing ha control plane giving me problems. kube-apiserver , kube-controller-manager services elected leader keep failing , restarting control planes on other 2 non-leader masters come fine. behavior follows leader around cluster.

etcd looks happy:

member 7f44a7567a5e995 healthy: got healthy result https://10.1.15.117:2379 member 43d2258c438cbf4e healthy: got healthy result https://10.1.15.116:2379 member a83b22e9f907f471 healthy: got healthy result https://10.1.15.115:2379 cluster healthy

i've verified current leader by:

kubectl endpoints kube-controller-manager --namespace=kube-system  -o yaml   apiversion: v1 kind: endpoints metadata:   annotations:     control-plane.alpha.kubernetes.io/leader: '{"holderidentity":"df-dev-kube-test02","leasedurationseconds":15,"acquiretime":"2017-09-13t17:15:06z","renewtime":"2017-09-13t18:10:46z","leadertransitions":7}'   creationtimestamp: 2017-09-13t14:59:57z   name: kube-controller-manager   namespace: kube-system   resourceversion: "7814"   selflink: /api/v1/namespaces/kube-system/endpoints/kube-controller-manager   uid: 3599a959-9894-11e7-a34f-005056b24128 subsets: []


kube-apiserver logs: pastebin apiserver logs


kube-controller-manager logs: pastebin controller logs

considering various "storage" related errors in logs , since related leader elections, thought perhaps issue might've been etcd related. etcd kept reporting fine though. grins removed --experimental-encryption-provider-config=/var/lib/kubernetes/encryption-config.yaml \\from api-server.service , control pane happy.

i'm not sure if encryption breaking things or if automation of encryption hosed it.


Comments

Post a Comment

Popular posts from this blog

ZeroMQ on Windows, with Qt Creator -

-
i have installed zmq link given below on windows 7. http://zeromq.org/distro:microsoft-windows 1) have c++ project in qt-creator , i'm using zmq c++ binding (zhelpers.hpp etc). 2) have added .lib installed zmq folder qtcreator 3) while compiling, compiler looks _imp__zmq_verion, in .dll files present in .lib have __imp_zmq_version. 4) causing undefined reference zmq_version. anyone else faced problem , has solution ?
Read more

unity3d - Unity SceneManager.LoadScene quits application -

-
i have 2 different scenes 1 named 'main' , other 'game' both added scenes in load. when use scenemanager.loadscene("game"); works no problem when tried return main scene code if (input.getkeydown (keycode.escape)) { scenemanager.loadscene("main"); } strangely game quits. thanks. sorry interruption. somewhere on other script found code. if (input.getkeydown (keycode.escape)) { application.quit (); } that's bad, know...
Read more

python - Error while using APScheduler: 'NoneType' object has no attribute 'now' -

-
i have python app hosting on heroku. use apscheduler call function every day @ time. strange thing 1 time function runs after while have following error log: exception in thread apscheduler (most raised during interpreter shutdown): 2017-09-06t15:03:28.347848+00:00 app[web.1]: traceback (most recent call last): 2017-09-06t15:03:28.347858+00:00 app[web.1]: file "/app/.heroku/python/lib/python2.7/threading.py", line 810, in __bootstrap_inner 2017-09-06t15:03:28.347859+00:00 app[web.1]: file "/app/.heroku/python/lib/python2.7/threading.py", line 763, in run 2017-09-06t15:03:28.347861+00:00 app[web.1]: file "/app/.heroku/python/lib/python2.7/site-packages/apscheduler/schedulers/blocking.py", line 27, in _main_loop 2017-09-06t15:03:28.348167+00:00 app[web.1]: <type 'exceptions.attributeerror'>: 'nonetype' object has no attribute 'now' any guesses? the full code long here key parts: def startdayauto(): global ...
Read more