OpenShift v3.11 インストール➂
◆再インストールしてみる
別の場所でこける
[root@all-openshift openshift-ansible]# ansible-playbook playbooks/deploy_cluster.yml ～～～ TASK [openshift_control_plane : Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered] *********************************************************************************************************************************************** Sunday 26 January 2020 03:20:04 -0800 (0:00:01.533) 0:09:27.253 ******** FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (60 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (59 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (58 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (57 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (56 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (55 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (54 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (53 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (52 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (51 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (50 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (49 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (48 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (47 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (46 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (45 retries left). FAILED - RETRYING: Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered (44 retries left). ～～～ fatal: [localhost]: FAILED! => {"attempts": 60, "changed": true, "cmd": ["oc", "--config=/etc/origin/master/admin.kubeconfig", "get", "--raw", "/apis/servicecatalog.k8s.io/v1beta1"], "delta": "0:00:00.382671", "end": "2020-01-26 03:26:14.651049", "msg": "non-zero return code", "rc": 1, "start": "2020-01-26 03:26:14.268378", "stderr": "Error from server (ServiceUnavailable): the server is currently unable to handle the request", "stderr_lines": ["Error from server (ServiceUnavailable): the server is currently unable to handle the request"], "stdout": "", "stdout_lines": []} <span style="color: #ffffff"><b>/var/log/messages</b></span> Jan 26 03:21:17 localhost python: ansible-command Invoked with warn=True executable=None _uses_shell=False _raw_params=oc --config=/etc/origin/master/admin.kubeconfig get --raw /apis/servicecatalog.k8s.io/v1beta1 removes=None argv=None creates=None chdir=None stdin=None Jan 26 03:21:17 localhost origin-node: I0126 03:21:17.787417 124479 fsHandler.go:135] du and find on following dirs took 1.571076464s: [/var/lib/docker/overlay2/381c1312b9a7f30dada4bfcc5a50cb04b18883f809c1999a6c19dd8369119dfd/diff /var/lib/docker/containers/a4c5ef77f2b3246916c24eabccda9e47b0cb8a90bf062e3da599c959e12a2b11]; will not log again for this container unless duration exceeds 2s Jan 26 03:21:23 localhost python: ansible-command Invoked with warn=True executable=None _uses_shell=False _raw_params=oc --config=/etc/origin/master/admin.kubeconfig get --raw /apis/servicecatalog.k8s.io/v1beta1 removes=None argv=None creates=None chdir=None stdin=None Jan 26 03:21:29 localhost python: ansible-command Invoked with warn=True executable=None _uses_shell=False _raw_params=oc --config=/etc/origin/master/admin.kubeconfig get --raw /apis/servicecatalog.k8s.io/v1beta1 removes=None argv=None creates=None chdir=None stdin=None Jan 26 03:21:35 localhost python: ansible-command Invoked with warn=True executable=None _uses_shell=False _raw_params=oc --config=/etc/origin/master/admin.kubeconfig get --raw /apis/servicecatalog.k8s.io/v1beta1 removes=None argv=None creates=None chdir=None stdin=None Jan 26 03:21:41 localhost python: ansible-command Invoked with warn=True executable=None _uses_shell=False _raw_params=oc --config=/etc/origin/master/admin.kubeconfig get --raw /apis/servicecatalog.k8s.io/v1beta1 removes=None argv=None creates=None chdir=None stdin=None
https://bugzilla.redhat.com/show_bug.cgi?id=1628881
このバグかなあ？
[root@all-openshift ~]# oc get --raw /apis/metrics.k8s.io/v1beta1 Error from server (NotFound): the server could not find the requested resource [root@all-openshift ~]# [root@all-openshift ~]# oc get apiservices NAME CREATED AT v1. 2020-01-26T09:06:15Z v1.apps 2020-01-26T09:06:15Z v1.apps.openshift.io 2020-01-26T09:06:15Z v1.authentication.k8s.io 2020-01-26T09:06:15Z v1.authorization.k8s.io 2020-01-26T09:06:15Z v1.authorization.openshift.io 2020-01-26T09:06:15Z v1.autoscaling 2020-01-26T09:06:15Z v1.batch 2020-01-26T09:06:15Z v1.build.openshift.io 2020-01-26T09:06:15Z v1.image.openshift.io 2020-01-26T09:06:15Z v1.monitoring.coreos.com 2020-01-26T09:16:33Z v1.network.openshift.io 2020-01-26T09:06:15Z v1.networking.k8s.io 2020-01-26T09:06:15Z v1.oauth.openshift.io 2020-01-26T09:06:15Z v1.project.openshift.io 2020-01-26T09:06:15Z v1.quota.openshift.io 2020-01-26T09:06:15Z v1.rbac.authorization.k8s.io 2020-01-26T09:06:15Z v1.route.openshift.io 2020-01-26T09:06:15Z v1.security.openshift.io 2020-01-26T09:06:15Z v1.storage.k8s.io 2020-01-26T09:06:15Z v1.template.openshift.io 2020-01-26T09:06:15Z v1.user.openshift.io 2020-01-26T09:06:15Z v1beta1.admissionregistration.k8s.io 2020-01-26T09:06:15Z v1beta1.apiextensions.k8s.io 2020-01-26T09:06:15Z v1beta1.apps 2020-01-26T09:06:15Z v1beta1.authentication.k8s.io 2020-01-26T09:06:15Z v1beta1.authorization.k8s.io 2020-01-26T09:06:15Z v1beta1.batch 2020-01-26T09:06:15Z v1beta1.certificates.k8s.io 2020-01-26T09:06:15Z v1beta1.events.k8s.io 2020-01-26T09:06:15Z v1beta1.extensions 2020-01-26T09:06:15Z v1beta1.policy 2020-01-26T09:06:15Z v1beta1.rbac.authorization.k8s.io 2020-01-26T09:06:15Z v1beta1.scheduling.k8s.io 2020-01-26T09:06:15Z v1beta1.servicecatalog.k8s.io 2020-01-26T09:24:13Z v1beta1.storage.k8s.io 2020-01-26T09:06:15Z v1beta2.apps 2020-01-26T09:06:15Z v2beta1.autoscaling 2020-01-26T09:06:15Z [root@all-openshift ~]#
必要なコンポーネント入っているし、もういいんじゃないか疑惑
[root@all-openshift ~]# oc get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default docker-registry-1-6nldk 1/1 Running 0 15m default registry-console-1-tfmf4 1/1 Running 0 15m default router-1-pmkw9 1/1 Running 3 2h kube-system master-api-all-openshift 1/1 Running 3 31m kube-system master-controllers-all-openshift 1/1 Running 5 30m kube-system master-etcd-all-openshift 1/1 Running 0 30m openshift-console console-67c994c776-5lrj2 1/1 Running 2 2h openshift-monitoring alertmanager-main-0 3/3 Running 0 2h openshift-monitoring alertmanager-main-1 3/3 Running 0 2h openshift-monitoring alertmanager-main-2 3/3 Running 0 2h openshift-monitoring cluster-monitoring-operator-8578656f6f-888ts 1/1 Running 0 2h openshift-monitoring grafana-6b9f85786f-c8x6p 2/2 Running 0 2h openshift-monitoring kube-state-metrics-c4f86b5f8-xv79d 3/3 Running 0 2h openshift-monitoring node-exporter-q4f5j 2/2 Running 0 2h openshift-monitoring prometheus-k8s-0 4/4 Running 1 2h openshift-monitoring prometheus-k8s-1 4/4 Running 1 2h openshift-monitoring prometheus-operator-6644b8cd54-84skd 1/1 Running 0 2h openshift-node sync-lv2lb 1/1 Running 0 2h openshift-sdn ovs-pkvdq 1/1 Running 0 2h openshift-sdn sdn-ggdv7 1/1 Running 0 2h openshift-web-console webconsole-7fc8759f7b-6b4cg 1/1 Running 0 2h [root@all-openshift ~]# [root@all-openshift ~]#
So you have 2 choices installing the cluster after running prerequisites: run the deploy-cluster playbook (i think this is what you've used) run individual playbooks found in the docs: https://docs.okd.io/3.11/install/running_install.html#running-the-installation-playbooks (Individual Component Playbook Run Order) What you really need from all those playbooks to have a minimal cluster is : Health Check | ~/openshift-ansible/playbooks/openshift-checks/pre-install.yml Node Bootstrap | ~/openshift-ansible/playbooks/openshift-node/bootstrap.yml etcd Install | ~/openshift-ansible/playbooks/openshift-etcd/config.yml Master Install | ~/openshift-ansible/playbooks/openshift-master/config.yml Master Additional Install | ~/openshift-ansible/playbooks/openshift-master/additional_config.yml Node Join | ~/openshift-ansible/playbooks/openshift-node/join.yml Hosted Install | ~/openshift-ansible/playbooks/openshift-hosted/config.yml Web Console Install | ~/openshift-ansible/playbooks/openshift-web-console/config.yml After all these run with success you can then move to installing logging/metrics etc. Now if you feel you need to reinstall from scratch run the uninstall playbook : ~/openshift-ansible/playbooks/adhoc/uninstall.yml Just remember after running uninstall playbook you need to: restore /etc/resolv.conf back to whatever you had before installing openshift. yum remove docker-* crio -y && yum clean all && rm -Rf /var/lib/dockershim/ /var/lib/docker /var/lib/containers /etc/sysconfig/docker* /etc/sysconfig/cri-o* /etc/docker /etc/crio /etc/origin /etc/cni reboot the machine try again reinstalling openshift
# Catalog からPODデプロイ成功 [root@all-openshift ~]# oc project devdev Already on project "devdev" on server "https://all-openshift:8443". [root@all-openshift ~]# oc get all NAME READY STATUS RESTARTS AGE pod/httpd-example-1-build 1/1 Running 0 1m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/httpd-example ClusterIP 172.30.51.68 <none> 8080/TCP 2m NAME REVISION DESIRED CURRENT TRIGGERED BY deploymentconfig.apps.openshift.io/httpd-example 0 1 0 config,image(httpd-example:latest) NAME TYPE FROM LATEST buildconfig.build.openshift.io/httpd-example Source Git 1 NAME TYPE FROM STATUS STARTED DURATION build.build.openshift.io/httpd-example-1 Source Git@0ac6da9 Running About a minute ago NAME DOCKER REPO TAGS UPDATED imagestream.image.openshift.io/httpd-example docker-registry.default.svc:5000/devdev/httpd-example NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD route.route.openshift.io/httpd-example httpd-example-devdev.router.default.svc.cluster.local httpd-example <all> None [root@all-openshift ~]#
後日談
・アンインストールして再度入れなおしました。
勝手な想像ですが、/etc/resolv.conf が勝手にローカルホストに変わった影響かと思った。(SkyDNSとdnsmasq）
なので、おとなしくアンインストールしてから入れ直すのが確実。。。
Now if you feel you need to reinstall from scratch run the uninstall playbook : ~/openshift-ansible/playbooks/adhoc/uninstall.yml
INSTALLER STATUS **************************************************************************************************************************************************************************************************************************** Initialization : Complete (0:00:21) Health Check : Complete (0:00:59) Node Bootstrap Preparation : Complete (0:11:13) etcd Install : Complete (0:01:22) Master Install : Complete (0:11:31) Master Additional Install : Complete (0:02:06) Node Join : Complete (0:00:58) Hosted Install : Complete (0:01:40) Cluster Monitoring Operator : Complete (0:00:54) Web Console Install : Complete (0:04:37) Console Install : Complete (0:02:34) Sunday 26 January 2020 04:38:49 -0800 (0:00:00.332) 0:40:23.788 ******** =============================================================================== openshift_node : Install node, clients, and conntrack packages --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 525.79s openshift_ca : Install the base package for admin tooling -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 239.27s openshift_web_console : Verify that the console is running ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 236.43s openshift_control_plane : Wait for all control plane pods to come up and become ready ---------------------------------------------------------------------------------------------------------------------------------------------- 111.73s openshift_console : Waiting for console rollout to complete ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 81.59s Run health checks (install) - EL ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 59.00s openshift_cluster_monitoring_operator : Wait for the ServiceMonitor CRD to be created ----------------------------------------------------------------------------------------------------------------------------------------------- 32.60s openshift_manageiq : Configure role/user permissions -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 23.77s openshift_console : Ensure openshift-console project exists ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 20.81s openshift_control_plane : Wait for APIs to become available ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 15.02s openshift_node_group : Wait for sync DS to set annotations on master nodes ---------------------------------------------------------------------------------------------------------------------------------------------------------- 14.45s openshift_hosted : Grant the router service account(s) access to the appropriate scc ------------------------------------------------------------------------------------------------------------------------------------------------ 12.85s openshift_excluder : Install openshift excluder - yum ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 12.74s openshift_hosted : Create OpenShift router ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 12.45s openshift_hosted : Create default projects ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 12.36s Approve node certificates when bootstrapping ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.76s openshift_excluder : Install docker excluder - yum ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.16s openshift_console : Copy console templates to temp directory ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.10s openshift_console : Apply the console template file ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.06s tuned : Restart tuned service -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 7.88s [root@all-openshift openshift-ansible]#
[root@all-openshift ~]# oc get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default docker-registry-1-pltl9 1/1 Running 0 17m default registry-console-1-lrldp 1/1 Running 0 17m default router-1-5rrjc 1/1 Running 0 18m dev-project1 httpd-example-1-build 1/1 Running 0 1m kube-system master-api-all-openshift 1/1 Running 0 26m kube-system master-controllers-all-openshift 1/1 Running 2 26m kube-system master-etcd-all-openshift 1/1 Running 0 26m openshift-console console-6b758cf588-4xkcv 1/1 Running 0 10m openshift-monitoring alertmanager-main-0 3/3 Running 0 8m openshift-monitoring alertmanager-main-1 3/3 Running 0 6m openshift-monitoring alertmanager-main-2 3/3 Running 0 6m openshift-monitoring cluster-monitoring-operator-8578656f6f-nhcs9 1/1 Running 0 17m openshift-monitoring grafana-6b9f85786f-hhjbf 2/2 Running 0 16m openshift-monitoring kube-state-metrics-c4f86b5f8-kj7vj 3/3 Running 0 5m openshift-monitoring node-exporter-ks4s5 2/2 Running 0 6m openshift-monitoring prometheus-k8s-0 4/4 Running 1 15m openshift-monitoring prometheus-k8s-1 4/4 Running 1 9m openshift-monitoring prometheus-operator-6644b8cd54-bp99q 1/1 Running 0 17m openshift-node sync-7jhwp 1/1 Running 0 23m openshift-sdn ovs-ctsvc 1/1 Running 0 22m openshift-sdn sdn-2wrz2 1/1 Running 0 22m openshift-web-console webconsole-b5c499c98-ns6ww 1/1 Running 0 16m [root@all-openshift ~]# [root@all-openshift ~]#
デフォルトで入ってくる openshift-monitoring の grafana は便利そう！
