k8s インスコメモ再
cgroup にだいぶ悩まされてしまったが何とか完成したようだ
# curl -LO https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml # curl -LO https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml (変更前) 10.xx.xx.xx/16 ↓ (変更後) 10.0.0.0/16 <----- # kubectl apply -f kube-flannel.yml podsecuritypolicy.policy/psp.flannel.unprivileged created clusterrole.rbac.authorization.k8s.io/flannel created clusterrolebinding.rbac.authorization.k8s.io/flannel created serviceaccount/flannel created configmap/kube-flannel-cfg created daemonset.apps/kube-flannel-ds-amd64 created daemonset.apps/kube-flannel-ds-arm64 created daemonset.apps/kube-flannel-ds-arm created daemonset.apps/kube-flannel-ds-ppc64le created daemonset.apps/kube-flannel-ds-s390x created # docker info | grep -i cgroup WARNING: You're not using the default seccomp profile Cgroup Driver: systemd ' # cat /etc/sysconfig/kubelet KUBELET_EXTRA_ARGS=--cgroup-driver=systemd
直したところで再実行
＜全部＞ # kubeadm reset # systemctl daemon-reload # systemctl stop docker ; systemctl start docker # systemctl restart kubelet # systemctl status kubelet ● kubelet.service - kubelet: The Kubernetes Node Agent Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/kubelet.service.d mq10-kubeadm.conf Active: activating (auto-restart) (Result: exit-code) since 金 2020-02-07 02:50:02 JST; 7s ago Docs: https://kubernetes.io/docs/ Process: 96475 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255) Main PID: 96475 (code=exited, status=255) 2月 07 02:50:02 kube-master systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a 2月 07 02:50:02 kube-master systemd[1]: Unit kubelet.service entered failed state. 2月 07 02:50:02 kube-master systemd[1]: kubelet.service failed. #<--------------------------------------この段階ではエラーのままでよし ＜マスター＞ # kubeadm init ： kubeadm join 192.168.253.132:6443 --token u1q8mn.gigcbfjnzejnmezd \ --discovery-token-ca-cert-hash sha256:2a74b87b1639a32548545a975d5113eea9bcb6cddc3b9f2b64c45a03cd23e465 ＜ノード＞ # kubeadm join 192.168.253.132:6443 --token u1q8mn.gigcbfjnzejnmezd \ > --discovery-token-ca-cert-hash sha256:2a74b87b1639a32548545a975d5113eea9bcb6cddc3b9f2b64c45a03cd23e465 W0207 02:58:20.642775 31598 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set. [preflight] Running pre-flight checks [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
ノードとPod の状態確認 OK
# kubectl get nodes NAME STATUS ROLES AGE VERSION kube-master Ready master 71s v1.17.2 kube-node1 Ready <none> 15s v1.17.2 kube-node2 Ready <none> 37s v1.17.2 # kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-6955765f44-7tm6k 1/1 Running 0 4m8s kube-system coredns-6955765f44-hqs9m 1/1 Running 0 4m8s kube-system etcd-kube-master 1/1 Running 0 4m12s kube-system kube-apiserver-kube-master 1/1 Running 0 4m12s kube-system kube-controller-manager-kube-master 1/1 Running 0 4m12s kube-system kube-proxy-8tjjl 1/1 Running 0 4m8s kube-system kube-proxy-jdfbj 1/1 Running 0 3m23s kube-system kube-proxy-xkwgc 1/1 Running 0 3m45s kube-system kube-scheduler-kube-master 1/1 Running 0 4m12s
ノードにラベルを張ります。
# kubectl label node kube-node1 node-role.kubernetes.io/node= node/kube-node1 labeled # kubectl label node kube-node2 node-role.kubernetes.io/node= node/kube-node2 labeled # kubectl get nodes NAME STATUS ROLES AGE VERSION kube-master Ready master 18h v1.17.2 kube-node1 Ready node 18h v1.17.2 kube-node2 Ready node 18h v1.17.2
# kubectl label nodes kube-node1 type=worker node/kube-node1 labeled # kubectl label nodes kube-node2 type=worker node/kube-node2 labeled # kubectl label nodes kube-master type=master node/kube-master labeled # kubectl get nodes -L type NAME STATUS ROLES AGE VERSION TYPE kube-master Ready master 19h v1.17.2 master kube-node1 Ready node 19h v1.17.2 worker kube-node2 Ready node 19h v1.17.2 worker
と、思った矢先、フラネルがCrashLoopBack
# kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system coredns-6955765f44-7tm6k 0/1 Completed 0 18h <none> kube-master <none> <none> kube-system coredns-6955765f44-hqs9m 0/1 Completed 0 18h <none> kube-master <none> <none> kube-system etcd-kube-master 1/1 Running 1 18h 192.168.253.132 kube-master <none> <none> kube-system kube-apiserver-kube-master 1/1 Running 4 18h 192.168.253.132 kube-master <none> <none> kube-system kube-controller-manager-kube-master 1/1 Running 2 18h 192.168.253.132 kube-master <none> <none> kube-system kube-flannel-ds-amd64-5rhdr 0/1 CrashLoopBackOff 4 2m35s 192.168.253.132 kube-master <none> <none> kube-system kube-flannel-ds-amd64-nbv54 0/1 CrashLoopBackOff 4 2m35s 192.168.253.134 kube-node2 <none> <none> kube-system kube-flannel-ds-amd64-x2d6l 0/1 CrashLoopBackOff 4 2m35s 192.168.253.133 kube-node1 <none> <none> kube-system kube-proxy-8tjjl 1/1 Running 1 18h 192.168.253.132 kube-master <none> <none> kube-system kube-proxy-jdfbj 1/1 Running 0 18h 192.168.253.133 kube-node1 <none> <none> kube-system kube-proxy-xkwgc 1/1 Running 0 18h 192.168.253.134 kube-node2 <none> <none> kube-system kube-scheduler-kube-master 1/1 Running 2 18h 192.168.253.132 kube-master <none> <none>
四苦八苦した後
# kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system coredns-6955765f44-kxqw9 1/1 Running 0 17m 10.244.0.2 kube-master <none> <none> kube-system coredns-6955765f44-tnlb4 1/1 Running 0 17m 10.244.0.3 kube-master <none> <none> kube-system etcd-kube-master 1/1 Running 0 18m 192.168.253.132 kube-master <none> <none> kube-system kube-apiserver-kube-master 1/1 Running 0 18m 192.168.253.132 kube-master <none> <none> kube-system kube-controller-manager-kube-master 1/1 Running 0 18m 192.168.253.132 kube-master <none> <none> kube-system kube-flannel-ds-amd64-wb57n 1/1 Running 0 7m50s 192.168.253.134 kube-node2 <none> <none> kube-system kube-flannel-ds-amd64-z677j 1/1 Running 0 7m50s 192.168.253.132 kube-master <none> <none> kube-system kube-flannel-ds-amd64-ztx7c 1/1 Running 0 7m50s 192.168.253.133 kube-node1 <none> <none> kube-system kube-proxy-dftkx 1/1 Running 0 10m 192.168.253.134 kube-node2 <none> <none> kube-system kube-proxy-gj52j 1/1 Running 0 11m 192.168.253.133 kube-node1 <none> <none> kube-system kube-proxy-lbxfx 1/1 Running 0 17m 192.168.253.132 kube-master <none> <none> kube-system kube-scheduler-kube-master 1/1 Running 0 18m 192.168.253.132 kube-master <none> <none>
原因
時刻同期が３台ズレてました。
# ntpdate ntp.nict.jp 8 Feb 02:41:18 ntpdate[40560]: adjust time server 133.243.238.164 offset 0.000288 sec
