[Kubernetes] Kubernetes 설치 -1

Kubernetes(k8s) 설치

마스터(Master) 노드/워커(Worker) 노드 적용

호스트명 변경

$ hostnamectl set-hostname {호스트이름}

각 노드에 호스트 파일 등록

$ vim /etc/hosts

# kubernetes cluster
10.255.255.111 bk8sm1
10.255.255.121 bk8sn1
10.255.255.122 bk8sn2
10.255.255.123 bk8sn3

컨테이너 런타임 설치(도커 설치)

$ curl -fsSL https://get.docker.com -o get-docker.sh 
$ chmod +x get-docker.sh 
$ sh get-docker.sh 
$ usermod -aG docker $USER

$ systemctl enable docker 
$ systemctl start docker  

방화벽(firewalld) 비활성화

$ systemctl disable firewalld
$ systemctl stop firewalld

iptables 편집

$ cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

$ sysctl --system

SELinux 비활성화

$ sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config

스왑(swap) 메모리 비활성화(스왑 끄기)

###swap 확인
$ swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       4063228 8       -2

###swap off
$ swapoff -a

###fstab에서 제거
$ vim /etc/fstab
/dev/mapper/centos-root /                       xfs     defaults        0 0
UUID=23731053-4f9b-48f1-be76-c4dad7c47f22 /boot xfs     defaults        0 0
#/dev/mapper/centos-swap swap                   swap    defaults        0 0

Kubernetes Repo 추가

$ cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

kubelet, kubeadm, kubectl 설치

$ yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

$ systemctl enable kubelet 
$ systemctl start kubelet

도커 데몬 드라이버 교체

$ sudo cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

$ systemctl daemon-reload
$ systemctl restart docker

 

---

Kubernetes 클러스터를 배포하는 방법

마스터 노드 초기화

kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.0.6

###CNI는 calico사용 (10.244.0.0/16)
###apiserver-advertise-address 에 master 서버 아이피(10.255.255.111) 입력
root@bk8sm1:~$ kubeadm init --apiserver-advertise-address=10.255.255.111 --pod-network-cidr=10.244.0.0/16 
...
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.255.255.111:6443 --token negofj.s6eny4j9dswb6fa0 \
    --discovery-token-ca-cert-hash sha256:ddfbc876d7aec4aeaec7019ec7578ad12064d056272845c9c9b2dbaa17051652 

 

오류 발생 시 초기화

 kubeadm reset

$ kubeadm reset         
[reset] WARNING: Changes made to this host by 'kubeadm init' or 'kubeadm join' will be reverted.
[reset] Are you sure you want to proceed? [y/N]: y
[preflight] Running pre-flight checks
W1005 11:41:03.058868    2769 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]

The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the "iptables" command.

If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.

The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.

 

환경 변수 편집

###일반 사용자에 환경변수 등록
root@bk8sm1:~$ mkdir -p $HOME/.kube
root@bk8sm1:~$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config          
root@bk8sm1:~$ chown $(id -u):$(id -g) $HOME/.kube/config
root@bk8sm1:~$ kubectl config view                                     
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://10.255.255.111:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
    
###root에 환경변수 등록
root@bk8sm1:~$ export KUBECONFIG=/etc/kubernetes/admin.conf

 

POD 네트워드 설정(CNI 설정)

 캘리코(Calico) 사용

root@bk8sm1:~$ kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml

플란넬(Flannel) 사용

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

 

클러스터 상태 확인

root@bk8sm1:~$ kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
bk8sm1   Ready    master   7m26s   v1.19.2

root@bk8sm1:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                             READY   STATUS    RESTARTS   AGE
kube-system   coredns-f9fd979d6-hdz6v          1/1     Running   0          7m6s
kube-system   coredns-f9fd979d6-k8f29          1/1     Running   0          7m6s
kube-system   etcd-bk8sm1                      1/1     Running   0          7m17s
kube-system   kube-apiserver-bk8sm1            1/1     Running   0          7m17s
kube-system   kube-controller-manager-bk8sm1   1/1     Running   0          7m17s
kube-system   kube-flannel-ds-zs9r6            1/1     Running   0          61s
kube-system   kube-proxy-wbs9r                 1/1     Running   0          7m6s
kube-system   kube-scheduler-bk8sm1            1/1     Running   0          7m17s

 

---

 

워커 노드 클러스터 구성(워커 노드 join)

root@bk8sn1:~$ kubeadm join 10.255.255.111:6443 --token negofj.s6eny4j9dswb6fa0 \
    --discovery-token-ca-cert-hash sha256:ddfbc876d7aec4aeaec7019ec7578ad12064d056272845c9c9b2dbaa17051652

...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

 

---

 

마스터 노드에서 노드 정보 확인

root@bk8sm1:~$ kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
bk8sm1   Ready    master   11m     v1.19.2
bk8sn1   Ready    <none>   2m12s   v1.19.2
bk8sn2   Ready    <none>   2m12s   v1.19.2
bk8sn3   Ready    <none>   2m12s   v1.19.2

 

참고 URL : phoenixNAP

How to Install Kubernetes on CentOS 7 (Step by Step)