우분투에서 TLS를 사용하는 etcd 클러스터를 설정하는 방법

우분투에서 TLS를 사용하는 etcd 클러스터를 설정하는 방법

테스트 환경

역할	호스트 이름	아이피 주소	비고
etcd 서버	node111	192.168.10.111
etcd 서버	node112	192.168.10.112
etcd 서버	node113	192.168.10.113

 

APT 패키지 리스트 업데이트

sudo apt-get update

etcd 설치

sudo apt-get install -y etcd

TLS/SSL 인증서 생성

curl -fsSL https://raw.githubusercontent.com/anti1346/zz/main/ubuntu/sslcert_generator.sh -o sslcert_generator.sh

vim sslcert_generator.sh

# 설정 값 정의
export NAME1="node111"
export ADDRESS1="192.168.10.111"
export NAME2="node112"
export ADDRESS2="192.168.10.112"
export NAME3="node113"
export ADDRESS3="192.168.10.113"

chmod +x sslcert_generator.sh

bash sslcert_generator.sh

tar cfz ssl.tar.gz ssl

인증서 파일 배포

생성된 인증서 파일을 각 노드에 복사합니다.

scp ssl.tar.gz ubuntu@127.0.0.1:~

scp ssl.tar.gz ubuntu@192.168.10.112:~

scp ssl.tar.gz ubuntu@192.168.10.113:~

각 노드에서 적절한 위치로 이동시키고 권한을 설정합니다.

sudo mkdir -p /etc/etcd /var/lib/etcd

tar xfz /home/ubuntu/ssl.tar.gz -C /etc/etcd

sudo chown -R etcd:etcd /etc/etcd

sudo chown -R etcd:etcd /var/lib/etcd

sudo chmod -R 700 /var/lib/etcd

etcd 서비스 구성

• 설치가 완료되면 etcd 서비스를 구성해야 합니다.
• 기본적으로 /etc/default/etcd 파일을 편집하여 환경 변수를 설정할 수 있습니다.

• node111

cat <<EOF | sudo tee /etc/default/etcd
ETCD_NAME="node111"
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.10.111:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.10.111:2379,https://127.0.0.1:2379"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.10.111:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.10.111:2380"
ETCD_INITIAL_CLUSTER="node111=https://192.168.10.111:2380,node112=https://192.168.10.112:2380,node113=https://192.168.10.113:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_CERT_FILE="/etc/etcd/ssl/node.crt"
ETCD_KEY_FILE="/etc/etcd/ssl/node.key"
ETCD_CLIENT_CERT_AUTH="true"

ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/peer.crt"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/peer.key"
ETCD_PEER_CLIENT_CERT_AUTH="true"
EOF

• node112

cat <<EOF | sudo tee /etc/default/etcd
ETCD_NAME="node112"
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.10.112:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.10.112:2379,https://127.0.0.1:2379"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.10.112:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.10.112:2380"
ETCD_INITIAL_CLUSTER="node111=https://192.168.10.111:2380,node112=https://192.168.10.112:2380,node113=https://192.168.10.113:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_CERT_FILE="/etc/etcd/ssl/node.crt"
ETCD_KEY_FILE="/etc/etcd/ssl/node.key"
ETCD_CLIENT_CERT_AUTH="true"

ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/peer.crt"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/peer.key"
ETCD_PEER_CLIENT_CERT_AUTH="true"
EOF

• node113

cat <<EOF | sudo tee /etc/default/etcd
ETCD_NAME="node113"
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.10.113:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.10.113:2379,https://127.0.0.1:2379"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.10.113:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.10.113:2380"
ETCD_INITIAL_CLUSTER="node111=https://192.168.10.111:2380,node112=https://192.168.10.112:2380,node113=https://192.168.10.113:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_CERT_FILE="/etc/etcd/ssl/node.crt"
ETCD_KEY_FILE="/etc/etcd/ssl/node.key"
ETCD_CLIENT_CERT_AUTH="true"

ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.crt"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/peer.crt"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/peer.key"
ETCD_PEER_CLIENT_CERT_AUTH="true"
EOF

728x90

환경 변수 설정

• etcd 3.x 버전부터는 API v3가 기본값이지만 명시적으로 설정해보는 것이 좋습니다.

export ETCDCTL_API=3

etcd 서비스 시작 및 활성화

• etcd 서비스를 시작하고 시스템 부팅 시 자동으로 시작되도록 설정합니다.

sudo systemctl daemon-reload

sudo systemctl start etcd

sudo systemctl enable etcd

etcd 서비스 재시작

sudo systemctl restart etcd

etcd 상태 확인

• etcd 서비스가 정상적으로 실행 중인지 확인합니다.

sudo systemctl status etcd

지정된 엔드포인의 health 상태 확인

etcdctl endpoint health \
--cert=/etc/etcd/ssl/peer.crt \
--key=/etc/etcd/ssl/peer.key \
--cacert=/etc/etcd/ssl/ca.crt \
--endpoints=https://$(hostname -I | awk '{print $1}'):2379

https://192.168.10.111:2379 is healthy: successfully committed proposal: took = 13.775173ms

클러스터 전체의 health 상태 확인

etcdctl endpoint health --cluster \
--cert=/etc/etcd/ssl/peer.crt \
--key=/etc/etcd/ssl/peer.key \
--cacert=/etc/etcd/ssl/ca.crt \
--endpoints=https://$(hostname -I | awk '{print $1}'):2379

https://192.168.10.113:2379 is healthy: successfully committed proposal: took = 23.666344ms
https://192.168.10.112:2379 is healthy: successfully committed proposal: took = 23.812737ms
https://192.168.10.111:2379 is healthy: successfully committed proposal: took = 23.470701ms

클러스터 전체의 상태 확인

etcdctl endpoint status --cluster -w table \
--cert=/etc/etcd/ssl/peer.crt \
--key=/etc/etcd/ssl/peer.key \
--cacert=/etc/etcd/ssl/ca.crt \
--endpoints=https://$(hostname -I | awk '{print $1}'):2379

cluster_status

etcd 클라이언트의 모든 멤버를 목록 확인

etcdctl member list \
--cert=/etc/etcd/ssl/peer.crt \
--key=/etc/etcd/ssl/peer.key \
--cacert=/etc/etcd/ssl/ca.crt \
--endpoints=https://$(hostname -I | awk '{print $1}'):2379

641086671da4b1a4, started, node113, https://192.168.10.113:2380, https://192.168.10.113:2379
a36ab5eacdd05d40, started, node111, https://192.168.10.111:2380, https://192.168.10.111:2379
e638a48b54bf415c, started, node112, https://192.168.10.112:2380, https://192.168.10.112:2379

테이터 쓰기 및 읽기

테이터 쓰기

etcdctl put foo bar \
--cert=/etc/etcd/ssl/peer.crt \
--key=/etc/etcd/ssl/peer.key \
--cacert=/etc/etcd/ssl/ca.crt \
--endpoints=https://$(hostname -I | awk '{print $1}'):2379

OK

테이터 읽기

etcdctl get foo \
--cert=/etc/etcd/ssl/peer.crt \
--key=/etc/etcd/ssl/peer.key \
--cacert=/etc/etcd/ssl/ca.crt \
--endpoints=https://$(hostname -I | awk '{print $1}'):2379

foo
bar

 

참고URL

- zhaowenyu : etcd-doc