본문 바로가기
리눅스

우분투에서 Elasticsearch 8.x 설치 및 클러스터 구성 방법

변군이글루 2024. 2. 26. 13:49
반응형

우분투에서 Elasticsearch 8.x 설치 및 클러스터 구성 방법

운영 체제 환경 설정

호스트 파일에 노드 추가(/etc/hosts)

cat <<EOF >> /etc/hosts
### elasticsearch
192.168.56.71  node1
192.168.56.72  node2
192.168.56.73  node3
EOF

Elasticsearch 사용자의 ulimit을 설정

cat << EOF >> /etc/security/limits.conf

### elasticsearch
elasticsearch       hard    nofile   65535
elasticsearch       soft    nofile   65536
elasticsearch       hard    nproc    65536
elasticsearch       soft    nproc    65536
elasticsearch       hard    memlock  unlimited
elasticsearch       soft    memlock  unlimited
EOF

시스템의 커널 파라미터를 변경

  • 가상 메모리 지도의 최대 항목 수를 설정
echo "vm.max_map_count = 262144" | sudo tee -a /etc/sysctl.conf
  • 시스템이 스왑 파일을 사용하는 정책을 설정
echo "vm.swappiness = 1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

스왑 비활성화(disable swapping)

sudo swapoff -a
sudo sed -i '/swap/ s/^/#/' /etc/fstab

Elasticsearch 설치

sudo apt-get install -y apt-transport-https

Elasticsearch GPG 키 추가

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Elasticsearch 저장소 추가

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

APT 저장소 업데이트

sudo apt-get update

Elasticsearch 패키지 설치

sudo apt-get install -y elasticsearch

systemd 데몬 리로드

sudo systemctl daemon-reload

Elasticsearch 서비스 시작 및 활성화

sudo systemctl --now enable elasticsearch.service

Elasticsearch cert 디렉토리 백업

cd /etc/elasticsearch
tar cfz elasticsearch_certs-${HOSTNAME}-$(date '+%Y%m%d_%H%M').tar.gz certs

Elasticsearch에서 TLS/SSL을 사용하기 위해 인증서 생성

  • 자체 서명된 루트 인증 기관(Certificate Authority, CA)를 생성
/usr/share/elasticsearch/bin/elasticsearch-certutil ca \
--out /etc/elasticsearch/certs/elastic-stack-ca.p12 \
--days 3650
  • 서버 및 클라이언트용 TLS/SSL 인증서를 생성
/usr/share/elasticsearch/bin/elasticsearch-certutil cert \
--ca /etc/elasticsearch/certs/elastic-stack-ca.p12 \
--out /etc/elasticsearch/certs/elastic-certificates.p12 \
--days 3650
  • 소유자 설정
chown root:elasticsearch /etc/elasticsearch/certs/elastic-certificates.p12
  • 권한 설정
chmod g+wr /etc/elasticsearch/certs/elastic-certificates.p12

node2, node3 인증서 파일 전송(보내기)

rsync -avz /etc/elasticsearch/certs root@192.168.56.72:/etc/elasticsearch/.
rsync -avz /etc/elasticsearch/certs root@192.168.56.73:/etc/elasticsearch/.
더보기

sshd 설정

---

sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
sudo systemctl restart sshd

---

keystore 설정

  • elasticsearch keystore 파일 생성
/usr/share/elasticsearch/bin/elasticsearch-keystore create
  • elasticsearch keystore 추가 Elasticsearch 노드 간의 TLS/SSL 보안 통신을 설정
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
  • HTTP 통신을 위한 TLS/SSL 보안 통신을 설정
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
/usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password
  • elasticsearch keystore 목록
/usr/share/elasticsearch/bin/elasticsearch-keystore list
  • http.p12의 비밀번호 확인
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password
  • transport.p12의 비밀번호 확인
/usr/share/elasticsearch/bin/elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password

Elasticsearch 서비스의 systemd 유닛 파일 설정

  • elasticsearch.service 파일 편집
vim /usr/lib/systemd/system/elasticsearch.service
[Service]
LimitMEMLOCK=infinity
  • systemd 데몬 리로드
sudo systemctl daemon-reload
  • Elasticsearch 서비스 재시작
sudo systemctl restart elasticsearch.service

Elasticsearch 환경 설정

  • Elasticsearch 환경 설정 파일 편집(/etc/elasticsearch/elasticsearch.yml)
vim /etc/elasticsearch/elasticsearch.yml
cluster.name: my-cluster

node.name: node1

path.data: /var/lib/elasticsearch

path.logs: /var/log/elasticsearch

bootstrap.memory_lock: true

network.host: 192.168.56.71

http.port: 9200

discovery.seed_hosts: ["node1", "node2", "node3"]

cluster.initial_master_nodes: ["node1", "node2", "node3"]

xpack.security.enabled: true

xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

http.host: 0.0.0.0
  • Elasticsearch 서비스 시작/재시작/중지
sudo systemctl restart elasticsearch.service
sudo systemctl stop elasticsearch.service
  • Elasticsearch 서비스 상태 확인
sudo systemctl status elasticsearch

elastic 사용자 패스워드 재설정

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
$ /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
This tool will reset the password of the [elastic] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y


Password for the [elastic] user successfully reset.
New value: 768NrzAwNO2vFz-VcRGI

built-in superuser(elastic)의 비밀번호를 셸에 환경 변수로 저장

export ELASTIC_PASSWORD="768NrzAwNO2vFz-VcRGI"
echo $ELASTIC_PASSWORD

elasticsearch data 초기화

더보기

---

sudo systemctl stop elasticsearch.service
rm -rf /var/lib/elasticsearch
mkdir -p /var/lib/elasticsearch
chown elasticsearch.elasticsearch /var/lib/elasticsearch
sudo systemctl start elasticsearch.service

---

Elasticsearch API 사용

Elasticsearch API를 통해 클러스터 상태를 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200
$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200
{
  "name" : "node1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Jn0qAEnrTMyxK1G0XDvW1Q",
  "version" : {
    "number" : "8.12.1",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "6185ba65d27469afabc9bc951cded6c17c21e3f3",
    "build_date" : "2024-02-01T13:07:13.727175297Z",
    "build_snapshot" : false,
    "lucene_version" : "9.9.2",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Cluster Health 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/health?pretty
$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "my-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

클러스터 통계 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/stats?pretty

노드 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/nodes?v
$ curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/nodes?v
ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
192.168.56.72            8          97   7    0.46    0.52     0.41 cdfhilmrstw -      node2
192.168.56.73           26          89   9    0.77    0.58     0.39 cdfhilmrstw *      node3
192.168.56.71           13          75  10    0.49    0.58     0.45 cdfhilmrstw -      node1

인덱스 정보 확인

curl -k -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/indices?pretty
728x90

Kibana 설치

Kibana 패키지 설치

sudo apt-get install -y kibana

Kibana에서 사용하는 암호화 키를 생성

  • Kibana의 설정 파일인 kibana.yml에 키를 추가
/usr/share/kibana/bin/kibana-encryption-keys generate
$ /usr/share/kibana/bin/kibana-encryption-keys generate
## Kibana Encryption Key Generation Utility

The 'generate' command guides you through the process of setting encryption keys for:

xpack.encryptedSavedObjects.encryptionKey
    Used to encrypt stored objects such as dashboards and visualizations
    https://www.elastic.co/guide/en/kibana/current/xpack-security-secure-saved-objects.html#xpack-security-secure-saved-objects

xpack.reporting.encryptionKey
    Used to encrypt saved reports
    https://www.elastic.co/guide/en/kibana/current/reporting-settings-kb.html#general-reporting-settings

xpack.security.encryptionKey
    Used to encrypt session information
    https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#security-session-and-cookie-settings


Already defined settings are ignored and can be regenerated using the --force flag.  Check the documentation links for instructions on how to rotate encryption keys.
Definitions should be set in the kibana.yml used configure Kibana.

Settings:
xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3
cat <<EOF >> /etc/kibana/kibana.yml

## Kibana Encryption Key Generation Utility
xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3
EOF

kibana_system의 비밀번호를 재설정

/usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system
$ /usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system
This tool will reset the password of the [kibana_system] user.
You will be prompted to enter the password.
Please confirm that you would like to continue [y/N]y


Enter password for [kibana_system]:
Re-enter password for [kibana_system]:
Password for the [kibana_system] user successfully reset.

Kibana에서 사용할 인증서를 생성

mkdir -p /etc/kibana/certs
cd /etc/kibana/certs
cp /etc/elasticsearch/certs/elastic-stack-ca.p12 /etc/kibana/certs/.
cp /etc/elasticsearch/certs/elastic-certificates.p12 /etc/kibana/certs/.
openssl pkcs12 -in elastic-certificates.p12 -cacerts -nokeys -out CA.pem
openssl pkcs12 -in elastic-certificates.p12 -nocerts -nodes -out client.key
openssl pkcs12 -in elastic-certificates.p12 -clcerts -nokeys -out client.crt
chown -R kibana.kibana /etc/kibana/certs

Kibana 설정

  • Kibana 환경 설정 파일 편집(/etc/kibana/kibana.yml)
vim /etc/kibana/kibana.yml
server.port: 5601

server.host: "192.168.10.73"

elasticsearch.hosts: ["https://node1:9200","https://node2:9200","https://node3:9200"]

elasticsearch.username: "kibana_system"
elasticsearch.password: "kibana_system_password"

elasticsearch.ssl.certificate: /etc/kibana/certs/client.crt
elasticsearch.ssl.key: /etc/kibana/certs/client.key

elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/CA.pem" ]

elasticsearch.ssl.verificationMode: certificate

logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file

pid.file: /run/kibana/kibana.pid

xpack.encryptedSavedObjects.encryptionKey: 909b3e89488a1239a959b6ea5f6cf1da
xpack.reporting.encryptionKey: 1986a90ef4b9ed39df7b64728b615e1b
xpack.security.encryptionKey: a85ad1850c42c9409477a32b6d9fcce3

systemd 데몬 리로드

sudo systemctl daemon-reload

Kibana 서비스 시작 및 활성화

sudo systemctl --now enable kibana.service

Kibana 서비스 시작/재시작/중지

sudo systemctl restart kibana
sudo systemctl status kibana

Kibana 접속(웹 브라우저)

http://localhost:5601

Elasticsearch_Kibana

 

참고URL

- elasticsearch guide : Install Elasticsearch with Debian Package

- kibana guide : Install Kibana with Debian package

 

728x90
반응형
저작자표시 비영리

'리눅스' 카테고리의 다른 글

html2text 명령어  (0) 2024.02.28
우분투에 CBand 모듈을 설치하고 설정하는 방법  (0) 2024.02.26
리눅스에서 ulimit을 설정하는 방법  (0) 2024.02.26
우분투에서 Elasticsearch를 설치하고 Kibana를 사용하여 클러스터 상태를 확인하는 방법  (0) 2024.02.23
kubectl 명령어의 자동 완성을 활성화하는 방법  (0) 2024.02.17

'리눅스' Related Articles

티스토리툴바