ubuntu 환경에서 elasticsearch와 kibana를 설치하는 방법

ubuntu 환경에서 elasticsearch와 kibana를 설치하는 방법(8.x)

테스트 환경

$ lsb_release -d
Description:	Ubuntu 22.04.1 LTS

Elasticsearch 설치하기

Elasticsearch의 공식 GPG key 다운로드하기

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK

apt-transport-https 패키지 설치

apt-get install -y apt-transport-https

Elasticsearch 패키지 저장소 추가하기

echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

$ echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
deb https://artifacts.elastic.co/packages/8.x/apt stable main

패키지 업데이트 후 Elasticsearch 설치하기

apt-get update && apt-get install -y elasticsearch

--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : MGNCiFpQU_9*-W5EtuLS

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------

Elasticsearch 설정 파일 확인

cat /etc/elasticsearch/elasticsearch.yml | egrep -v '^$|^#'

$ cat /etc/elasticsearch/elasticsearch.yml | egrep -v '^$|^#'
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["node1"]
http.host: 0.0.0.0

Elasticsearch 시작하기 및 Elasticsearch가 부팅 시 자동 시작하도록 설정하기

systemctl --now enable elasticsearch

Elasticsearch 설정 파일 편집(true -> false)

• xpack.security.enabled: false
• xpack.security.enrollment.enabled: false
• xpack.security.http.ssl.enabled: false
• xpack.security.transport.ssl.enabled: false

vim /etc/elasticsearch/elasticsearch.yml

...
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
...
# Enable security features
xpack.security.enabled: false

xpack.security.enrollment.enabled: false

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: false
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
...

Elasticsearch 재시작

systemctl restart elasticsearch

systemctl status elasticsearch

jq 패키지 설치

apt install -y jq

curl test

curl -s http://127.0.0.1:9200 -k | jq

$ curl -s http://127.0.0.1:9200 -k | jq
{
  "name": "ip-10-201-13-131",
  "cluster_name": "elasticsearch",
  "cluster_uuid": "IcZQ4wjTR1e9Hym9T8RdQg",
  "version": {
    "number": "8.6.2",
    "build_flavor": "default",
    "build_type": "deb",
    "build_hash": "2d58d0f136141f03239816a4e360a8d17b6d8f29",
    "build_date": "2023-02-13T09:35:20.314882762Z",
    "build_snapshot": false,
    "lucene_version": "9.4.2",
    "minimum_wire_compatibility_version": "7.17.0",
    "minimum_index_compatibility_version": "7.0.0"
  },
  "tagline": "You Know, for Search"
}

Kibana 설치하기

Kibana 패키지 저장소 추가하기

echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

패키지 업데이트 후 Kibana 설치하기

apt-get update && apt-get install -y kibana

Kibana 설정 파일 확인

cat /etc/kibana/kibana.yml | egrep -v '^$|^#'

$ cat /etc/kibana/kibana.yml | egrep -v '^$|^#'
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
pid.file: /run/kibana/kibana.pid

Kibana 시작하기 및 Kibana가 부팅 시 자동 시작하도록 설정하기

systemctl --now enable kibana

systemctl status kibana

Kibana 설정 파일 열기

• Kibana가 Elasticsearch에 연결하도록 설정하기

vim /etc/kibana/kibana.yml

...
# =================== System: Kibana Server ===================
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"
...
# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://localhost:9200"]
...
xpack.reporting.roles.enabled: false
...

Kibana 재시작

systemctl restart kibana

systemctl status kibana

Elasticsearch 보안 설정

Elasticsearch 설정 파일 편집

vim /etc/elasticsearch/elasticsearch.yml

Elasticsearch 설정 파일 편집

• cluster.name: my-application
• node.name: node-1
• bootstrap.memory_lock: true
• xpack.security.enabled: true 
• xpack.security.enrollment.enabled: true 
• xpack.security.http.ssl.enabled: true 
• xpack.security.transport.ssl.enabled: true 
• cluster.initial_master_nodes: ["node-1"]

cat /etc/elasticsearch/elasticsearch.yml | egrep -v '^$|^#'

cluster.name: my-application
node.name: node-1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
http.port: 9200
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["node-1"]
http.host: 0.0.0.0

Elasticsearch 재시작

systemctl restart elasticsearch

systemctl status elasticsearch

elastic 패스워드 재설정

• elastic / 5ZyOq72XkKciraI6PcXE

/usr/share/elasticsearch/bin/elasticsearch-reset-password --username elastic -i

$ /usr/share/elasticsearch/bin/elasticsearch-reset-password --username elastic -i
This tool will reset the password of the [elastic] user.
You will be prompted to enter the password.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]:
Re-enter password for [elastic]:
Password for the [elastic] user successfully reset.

curl test

export es_user=elastic

export es_password='5ZyOq72XkKciraI6PcXE'

curl -s -u ${es_user}:${es_password} https://127.0.0.1:9200 -k | jq

$ curl -s -u ${es_user}:${es_password} https://127.0.0.1:9200 -k | jq
{
  "name": "node-1",
  "cluster_name": "my-application",
  "cluster_uuid": "IcZQ4wjTR1e9Hym9T8RdQg",
  "version": {
    "number": "8.6.2",
    "build_flavor": "default",
    "build_type": "deb",
    "build_hash": "2d58d0f136141f03239816a4e360a8d17b6d8f29",
    "build_date": "2023-02-13T09:35:20.314882762Z",
    "build_snapshot": false,
    "lucene_version": "9.4.2",
    "minimum_wire_compatibility_version": "7.17.0",
    "minimum_index_compatibility_version": "7.0.0"
  },
  "tagline": "You Know, for Search"
}

Kibana Elasticsearch  연동

kibana_system 패스워드 재설정

• elasticsearch.username: "kibana_system"
• elasticsearch.password: "lAfqNm6q5ZsjnFvcqGN2"

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system -i

$ /usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system -i
This tool will reset the password of the [kibana_system] user.
You will be prompted to enter the password.
Please confirm that you would like to continue [y/N]y


Enter password for [kibana_system]: 
Re-enter password for [kibana_system]: 
Password for the [kibana_system] user successfully reset.

Kibana 설정 파일 편집

• Kibana가 Elasticsearch에 연결하도록 설정하기

vim /etc/kibana/kibana.yml

...
# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://localhost:9200"]

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
elasticsearch.username: "kibana_system"
elasticsearch.password: "lAfqNm6q5ZsjnFvcqGN2"
...
xpack.reporting.roles.enabled: false

elasticsearch.ssl.verificationMode: none

Kibana 설정 파일 확인

cat /etc/kibana/kibana.yml | egrep -v '^$|^#'

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["https://localhost:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "lAfqNm6q5ZsjnFvcqGN2"
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
pid.file: /run/kibana/kibana.pid
xpack.reporting.roles.enabled: false
elasticsearch.ssl.verificationMode: none

Kibana 재시작

systemctl restart kibana

systemctl status kibana

---

시스템 구성 파일(system configuration file) 수정

$ tail -f /var/log/syslog
Feb 18 23:05:45 es-01 systemd-entrypoint[32965]: bootstrap check failure [1] of [1]: memory locking requested for elasticsearch process but memory is not locked
Feb 18 23:05:45 es-01 systemd-entrypoint[32965]: ERROR: Elasticsearch did not exit normally - check the logs at /var/log/elasticsearch/my-application.log
Feb 18 23:05:47 es-01 systemd-entrypoint[32965]: ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.

• limits.conf 설정

vim /etc/security/limits.conf

*		soft	nofile	65536
*		hard	nofile	65536
*		soft	memlock	unlimited
*		hard	memlock	unlimited

• LimitMEMLOCK=infinity 추가

vim /usr/lib/systemd/system/elasticsearch.service

...
 46 # Specifies the maximum file size
 47 LimitFSIZE=infinity
 48
 49 LimitMEMLOCK=infinity
 50
 51 # Disable timeout logic and wait until process is stopped
 52 TimeoutStopSec=0
...

systemctl daemon-reload

systemctl restart elasticsearch

 

참고URL

- elasticsearch 설치 : https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html

- kibana 설치 : https://www.elastic.co/guide/en/kibana/current/deb.html