docker efk stack(elasticsearch cluster)

docker efk stack(elasticsearch cluster)

EFK 간략한 소개

이미지URL-https://miro.medium.com/max/1276/1*JitG2uTXpuItJDwX_Bf66g.jpeg

EFK Stack / Dcoker and OpenSSL

이미지URL-https://www.youtube.com/watch?v=31wJJPZgWrQ&ab_channel=ShashankSharmaDevOps

[node1, node2, node3]

github : https://github.com/anti1346/docker-efkstack

git clone https://github.com/anti1346/docker-efkstack.git

alias 등록

echo "alias efk='cd /docker-container/docker-efkstack/node1'" >> ~/.bashrc
echo "alias efk='cd /docker-container/docker-efkstack/node2'" >> ~/.bashrc
echo "alias efk='cd /docker-container/docker-efkstack/node3'" >> ~/.bashrc

디렉터리 이동

efk

elasticsearch 데이터 디렉토리 생성

mkdir esdata; chown -R 1000.1000 esdata

kibana 데이터 디렉토리 생성

mkdir -p kibana/data; chown -R 1000.1000 kibana/data

fluentd 데이터 디렉토리 생성

mkdir -p fluentd/log; chown -R 999.999 fluentd/log

[node1]

docker-compose up -d

docker-compose ps

$ docker-compose ps
 Name                Command                  State                                              Ports
----------------------------------------------------------------------------------------------------------------------------------------------
es01      /bin/tini -- /usr/local/bi ...   Up (healthy)   0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp
fluentd   tini -- /bin/entrypoint.sh ...   Up             0.0.0.0:24224->24224/tcp, 0.0.0.0:24224->24224/udp, 5140/tcp, 0.0.0.0:8888->8888/tcp
kibana    /bin/tini -- /usr/local/bi ...   Up (healthy)   0.0.0.0:5601->5601/tcp
setup     /bin/tini -- /usr/local/bi ...   Exit 0

인증서 파일 전송(cert rsync)

rsync -avz -e ssh /docker-container/docker-efkstack/certs root@es02:/docker-container/docker-efkstack/

rsync -avz -e ssh /docker-container/docker-efkstack/certs root@es03:/docker-container/docker-efkstack/

[node2, node3]

docker-compose up -d

docker-compose ps

$ docker-compose ps
Name              Command                  State                           Ports
-----------------------------------------------------------------------------------------------------
es02   /bin/tini -- /usr/local/bi ...   Up (healthy)   0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp

728x90

elasticsearch cluster health

curl -s -XGET "https://localhost:9200/_cluster/health?pretty" -u "elastic:elastic1!" --cacert ../certs/ca/ca.crt

$ curl -s -XGET "https://localhost:9200/_cluster/health?pretty" -u "elastic:elastic1!" --cacert ../certs/ca/ca.crt
{
  "cluster_name" : "docker-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 20,
  "active_shards" : 40,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

elasticsearch node

curl -s -XGET "https://localhost:9200/_cat/nodes?v&pretty" -u "elastic:elastic1!" --cacert ../certs/ca/ca.crt

$ curl -s -XGET "https://localhost:9200/_cat/nodes?v&pretty" -u "elastic:elastic1!" --cacert ../certs/ca/ca.crt
ip              heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
3.3.3.3           46         100   3    0.16    0.15     0.16 cdfhilmrstw -      es03
1.1.1.1           46         100   2    0.09    0.16     0.14 cdfhilmrstw *      es01
2.2.2.2           62         100   0    0.02    0.04     0.06 cdfhilmrstw -      es02

fluent 로그 발생

curl -XPOST -d 'json={"json":"message"}' http://localhost:8888/debug.test

 

참고URL

- https://chowdera.com/2021/01/20210103093634009S.html

- https://medium.com/@jainishshah17/efk-stack-on-kubernetes-af8a95d7c3a0

- https://www.middlewareinventory.com/blog/fluentd-tomcat-elastic-search-example-efk/

- https://www.youtube.com/watch?v=31wJJPZgWrQ&ab_channel=ShashankSharmaDevOps