[리눅스] 엘라스틱서치 스냅샷 및 복원(Elasticsearch Snapshot and Restore)

엘라스틱서치 스냅샷  및 복원(엘라스틱서치 데이터 마이그레이션)

도커 컨테이너 설정

docker-compose.yml 편집

 backup 디렉토리 경로 설정

    environment:
      - path.repo=/opt/elasticsearch/backup

    volumes:
      - ./es_backup:/opt/elasticsearch/backup

vim docker-compose.yml

version: "3.2"

services:
  elasticsearch:
    build:
      context: docker/elasticsearch/
      args:
        ELASTIC_VERSION: $ELASTIC_VERSION
    image: elasticsearch
    container_name: elasticsearch
    hostname: elasticsearch
    environment:
      - "ES_JAVA_OPTS=-Xms4g -Xmx4g"
      - ELASTIC_PASSWORD=elastic
      - discovery.type=single-node
      - path.repo=/opt/elasticsearch/backup
    mem_limit: 4g
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./docker/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - ./es_data:/usr/share/elasticsearch/data
      - ./es_backup:/opt/elasticsearch/backup
    ports:
      - '9200:9200'
      - '9300:9300'
    healthcheck:
      test: ["CMD", "curl","-s" ,"-f", "-u", "elastic:elastic", "http://localhost:9200/_cat/health"]

백업파일이 저장될 디렉토리(es_backup) 생성 및 권한 설정

mkdir es_backup

chown -R 1000.1000 ./es_backup

elasticsearch 재기동

docker-compose restart

elasticsearch 스냅샷(내보내기)

저장소 등록(repository)

curl -X PUT localhost:9200/_snapshot/my_backup/

curl -X PUT localhost:9200/_snapshot/my_backup/ -H "Content-Type: application/json" -uelastic:elastic --data-binary @- <<DATA 
{ 
"type": "fs",
"settings": {"location": "/opt/elasticsearch/backup",
"compress": true
}
}
DATA

{"acknowledged":true}

리포지토리 확인

curl -GET localhost:9200/_snapshot/my_backup/?pretty

curl -GET localhost:9200/_snapshot/my_backup/?pretty -uelastic:elastic

{
  "my_backup" : {
    "type" : "fs",
    "settings" : {
      "compress" : "true",
      "location" : "/opt/elasticsearch/backup"
    }
  }
}

인덱스 목록 확인

curl -s -XGET localhost:9200/_cat/indices?pretty -uelastic:elastic

green  open .monitoring-kibana-7-2021.08.21 22NK8ILMSFulhN0-n04eTQ 1 0    17280       0    2.7mb    2.7mb
green  open .monitoring-kibana-7-2021.08.22 4o8OC3leSM-3mzv4bbmVbw 1 0    17278       0    2.7mb    2.7mb
green  open .monitoring-kibana-7-2021.08.23 SaBERyxgTpaYNc4dEULBsA 1 0     3148       0  602.9kb  602.9kb
green  open .monitoring-kibana-7-2021.08.24 4PokxPCQQ_ypi_3_UGG1DA 1 0      242       0  136.9kb  136.9kb
green  open .monitoring-kibana-7-2021.08.20 vcWvjzaJSsyCg4gibgKj6g 1 0    17278       0    2.7mb    2.7mb
green  open .monitoring-kibana-7-2021.08.18 7n8-zFt4TnCfmTIrdzhYYg 1 0    17278       0    2.8mb    2.8mb
green  open .monitoring-kibana-7-2021.08.19 0u198weLQpq3ifIybKCWfA 1 0    17278       0    2.8mb    2.8mb

스냅샷 생성

curl -X PUT localhost:9200/_snapshot/my_backup/snap?wait_for_completion=true

$ curl -X PUT localhost:9200/_snapshot/my_backup/snap?wait_for_completion=true -H "Content-Type: application/json" -uelastic:elastic --data-binary @- <<DATA
{
"indices":"efk-access-www-20210819",
"ignore_unavailable":true,
"include_global_state":true
}
DATA

{"snapshot":{"snapshot":"snap","uuid":"LbHRZD5RR7eYWzBpMcfgSA","version_id":7100299,"version":"7.10.2","indices":["efk-access-www-20210819"],"data_streams":[],"include_global_state":true,"state":"SUCCESS","start_time":"2021-08-24T03:32:29.981Z","start_time_in_millis":1629775949981,"end_time":"2021-08-24T03:32:32.983Z","end_time_in_millis":1629775952983,"duration_in_millis":3002,"failures":[],"shards":{"total":1,"failed":0,"successful":1}}}

elasticsearch 리스토어(가져오기)

저장소에 저장된 인덱스 목록 확인

curl -XGET localhost:9200/_snapshot/my_backup/_all?pretty

curl -XGET localhost:9200/_snapshot/my_backup/_all?pretty -uelastic:elastic

{
  "snapshots" : [
    {
      "snapshot" : "snap",
      "uuid" : "LbHRZD5RR7eYWzBpMcfgSA",
      "version_id" : 7100299,
      "version" : "7.10.2",
      "indices" : [
        "efk-access-www-20210819"
      ],
      "data_streams" : [ ],
      "include_global_state" : true,
      "state" : "SUCCESS",
      "start_time" : "2021-08-24T03:32:29.981Z",
      "start_time_in_millis" : 1629775949981,
      "end_time" : "2021-08-24T03:32:32.983Z",
      "end_time_in_millis" : 1629775952983,
      "duration_in_millis" : 3002,
      "failures" : [ ],
      "shards" : {
        "total" : 1,
        "failed" : 0,
        "successful" : 1
      }
    }
  ]
}

snap(스냅샷) 상태 확인

curl -XGET localhost:9200/_snapshot/my_backup/snap/_status?pretty

curl -XGET localhost:9200/_snapshot/my_backup/snap/_status?pretty -uelastic:elastic

{
  "snapshots" : [
    {
      "snapshot" : "snap",
      "repository" : "my_backup",
      "uuid" : "LbHRZD5RR7eYWzBpMcfgSA",
      "state" : "SUCCESS",
      "include_global_state" : true,
      "shards_stats" : {
        "initializing" : 0,
        "started" : 0,
        "finalizing" : 0,
        "done" : 1,
        "failed" : 0,
        "total" : 1
      },
      "stats" : {
        "incremental" : {
          "file_count" : 57,
          "size_in_bytes" : 91086278
        },
        "total" : {
          "file_count" : 57,
          "size_in_bytes" : 91086278
        },
        "start_time_in_millis" : 1629775949981,
        "time_in_millis" : 3002
      },
      "indices" : {
        "efk-access-www-20210819" : {
          "shards_stats" : {
            "initializing" : 0,
            "started" : 0,
            "finalizing" : 0,
            "done" : 1,
            "failed" : 0,
            "total" : 1
          },
          "stats" : {
            "incremental" : {
              "file_count" : 57,
              "size_in_bytes" : 91086278
            },
            "total" : {
              "file_count" : 57,
              "size_in_bytes" : 91086278
            },
            "start_time_in_millis" : 1629775949981,
            "time_in_millis" : 2802
          },
          "shards" : {
            "0" : {
              "stage" : "DONE",
              "stats" : {
                "incremental" : {
                  "file_count" : 57,
                  "size_in_bytes" : 91086278
                },
                "total" : {
                  "file_count" : 57,
                  "size_in_bytes" : 91086278
                },
                "start_time_in_millis" : 1629775949981,
                "time_in_millis" : 2802
              }
            }
          }
        }
      }
    }
  ]
}

리스토어(복원)

curl -XPOST localhost:9200/_snapshot/my_backup/snap/_restore

curl -XPOST localhost:9200/_snapshot/my_backup/snap/_restore -H "Content-Type: application/json" -uelastic:elastic --data-binary @- <<DATA
{
"indices":"efk-access-www-20210819",
"index_settings":{"index.number_of_replicas":0},
"ignore_index_settings":["index.refresh_interval"]
}
DATA

{"accepted":true}

스냅샷 제거

curl -XDELETE localhost:9200/_snapshot/my_backup/snap

curl -XDELETE localhost:9200/_snapshot/my_backup/snap -uelastic:elastic

{"acknowledged":true}

 

웹 콘솔 : Snapshot and Restore

Elastic > Management > Stack Management

 

Elastic > Management > Stack Management > Snapshot and Restore

 

저장소 등록 후 스냅샷 생성