[aws] Amazon Linux 2(AMZN2)에서 fluent-bit을 설치하는 방법

Amazon Linux 2(AMZN2)에서 fluent-bit을 설치하는 방법

테스트 환경

$ cat /etc/os-release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

epel 패키지 설치

amazon-linux-extras install -y epel

fluent-bit.repo 저장소 추가

• 'EOF' : 작은따옴표로 감싸야지 변수($releasever)를 그대로 입력할 수 있습니다.

cat <<'EOF' > /etc/yum.repos.d/fluent-bit.repo
...
EOF

cat <<'EOF' > /etc/yum.repos.d/fluent-bit.repo
[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/amazonlinux/$releasever/$basearch/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
repo_gpgcheck=1
enabled=1
EOF

$ yum repolist
Loaded plugins: extras_suggestions, fastestmirror, langpacks, priorities, update-motd
Loading mirror speeds from cached hostfile
 * epel: mirror-jp.misakamikoto.network
245 packages excluded due to repository priority protections
repo id                                    repo name                                                      status
amzn2-core/2/x86_64                        Amazon Linux 2 core repository                                     30,036
amzn2extra-docker/2/x86_64                 Amazon Extras repo for docker                                          79
amzn2extra-epel/2/x86_64                   Amazon Extras repo for epel                                             1
amzn2extra-nginx1/2/x86_64                 Amazon Extras repo for nginx1                                         104
amzn2extra-php7.3/2/x86_64                 Amazon Extras repo for php7.3                                         685
epel/x86_64                                Extra Packages for Enterprise Linux 7 - x86_64                 13,501+245
fluent-bit/2/x86_64                        Fluent Bit                                                        141+143
repolist: 44,547

fluent-bit 설치하기

fluent-bit 패키지 설치

yum install -y fluent-bit

fluent-bit 심볼릭 링크 설정

ln -s /opt/fluent-bit/bin/fluent-bit /usr/local/bin/fluent-bit

fluent-bit 버전 정보 확인

fluent-bit --version

$ fluent-bit --version
Fluent Bit v2.0.8
Git commit:

fluent-bit.conf 설정 파일

$ cat /etc/fluent-bit/fluent-bit.conf | egrep -v '^$|#'
[SERVICE]
    flush        1
    daemon       Off
    log_level    info
    parsers_file parsers.conf
    plugins_file plugins.conf
    http_server  Off
    http_listen  0.0.0.0
    http_port    2020
    storage.metrics on
[INPUT]
    name cpu
    tag  cpu.local
    interval_sec 1
[OUTPUT]
    name  stdout
    match *

fluent-bit -c /etc/fluent-bit/fluent-bit.conf --dry-run

$ fluent-bit -c /etc/fluent-bit/fluent-bit.conf --dry-run
Fluent Bit v2.0.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

configuration test is successful

systemctl --now enable fluent-bit

systemctl status fluent-bit

$ systemctl status fluent-bit
● fluent-bit.service - Fluent Bit
   Loaded: loaded (/usr/lib/systemd/system/fluent-bit.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2023-02-22 20:13:44 KST; 14s ago
     Docs: https://docs.fluentbit.io/manual/
 Main PID: 19106 (fluent-bit)
   CGroup: /system.slice/fluent-bit.service
           └─19106 /opt/fluent-bit/bin/fluent-bit -c //etc/fluent-bit/fluent-bit.conf
...
Hint: Some lines were ellipsized, use -l to show in full.

---

fluent-bit.conf 설정 파일 변경

vim fluent-bit.conf

[SERVICE]
    flush        1
    daemon       Off
    log_level    error
    parsers_file parsers.conf
    plugins_file plugins.conf

    http_server  On
    http_listen  0.0.0.0
    http_port    2020

    storage.metrics on

#[INPUT]
#    name cpu
#    tag  cpu.local
#    interval_sec 1

[INPUT]
    Name   tail
    Tag    nginx.access
    Path   /var/log/nginx/access.log
    Parser nginx

[FILTER]
    Name   record_modifier
    Match  nginx.access
    Record hostname ${HOSTNAME}

#[OUTPUT]
#    Name  es
#    Host  elasticsearch.scbyun.com
#    Port  9200
#    Match *
#    Index fluentbit
#    http_user elastic
#    http_passwd elastic

[OUTPUT]
    Name   forward
    Match  nginx.access
    Host   fluentd.scbyun.com
    Port   24224
 
# [OUTPUT]
#    name  stdout
#    match *

fluent-bit -c /etc/fluent-bit/fluent-bit.conf --dry-run

$ fluent-bit -c /etc/fluent-bit/fluent-bit.conf --dry-run
Fluent Bit v2.0.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

configuration test is successful

error

$ fluent-bit -c /etc/fluent-bit/fluent-bit.conf --dry-run
Fluent Bit v2.0.8
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/02/22 21:30:18] [error] [config] extra indentation level found
[2023/02/22 21:30:18] [error] [config] error in /etc/fluent-bit/fluent-bit.conf:30: invalid indentation level
[2023/02/22 21:30:18] [error] configuration file contains errors, aborting.​

systemctl restart fluent-bit

systemctl status fluent-bit