aws ecs 通过efs挂载实现动态更新firelens日志配置

zhaojiew10

已于 2024-05-25 22:19:06 修改

阅读量957

点赞数 1

分类专栏： AWS 文章标签： aws 云计算 java

于 2022-12-01 18:19:01 首次发布

本文链接：https://blog.csdn.net/sinat_41567654/article/details/128136483

版权

AWS 专栏收录该内容

72 篇文章 4 订阅

订阅专栏

参考：

aws firelens

firelens是ecs平台上容器独有的log driver，firelens可以和 fluentbit 和 fluentd 集成（实际上还是使用fluentbit容器对日志进行处理），将fluentbit托管在firelens下能够更好地和aws service集成，并且更灵活地进行配置。

值得一提的是，firelens官方提供的example比较全面，降低了配置难度

FireLens for Fluent Bit 支持将日志发送到：

Amazon CloudWatch Logs
Amazon Kinesis Data Firehose
Amazon Kinesis Data Streams
Fluent Bit 原生支持的所有目标

FireLens 在端口 24224 上侦听，因此为了确保从任务外部无法访问 FireLens 日志路由器，不得允许任务使用的安全组中端口 24224 上的入站流量。对于使用 awsvpc 网络模式的任务，这是与任务关联的安全组。对于使用 host 网络模式的任务，它是与托管任务的 Amazon EC2 实例关联的安全组。对于使用 bridge 网络模式的任务，请勿创建任何使用端口 24224 的端口映射。

注意事项：

windows不支持firelens
FireLens 在端口 24224 上侦听，需要确保任务外部无法访问此端口（安全组）

默认情况下，FireLens 将 集群、任务定义名称、集群的ARN 作为元数据键添加到 stdout/stderr 容器日志中，可以不配置

"firelensConfiguration":{
   "type":"fluentbit",
   "options":{
      "enable-ecs-log-metadata":"false",
      "config-file-type":"file",
      "config-file-value":"/extra.conf"
}

如果需要获取ecr或者secret manager需要配置taskexecrole
如果配置从s3获取配置文件，需要s3的相关权限

配置ecs任务

使用python的bootle创建简单的web server

from bottle import route, error, get
import logging

def logger_config(log_path,logging_name):
    logger = logging.getLogger(logging_name)
    logger.setLevel(level=logging.DEBUG)
    handler = logging.FileHandler(log_path, encoding='UTF-8')
    handler.setLevel(logging.INFO)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    handler.setFormatter(formatter)
    console = logging.StreamHandler()
    console.setLevel(logging.DEBUG)
    logger.addHandler(handler)
    logger.addHandler(console)
    return logger

logger = logger_config("infolog.txt","test-log-info")

@route('/log')
def mylog():
    logger.info("info")
    logger.error("error")
    logger.debug("debug")
    logger.warning("warning")

@get('/')
def health():
    return 'success'

@error(404)
def error404(error):
    return 'Nothing here, sorry'

run(host='0.0.0.0', port=8090)

将脚本打包上传到ecr中

FROM python:latest
WORKDIR /opt/web
COPY . .
EXPOSE 80
ENTRYPOINT python ./web.py

创建任务定义引用如下，创建服务启动即可（这里使用ec2平台，便于查看容器）

{
    "name": "myweb",
    "image": "xxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/xxxxxx",
    "executionRoleArn": "arn:aws-cn:iam::xxxxxxxxxx:role/ecsTaskExecutionRole",
    "containerDefinitions": [
        {
            // 指定业务容器地日志驱动为awsfirelens
            "logConfiguration": {
                "logDriver": "awsfirelens",
                "options": {
                    "log_group_name": "/aws/ecs/containerinsights/$(ecs_cluster)/application",
                    "log_stream_name": "$(ecs_task_id)",
                    "auto_create_group": "true",
                    "log_key": "log",
                    "region": "cn-north-1",
                    "Name": "cloudwatch",
                    "retry_limit": "2"
                }
            },
            // 容器暴露8090端口
            "portMappings": [
                {
                    "hostPort": 0,
                    "protocol": "tcp",
                    "containerPort": 80
                }
            ],
            // 挂载日志卷
            "mountPoints": [
                {
                    "readOnly": null,
                    "containerPath": "/opt/web/",
                    "sourceVolume": "var-log"
                }
            ]
        },
        {
            "name": "log_router",
            "image": "128054284489.dkr.ecr.cn-north-1.amazonaws.com.cn/aws-for-fluent-bit:latest",
            // 配置firelens类型为fluentbit，配置文件位置为/configs/extra-myweb.conf
            "firelensConfiguration": {
                "type": "fluentbit",
                "options": {
                    "config-file-type": "file",
                    "config-file-value": "/configs/extra-myweb.conf"
                }
            },
            // 将fluentbit本身的日志发送到/ecs/myweb日志组
            "logConfiguration": {
                "logDriver": "awslogs",
                "secretOptions": null,
                "options": {
                    "awslogs-group": "/ecs/myweb",
                    "awslogs-region": "cn-north-1",
                    "awslogs-stream-prefix": "ecs"
                }
            },
            "mountPoints": [
                // 挂载firelens-conf卷到/configs下
                {
                    "readOnly": null,
                    "containerPath": "/configs",
                    "sourceVolume": "firelens-conf"
                }，
                // 挂载日志卷
                {
                    "readOnly": null,
                    "containerPath": "/opt/web/",
                    "sourceVolume": "var-log"
                }
            ]
        }
    ],
    "volumes": [
        // 创建efs卷存放配置文件
        {
            "fsxWindowsFileServerVolumeConfiguration": null,
            "efsVolumeConfiguration": {
                "transitEncryptionPort": null,
                "fileSystemId": "fs-xxxxxxx",
                "authorizationConfig": {
                    "iam": "ENABLED",
                    "accessPointId": null
                },
                "transitEncryption": "ENABLED",
                "rootDirectory": "/test-ecs/firelens-config"
            },
            "name": "firelens-conf",
            "host": null,
            "dockerVolumeConfiguration": null
        }，
        // 创建日志卷
        {
            "name": "var-log",
            "host": {
                "sourcePath": null
            }
        }
    ],
    "memory": "512",
    "cpu": "1024",
    "taskRoleArn": "arn:aws-cn:iam::xxxxxxxxxx:role/ecsTaskRole",
    "family": "myweb",
    "requiresCompatibilities": [
        "EC2"
    ],
    "networkMode": "bridge"
}

创建fluentbit配置文件，抓取

[INPUT]
    Name              tail
    Tag               myweb-firelens
    Path              /opt/web/infolog.txt
    DB                /var/log/myweb_service.db
    DB.locking        true
    Skip_Long_Lines   On
    Refresh_Interval  10
    Rotate_Wait       30

[OUTPUT]
    Name  stdout
    Match myweb-firelens

进入ecs 的ec2实例查看fluentbit容器的配置文件，可以看到在其中引用了@INCLUDE /configs/extra-myweb.conf配置

bash-4.2# cat /fluent-bit/etc/fluent-bit.conf

[INPUT]
    Name forward
    unix_path /var/run/fluent.sock

[INPUT]
    Name forward
    Listen 0.0.0.0
    Port 24224

[INPUT]
    Name tcp
    Tag firelens-healthcheck
    Listen 127.0.0.1
    Port 8877

[FILTER]
    Name record_modifier
    Match *
    Record ec2_instance_id i-xxxxxx
    Record ecs_cluster worktest
    Record ecs_task_arn arn:aws-cn:ecs:cn-north-1:xxxxxxxx:task/worktest/0a5ecaa32f784edab3a723b8c8390a06
    Record ecs_task_definition myweb:9

@INCLUDE /configs/extra-myweb.conf

[OUTPUT]
    Name null
    Match firelens-healthcheck

[OUTPUT]
    Name cloudwatch
    Match myweb-firelens*
    auto_create_group true
    log_group_name /aws/ecs/containerinsights/$(ecs_cluster)/application
    log_key log
    log_stream_name $(ecs_task_id)
    region cn-north-1
    retry_limit 2

在efs卷中配置的fluentbit extra-config，为input打上tag，抓取/opt/web/infolog.txt路径下日志

[INPUT]
    Name              tail
    Tag               myweb-firelens
    Path              /opt/web/infolog.txt
    DB                /var/log/myweb_service.db
    DB.locking        true
    Skip_Long_Lines   On
    Refresh_Interval  10
    Rotate_Wait       30

[OUTPUT]
    Name  stdout
    Match myweb-firelens

查看业务容器访问日志

$ cat infolog.txt
2022-09-27 13:23:47,244 - test-log-info - INFO - info
2022-09-27 13:23:47,244 - test-log-info - ERROR - error
2022-09-27 13:23:47,248 - test-log-info - WARNING - warning

我们将容器访问日志打到fluentbit的stdout中

$ docker logs d8963f1052a4
[0] myweb-firelens: [1669883566.583834793, {"log"=>"2022-09-27 13:23:47,244 - test-log-info - INFO - info", "ec2_instance_id"=>"i-xxxxxxxxxxxx", "ecs_cluster"=>"worktest", "ecs_task_arn"=>"arn:aws-cn:ecs:cn-north-1:xxxxxxxx:task/worktest/0a5ecaa32f784edab3a723b8c8390a06", "ecs_task_definition"=>"myweb:9"}]
[1] myweb-firelens: [1669883566.583836988, {"log"=>"2022-09-27 13:23:47,244 - test-log-info - ERROR - error", "ec2_instance_id"=>"i-xxxxxxxxxxxxxx", "ecs_cluster"=>"worktest", "ecs_task_arn"=>"arn:aws-cn:ecs:cn-north-1:xxxxxxxx:task/worktest/0a5ecaa32f784edab3a723b8c8390a06", "ecs_task_definition"=>"myweb:9"}]
[2] myweb-firelens: [1669883566.583837240, {"log"=>"2022-09-27 13:23:47,248 - test-log-info - WARNING - warning", "ec2_instance_id"=>"i-xxxxxxxxxxxxxx", "ecs_cluster"=>"worktest", "ecs_task_arn"=>"arn:aws-cn:ecs:cn-north-1:xxxxxxxx:task/worktest/0a5ecaa32f784edab3a723b8c8390a06", "ecs_task_definition"=>"myweb:9"}]