aws beanstalk 实例日志和环境状态日志的轮换和流式传输配置

参考资料

• https://docs.aws.amazon.com/zh_cn/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html

• 查看您的 Elastic Beanstalk 环境中的 Amazon EC2 实例的日志

• https://docs.aws.amazon.com/zh_cn/elasticbeanstalk/latest/dg/environments-cfg-logging.html

eb相关日志的说明

日志源

beanstalk环境中的相关日志主要包括以下几个

• 实例日志

  • Web 服务器

  • 应用程序服务器

  • beanstalk平台脚本和

  • cloudformation创建日志

• 环境状态日志

日志分类

按照日志的类型又可以分为

• 结尾日志，beanstalk的操作日志和webserver日志，将条目保存为文本文件
• 捆绑日志（windows平台不支持），yum 和 cron日志，cloudformation日志，打包为zip包

日志上传方式

日志的上传也有两种方式

• 将轮换的实例日志上传到环境的s3存储桶

• 将实例日志流式传输到CloudWatch Logs，相关配置项如下（包括实例日志和环境状态日志）

  aws:elasticbeanstalk:cloudwatch:logs:
    DeleteOnTerminate: 'false'
    RetentionInDays: '7'
    StreamLogs: 'false'
  aws:elasticbeanstalk:cloudwatch:logs:health:
    DeleteOnTerminate: 'false'
    HealthStreamingEnabled: 'false'
    RetentionInDays: '7'
  

日志配置项

日志配置主要涉及到的命名空间如下

• aws:elasticbeanstalk:hostmanager - 配置为将轮换日志上传到 Amazon S3
• aws:elasticbeanstalk:cloudwatch:logs - 配置为将实例日志流式传输到 CloudWatch
• aws:elasticbeanstalk:cloudwatch:logs:health - 配置为将环境运行状况流式传输到 CloudWatch

实例日志的配置

可以直接在命令行快速开启

eb logs --log-group xxxxx
eb logs --cloudwatch-logs enable --cloudwatch-log-source instance
eb logs --cloudwatch-logs enable --cloudwatch-log-source environment-health
eb logs --stream

流日志

在应用根目录增加以下配置，aws:elasticbeanstalk:cloudwatch:logs

$ cat .ebextensions/cloudwatch.config
option_settings:
  aws:elasticbeanstalk:cloudwatch:logs:
    DeleteOnTerminate: 'false'
    RetentionInDays: '1'
    StreamLogs: 'true'

配置完成后控制台可以看到日志组名称

默认收集的日志如下

s3日志

在应用根目录增加以下配置，aws:elasticbeanstalk:hostmanager

$ cat .ebextensions/hostmanagerlog.config
option_settings:
  aws:elasticbeanstalk:hostmanager:
    LogPublicationControl: true

上传的日志清单为

https://docs.aws.amazon.com/zh_cn/elasticbeanstalk/latest/dg/using-features.logging.html

查看应用对应的s3桶，eb实例每小时上传一次轮换日志

• 如果应用程序不是在环境平台默认配置的位置生成日志，可以使用配置文件 (.ebextensions) 扩展默认配置。

• 也可以将应用程序的日志文件添加到结尾日志、捆绑日志或日志轮换中

s3日志的存储位置为，名为elasticbeanstalk-region-account-id的存储桶，路径为resources/environments/logs/logtype/environment-id/instance-id``

日志名称规范为

• 结尾日志 –s3://elasticbeanstalk-us-west-2-123456789012/resources/environments/logs/tail/e-mpcwnwheky/i-0a1fd158
• 捆绑日志 –s3://elasticbeanstalk-us-west-2-123456789012/resources/environments/logs/bundle/e-mpcwnwheky/i-0a1fd158
• 轮换日志 –s3://elasticbeanstalk-us-west-2-123456789012/resources/environments/logs/publish/e-mpcwnwheky/i-0a1fd158

结尾日志和捆绑日志创建15分钟后删除（eb使用用户权限删除s3）

对于轮换日志，具体配置在/etc/logrotate.elasticbeanstalk.hourly/中

$ ll /etc/logrotate.elasticbeanstalk.hourly
total 28
-rw-r--r-- 1 root root 157 Apr  1 11:11 logrotate.elasticbeanstalk.eb-engine.conf
-rw-r--r-- 1 root root 156 Apr  1 11:11 logrotate.elasticbeanstalk.eb-hooks.conf
-rw-r--r-- 1 root root 171 Apr  1 11:45 logrotate.elasticbeanstalk.healthd.conf
-rw-r--r-- 1 root root 157 Apr  1 11:45 logrotate.elasticbeanstalk.nginx.conf
-rw-r--r-- 1 root root 156 Apr  1 11:11 logrotate.elasticbeanstalk.web-stderr.conf
-rw-r--r-- 1 root root 156 Apr  1 11:11 logrotate.elasticbeanstalk.web-stdout.conf
-rw-r--r-- 1 root root 162 Apr  1 11:45 logrotate.elasticbeanstalk.xray.conf

查看eb-engine日志配置

$ cat logrotate.elasticbeanstalk.eb-engine.conf
/var/log/eb-engine.log {
 su root root
 size 10M
 rotate 5
 missingok
 compress
 notifempty
 copytruncate
 dateext
 dateformat %s
 olddir /var/log/rotated
}

这些配置文件由定时任务配置进行调用

$ ls /etc/cron.hourly/
0anacron                                             cron.logrotate.elasticbeanstalk.eb-hooks.conf  cron.logrotate.elasticbeanstalk.web-stderr.conf
cron.logcleanup.elasticbeanstalk.healthd-proxy.conf  cron.logrotate.elasticbeanstalk.healthd.conf   cron.logrotate.elasticbeanstalk.web-stdout.conf
cron.logrotate.elasticbeanstalk.eb-engine.conf       cron.logrotate.elasticbeanstalk.nginx.conf     cron.logrotate.elasticbeanstalk.xray.conf

查看定时任务配置，可以看到关键程序为logrotate

$ cat /etc/cron.hourly/cron.logrotate.elasticbeanstalk.eb-engine.conf
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate -f /etc/logrotate.elasticbeanstalk.hourly/logrotate.elasticbeanstalk.eb-engine.conf

可以对默认的日志位置进行扩展，具体的配置路径

• 结尾日志 –/opt/elasticbeanstalk/tasks/taillogs.d/
• 捆绑日志 –/opt/elasticbeanstalk/tasks/bundlelogs.d/
• 轮换日志 –/opt/elasticbeanstalk/tasks/publishlogs.d/

扩展其实就是在.ebextension路径下在配置文件中设定日志配置的文件和内容

files:
  "/opt/elasticbeanstalk/tasks/taillogs.d/cwl-system.conf":
    content: |
      /var/log/awslogs-agent-setup.log
      /var/log/awslogs.log
      /var/log/eb-cwlogs.log
    mode : "000644"
  "/opt/elasticbeanstalk/tasks/bundlelogs.d/cwl-system.conf":
    content: |
      /var/log/awslogs-agent-setup.log
      /var/log/awslogs.log
      /var/log/eb-cwlogs.log
    mode : "000644"
  "/opt/elasticbeanstalk/tasks/systemtaillogs.d/cwl-system.conf":
    content: |
      /var/log/awslogs-agent-setup.log
      /var/log/awslogs.log
      /var/log/eb-cwlogs.log
    mode : "000644"
  "/opt/elasticbeanstalk/tasks/publishlogs.d/cwl-system.conf":
    content: |
      /var/log/awslogs-agent-setup.log    ## this isn't rotated
      /var/log/awslogs.log*.gz
    mode : "000644"

环境状态日志的配置

在应用根目录增加以下配置

$ cat .ebextensions/cloudwatch.config
option_settings:
  aws:elasticbeanstalk:healthreporting:system:
    SystemType: enhanced
  aws:elasticbeanstalk:cloudwatch:logs:health:
    DeleteOnTerminate: 'false'
    HealthStreamingEnabled: 'true'
    RetentionInDays: '1'

配置完成后控制台可以看到日志组名称

日志配置相关的仓库

• https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/awslogs-change-frequency.config
• https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/logs-streamtocloudwatch-linux-alami.config
• https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/logs-streamtocloudwatch-linux.config
• https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/instance-configuration/logs-uploadonterminate-linux.config