aws beanstalk 集成efs实现共享存储

参考资料

• https://docs.aws.amazon.com/zh_cn/elasticbeanstalk/latest/dg/services-efs.html
• Deploying WordPress on Elastic Beanstalk，Use the EB CLI to create an Elastic Beanstalk environment with an attached RDS DB and EFS file system to provide WordPress with a MySQL database and shared storage for uploaded files.

在efs中可以创建共享目录，以存储用户为应用程序上传或修改的文件。实现将应用在多个实例上扩展

beanstalk创建了webapp 用户，以应用程序目录所有者的身份在ec2实例上对其进行设置

beanstalk集成efs的两个关键配置

• storage-efs-createfilesystem.config，Resource资源，创建efs文件系统
• storage-efs-mountfilesystem.config，linux配置，在ec2实例上挂载efs

我们使用eb cli初始化python应用程序，并创建对应的环境

使用Resource创建efs文件系统

当然也可以使用已经存在的efs文件系统

$ cat .ebextensions/storage-efs-createfilesystem.config
option_settings:
  aws:elasticbeanstalk:customoption:
    VPCId: "vpc-00000000"
    SubnetA: "subnet-00000000"
    SubnetB: "subnet-00000000"

Resources:
  FileSystem:
    Type: AWS::EFS::FileSystem
    Properties:
      FileSystemTags:
      - Key: Name
        Value: "EB-EFS-FileSystem"
      PerformanceMode: "generalPurpose"
      Encrypted: "false"
      
  MountTargetA:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: {Ref: FileSystem}
      SecurityGroups:
      - {Ref: MountTargetSecurityGroup}
      SubnetId:
        Fn::GetOptionSetting: {OptionName: SubnetA}
  MountTargetB:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: {Ref: FileSystem}
      SecurityGroups:
      - {Ref: MountTargetSecurityGroup}
      SubnetId:
        Fn::GetOptionSetting: {OptionName: SubnetB}

  MountTargetSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for mount target
      SecurityGroupIngress:
      - FromPort: '2049'
        IpProtocol: tcp
        SourceSecurityGroupId:
          Fn::GetAtt: [AWSEBSecurityGroup, GroupId]
        ToPort: '2049'
      VpcId:
        Fn::GetOptionSetting: {OptionName: VPCId}

在Beanstalk实例上挂载efs

这里的option_settings字段中使用Ref函数引用了上一个配置文件中的FileSystem

$ cat .ebextensions/storage-efs-mountfilesystem.config
option_settings:
  aws:elasticbeanstalk:application:environment:
    FILE_SYSTEM_ID: '`{"Ref" : "FileSystem"}`'
    MOUNT_DIRECTORY: '/efs'

packages:
  yum:
    amazon-efs-utils: []

commands:
  01_mount:
    command: "/tmp/mount-efs.sh"

files:
  "/tmp/mount-efs.sh":
      mode: "000755"
      content : |
        #!/bin/bash

        EFS_MOUNT_DIR=$(/opt/elasticbeanstalk/bin/get-config environment -k MOUNT_DIRECTORY)
        EFS_FILE_SYSTEM_ID=$(/opt/elasticbeanstalk/bin/get-config environment -k FILE_SYSTEM_ID)

        echo "Mounting EFS filesystem ${EFS_FILE_SYSTEM_ID} to directory ${EFS_MOUNT_DIR} ..."

        echo 'Stopping NFS ID Mapper...'
        service rpcidmapd status &> /dev/null
        if [ $? -ne 0 ] ; then
            echo 'rpc.idmapd is already stopped!'
        else
            service rpcidmapd stop
            if [ $? -ne 0 ] ; then
                echo 'ERROR: Failed to stop NFS ID Mapper!'
                exit 1
            fi
        fi

        echo 'Checking if EFS mount directory exists...'
        if [ ! -d ${EFS_MOUNT_DIR} ]; then
            echo "Creating directory ${EFS_MOUNT_DIR} ..."
            mkdir -p ${EFS_MOUNT_DIR}
            if [ $? -ne 0 ]; then
                echo 'ERROR: Directory creation failed!'
                exit 1
            fi
        else
            echo "Directory ${EFS_MOUNT_DIR} already exists!"
        fi

        mountpoint -q ${EFS_MOUNT_DIR}
        if [ $? -ne 0 ]; then
            echo "mount -t efs -o tls ${EFS_FILE_SYSTEM_ID}:/ ${EFS_MOUNT_DIR}"
            mount -t efs -o tls ${EFS_FILE_SYSTEM_ID}:/ ${EFS_MOUNT_DIR}
            if [ $? -ne 0 ] ; then
                echo 'ERROR: Mount command failed!'
                exit 1
            fi
            chmod 777 ${EFS_MOUNT_DIR}
            runuser -l  ec2-user -c "touch ${EFS_MOUNT_DIR}/it_works"
            if [[ $? -ne 0 ]]; then
                echo 'ERROR: Permission Error!'
                exit 1
            else
                runuser -l  ec2-user -c "rm -f ${EFS_MOUNT_DIR}/it_works"
            fi
        else
            echo "Directory ${EFS_MOUNT_DIR} is already a valid mountpoint!"
        fi

        echo 'EFS mount complete.'

使用eb cli部署配置

$ eb deploy

登录示例查看部署结果，已经成功挂载

$ ls -al / | grep efs
drwxrwxrwx   2 root root 6144 Apr  1 10:54 efs

相关错误和解决

（1）yum makecache失败导致部署终止

在event中看到部署失败的错误

2023-04-01 18:08:39 UTC+0800 ERROR [Instance: i-0333d74d02a795321] Command failed on instance. Return code: 1 Output: Engine execution has encountered an error..
2023-04-01 18:08:35 UTC+0800 ERROR Instance deployment failed. For details, see 'eb-engine.log'.

查看对应的eb-engine.log

2023/04/01 10:08:31.986855 [INFO] Running command /bin/sh -c /opt/aws/bin/cfn-init -s arn:aws-cn:cloudformation:cn-north-1:037047667284:stack/awseb-e-ckmhwpf3kv-stack/8f1dca30-cf9e-11ed-ae99-0259e93a8964 -r AWSEBAutoScalingGroup --region cn-north-1 --configsets Infra-EmbeddedPreBuild
2023/04/01 10:08:35.458357 [INFO] Error occurred during build: Could not create yum cache (return code 1)
2023/04/01 10:08:35.458383 [ERROR] An error occurred during execution of command [app-deploy] - [PreBuildEbExtension]. Stop running the command. Error: EbExtension build failed. Please refer to /var/log/cfn-init.log for more details.

由于新部署实际是更新cloudfornamtion堆栈的操作，因此cfn-hup触发了cfn-init重新执行初始化，但是初始化行为报错。

查看/var/log/cfn-init.log日志，执行yum makecache报错

failure: repodata/filelists.sqlite.gz from amzn2-core: [Errno 256] No more mirrors to try.

删除缓存后重新执行

$ sudo rm -rf /var/cache/yum

（2）无法解析efs的dns

需要确保vpc开启dns解析

https://docs.aws.amazon.com/zh_cn/efs/latest/ug/troubleshooting-efs-mounting.html#mount-fails-dns-name