16. AWS Storage Gateway

Overview

• AWS Storage Gateway connects an on-premises software appliance with cloud-based storage to provide seamless integration with data security features between your on-premises IT environment and the AWS storage infrastructure.
• AWS Storage Gateway offers file-based file gateways (Amazon S3 File and Amazon FSx File), volume-based (Cached and Stored), and tape-based storage solutions

Required prerequisites

• Configure Microsoft Active Directory (AD).
• Ensure that there is sufficient network bandwidth between the gateway and AWS. A minimum of 100 Mbps is required to successfully download, activate, and update the gateway.
• Configure your private networking, VPN, or AWS Direct Connect between your Amazon Virtual Private Cloud (Amazon VPC) and the on-premises environment where you are deploying your gateway, You must also have on-premises access to FSx for Windows File Server.
• Make sure your gateway can resolve the name of your Active Directory Domain Controller. You can use DHCP in your Active Directory domain to handle resolution, or specify a DNS server manually from the Network Configuration settings menu in the gateway local console.
• Hardware requirements for on-premises VMs
  • Four virtual processors assigned to the VM
  • 16 GiB of reserved RAM for file gateways
  • 80 GiB of disk space for installation of VM image and system data
• Storage requirements

  • In addition to 80 GiB of disk space for the VM, you also need additional disks for your gateway.

    Gateway type	Cache (minimum)	Cache (maximum)
    File gateway	150 GiB	64 TiB

Network and firewall requirements

• Your gateway requires access to the internet, local networks, Domain Name Service (DNS) servers, firewalls, routers, and so on.

Amazon S3 File Gateway

• Amazon S3 File Gateway supports a file interface into Amazon Simple Storage Service (Amazon S3) and combines a service and a virtual software appliance.
• By using this combination, you can store and retrieve objects in Amazon S3 using industry-standard file protocols such as Network File System (NFS) and Server Message Block (SMB).
• The software appliance, or gateway, is deployed into your on-premises environment as a virtual machine (VM) running on VMware ESXi, Microsoft Hyper-V, or Linux Kernel-based Virtual Machine (KVM) hypervisor.
• The gateway provides access to objects in S3 as files or file share mount points.
• How to use a file gateway:
  • downloading a VM image for the file gateway.
  • You then activate the file gateway from the AWS Management Console or through the Storage Gateway API. You can also create a file gateway using an Amazon EC2 image.
  • After the file gateway is activated, you create and configure your file share and associate that share with your Amazon Simple Storage Service (Amazon S3) bucket.
    • Doing this makes the share accessible by clients using either the Network File System (NFS) or Server Message Block (SMB) protocol.
    • Files written to a file share become objects in Amazon S3, with the path as the key.
    • There is a one-to-one mapping between files and objects, and the gateway asynchronously updates the objects in Amazon S3 as you change the files.
    • Existing objects in the Amazon S3 bucket appear as files in the file system, and the key becomes the path
• Objects are encrypted with Amazon S3–server-side encryption keys (SSE-S3). All data transfer is done through HTTPS.
• The service optimizes data transfer between the gateway and AWS using multipart parallel uploads or byte-range downloads, to better use the available bandwidth.
• Local cache is maintained to provide low latency access to the recently accessed data and reduce data egress charges. 

•  File gateway converts files to S3 objects when uploading files to Amazon S3.
• Common file operations change file metadata, which results in the deletion of the current S3 object and the creation of a new S3 object