When you consider which storage service to choose for a specific use case, it’s important to be familiar with the different storage services that AWS has to offer. I chose AWS Storage Gateway and Amazon Simple Storage Service (Amazon S3) as the services to support the solution. I chose these services because the customer requires the Network File System (NFS) protocol to remain in place for all on-premises applications. However, the customer also wants to store the files (that they will access) in AWS. Storage Gateway supports this use case.
AWS Storage Gateway
AWS Storage Gateway connects an on-premises software appliance with cloud-based storage to provide near-seamless integration with data security features between your on-premises IT environment and the AWS storage infrastructure. You can use the service to store data in the AWS Cloud for scalable and cost-effective storage that helps maintain data security. AWS Storage Gateway offers file-based, volume-based, and tape-based storage solutions.
Morgan recommended Amazon S3 File Gateway for the customer’s solution.
Amazon S3 File Gateway
Amazon S3 File Gateway supports a file interface into Amazon S3, and it combines a service and a virtual software appliance. By using this combination, you can store and retrieve objects in Amazon S3 by using industry-standard file protocols, such as NFS and Server Message Block (SMB). The software appliance (or gateway) is deployed into your on-premises environment as a virtual machine (VM) that runs on VMware ESXi, Microsoft Hyper-V, or Linux Kernel-based Virtual Machine (KVM) hypervisor. The gateway provides access to objects in Amazon S3 as files or file-share mount points. With S3 File Gateway, you can do the following:
- Store and retrieve files directly by using NFS version 3 or version 4.1.
- Store and retrieve files directly by using SMB file system version 2 or version 3.
- Access data directly in Amazon S3 from any AWS Cloud application or service.
- Manage your Amazon S3 data by using lifecycle policies, S3 Cross-Region Replication (CRR), and versioning. You can think of a S3 File Gateway as a file system mount on Amazon S3.
Amazon S3 File Gateway is designed to simplify file storage in Amazon S3. It integrates to existing applications through industry-standard file system protocols, and it provides a cost-effective alternative to on-premises storage. Amazon S3 File Gateway also provides low-latency access to data through transparent local caching. It manages data transfer to and from AWS, and it optimizes and streams data in parallel. Amazon S3 File Gateway also buffers applications from network congestion, and manages bandwidth consumption.For more resources about Amazon S3 File Gateway, see the following:
- For more information about Amazon S3 File Gateway, see What is Amazon S3 File Gateway?
- For more information about AWS Storage Gateway, see AWS Storage Gateway Features.
Amazon EBS
An Amazon Elastic Block Store (Amazon EBS) volume is a durable, block-level storage device that you can attach to your Amazon Elastic Compute Cloud (Amazon EC2) instances. After you attach a volume to an instance, you can use it as you would use a physical hard drive. EBS volumes are flexible. For current-generation volumes attached to current-generation instance types, you can dynamically increase size, modify the provisioned IOPS capacity, and change volume type on live production volumes.
It’s important that you know about the different types of EBS volume types and sizes, and how IOPS are correlated with volume size and type.
Amazon EBS provides the following volume types, which differ in performance characteristics and price, so that you can tailor your storage performance and cost to the needs of your applications.
- Solid-state drives (SSD): Optimized for transactional workloads involving frequent read/write operations with small I/O size, where the dominant performance attribute is IOPS. SSD-backed volume types include General Purpose SSD volumes and Provisioned IOPS SSD volumes
- Hard disk drives (HDD): Optimized for large, streaming workloads where the dominant performance attribute is throughput. HDD-backed volume types include Throughput Optimized HDD and Cold HDD volumes.
- Previous generation: Hard disk drives that you can use for workloads with small datasets, where data is accessed infrequently and performance is not a priority. We recommend that you consider a current-generation volume type instead.
Note that the number of available IOPs increases with the size of the volume. Thus, if you need more IOPs, you must vertically scale the volume.For more information, see Amazon EBS volume types.
Amazon EFS
Amazon Elastic File System (Amazon EFS) provides a simple, serverless elastic file system that you can use with AWS Cloud services and on-premises resources. It is built to scale on demand to petabytes without disrupting applications. Amazon EFS can grow and shrink automatically as you add and remove files, so it minimizes the need to provision and manage capacity to accommodate growth. Amazon EFS has a web services interface that you can use to create and configure file systems quickly and easily. The service manages all the file storage infrastructure for you, meaning that you can reduce the complexity of deploying, patching, and maintaining complex file system configurations.
Amazon EFS supports NFS version 4 (NFSv4.1 and NFSv4.0), so the applications and tools that you use today work with Amazon EFS. Multiple compute instances — including Amazon EC2, Amazon Elastic Container Service (Amazon ECS), and AWS Lambda — can access an Amazon EFS file system at the same time. Amazon EFS can thus provide a common data source for workloads and applications that run on more than one compute instance or server.
For more information, see What is Amazon EFS?
Amazon S3
Amazon S3 is an object storage service that stores data as objects within buckets. An object is a file and any metadata that describes the file. A bucket is a container for objects.
To store your data in Amazon S3, you first create a bucket and specify a bucket name and AWS Region. Then, you upload your data to that bucket as objects in Amazon S3. Each object has a key (or key name), which is the unique identifier for the object within the bucket.
S3 provides features that you can configure to support your specific use case. For example, you can use S3 Versioning to keep multiple versions of an object in the same bucket, which means that you can restore objects that are accidentally deleted or overwritten.
Storage classes
Amazon S3 offers a range of storage classes that are designed for different use cases. For example, you could store mission-critical production data in S3 Standard for frequent access. You could also save on costs by storing infrequently accessed data in S3 Standard-IA or S3 One Zone-IA. Finally, you could archive data at a low costs in S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, and S3 Glacier Deep Archive.
You can store data with changing or unknown access patterns in S3 Intelligent-Tiering, which optimizes storage costs by automatically moving your data between four access tiers when your access patterns change. These four access tiers include two low-latency access tiers, which are optimized for frequent and infrequent access. The four access tiers also include two opt-in archive access tiers that are designed for asynchronous access to rarely accessed data.
For more resources about Amazon S3, see the following:
- For more information, see What is Amazon S3?
- For more information, including links out to hands-on tutorials, see Getting Started with Amazon S3.
- For more information about Amazon S3 storage classes, see Amazon S3 Storage Classes.
