Architecture Optimizations for providing restaurant menus by using QR codes

Serverless on AWS

Morgan commented about changing the system that generates QR codes into something serverless, or a container. AWS offers technologies for running code, managing data, and integrating applications — all without managing servers. Serverless technologies feature automatic scaling, built-in high availability, and a pay-for-use billing model to increase agility and optimize costs. These technologies also reduce infrastructure management tasks, such as capacity provisioning and patching, so that you can focus on writing code that serves your customers. Serverless applications start with AWS Lambda, an event-driven compute service that natively integrates with over 200 AWS services and software as a service (SaaS) applications.

For more information, see Serverless on AWS.

Amazon CloudFront

Then, Raf commented that Amazon CloudFront could be used with the Amazon Simple Storage Service (Amazon S3) bucket that is used to display the restaurant menus. Amazon CloudFront is a web service that speeds up the distribution of your static and dynamic web content (such as .html, .css, .js, and image files) to your users. CloudFront delivers your content through a worldwide network of data centers that are called edge locations. When a user requests content that you’re serving with CloudFront, the request is routed to the edge location that provides the lowest latency (time delay), so that content is delivered with the best possible performance. In addition, Amazon CloudFront allows custom SSL certificates (issued by AWS Certificate Manager, or ACM), custom domain names, and further distributed denial of service (DDoS) protection that is powered by AWS WAF and AWS Shield.

For more information, see Amazon CloudFront.

Amazon Cognito

Morgan commented about using Amazon Cognito instead of Amazon API Gateway to send data to Amazon Kinesis. This solution would require refactoring code in the JavaScript library that sends the clickstream data. However, it would also bypass API Gateway, and therefore reduce cost. You can use Amazon Cognito to add user sign-up, sign-in, and access control to your web applications and mobile apps. Amazon Cognito scales to millions of users and supports sign-in with social identity providers, such as Apple, Facebook, Google, and Amazon, and enterprise identity providers through SAML 2.0 and OpenID Connect. For more information, see Amazon Cognito. With Amazon Cognito, you can have a mobile JavaScript library that assumes a role with a web identity. That role must have a strict policy associated with it — for example, the policy could allow only the PutRecord API operation on a specific Kinesis data stream.

For more information, see AssumeRoleWithWebIdentity in the AWS Security Token Service API Reference.

Amazon CloudFormation and IaC

Later in the conversation, Raf talked about transforming the entire environment into a template that could be further replicated for each of the customer’s clients. This can be achieved by treating infrastructure as code (IaC) with a managed AWS service called AWS CloudFormation. CloudFormation helps you model and set up your AWS resources so that you can spend less time managing those resources, and more time focusing on your applications that run in AWS. You create a template that describes all the AWS resources that you want — for example, Amazon Elastic Compute Cloud (Amazon EC2) instances or Amazon Relational Database Service (Amazon RDS) DB instances — and CloudFormation provisions and configures those resources for you. You don’t need to individually create and configure AWS resources and figure out which resource is dependent on another resource. For example, this YAML template snippet creates an S3 bucket with the following configurations:

• Static website hosting that uses index.html as the index document and error.html as the error document
• Bucket policy that allows everyone to GET objects (otherwise, the bucket is private)
• Deletion policy that asks CloudFormation to retain the bucket if the CloudFormation stack is deleted (this is a good practice when you use data services, such as buckets and databases)

The snippet also outputs the https:// URL that’s used by static website hosting, which you can send to someone else, or copy and paste into a web browser after the bucket is created.

AWSTemplateFormatVersion: 2010-09-09
Resources:
  S3Bucket:
    Type: 'AWS::S3::Bucket'
    Properties:
      AccessControl: PublicRead
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
    DeletionPolicy: Retain
  BucketPolicy:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
      PolicyDocument:
        Id: MyPolicy
        Version: 2012-10-17
        Statement:
          - Sid: PublicReadForGetBucketObjects
            Effect: Allow
            Principal: '*'
            Action: 's3:GetObject'
            Resource: !Join 
              - ''
              - - 'arn:aws:s3:::'
                - !Ref S3Bucket
                - /*
      Bucket: !Ref S3Bucket
Outputs:
  WebsiteURL:
    Value: !GetAtt 
      - S3Bucket
      - WebsiteURL
    Description: URL for website hosted on S3
  S3BucketSecureURL:
    Value: !Join 
      - ''
      - - 'https://'
        - !GetAtt 
          - S3Bucket
          - DomainName
    Description: Name of S3 bucket to hold website content

For more examples of how to use CloudFormation templates, see Template snippets in the AWS CloudFormation User Guide.

Creating infrastructure templates is important because you can replicate customer workloads across multiple AWS accounts or Regions. You will learn more about how relevant and important this is in week 4 of this course.

For more information, see AWS CloudFormation.

Error retries and exponential backoff in AWS

Morgan also talked about error retries and exponential backoffs. Although this topic seems complicated, here are some best practices and observations about error retries and exponential backoffs. Numerous components on a network — such as DNS servers, switches, load balancers, and others — can generate errors anywhere in the life of a given request. The usual technique for dealing with these error responses in a networked environment is to implement retries in the client application. This technique increases the reliability of the application and reduces operational costs for the developer. If you are using one of the AWS SDKs, they already practice exponential backoffs by default. This default behavior would be one benefit of using Amazon Cognito on the client side to send data to Kinesis, instead of using a custom routine that sends data to API Gateway. You should implement a maximum delay interval and a maximum number of retries. The maximum delay interval and maximum number of retries are not necessarily fixed values. These values should be set based on the operation that’s being performed and other local factors, such as network latency.

For more information, see Error retries and exponential backoff in AWS in the AWS General Reference.

Data optimizations for Amazon Athena

Last, but not least — in the conversation, Raf talked about data optimization for Amazon Athena. This topic is broad, and it even has dedicated AWS Blog posts about the Top 10 Performance Tuning Tips for Amazon Athena.

The top three data best practices for Athena are compressing, partitioning, and converting your data into columnar formats.

In brief, Athena is a service that charges per amount of data scanned. By data scanned, AWS means the amount of data that Athena fetches to perform a query. As you know, data compression can reduce file sizes, which means that Athena scans less data per query. Because Athena scans less data, the queries also run faster. You can save from 30 percent to 90 percent on your per-query costs, and get better performance by applying data best practices.

Data partitioning is another technique to restrict the amount of data that’s scanned by each query. You can partition your data by any key. A common practice is to partition the data based on time, which often leads to a multi-level partitioning scheme. For example, a customer who has data coming in every hour might decide to partition by year, month, date, and hour. Say that you have a 5-TB file containing monthly information in it, and this file pertains to a whole year. If you constantly query things by adding statements such as WHERE Month = “February”, the system will scan the entire 5-TB file for each query. The system will probably ignore data that doesn’t belong to the month of February. This process is slower and more expensive than partitioning the data so that you have files that belong to each individual month. If you partitioned the data by month, a query that contains a clause like WHERE Month = “February” would only scan data that pertains to the month of February, thus making the query faster and cheaper.

For more information, see Partitioning data in Athena.

Another strategy is to use an AWS service to convert bucket data into columnar data formats. Apache Parquet and ORC are columnar storage formats that are optimized for the fast retrieval of data and they are also used in AWS analytical applications. Columnar storage formats have the following characteristics that make them suitable for use with Athena:

• Compression by column, with the compression algorithm selected for the column data type to save storage space in Amazon S3, and reduce disk space and I/O during query processing.
• Predicate pushdown in Parquet and ORC enables Athena queries to fetch only the blocks that it needs, thus improving query performance. When an Athena query obtains specific column values from your data, it uses statistics from data block predicates, such as max or min values, to determine whether to read or skip the block.
• Splitting data in Parquet and ORC enables Athena to split the reading of data to multiple readers and increase parallelism when it processes queries.

For more information, see Converting to columnar formats.