Should you find yourself in an interview scenario where you’re tasked with designing a system capable of scaling to millions of users, there are several crucial factors, design choices, and technological selections to keep in mind. It’s important to note that cloud providers like AWS inherently offer infrastructure on demand. Consequently, if your application is well-designed, it possesses the ability to scale automatically, albeit at a certain financial expense.
The following are some key features and design considerations that help scale your application.
Load Balancing
AWS provides different types of load balancers – Elastic Load Balancer (ELB), Application Load Balancer (ALB), and Network Load Balancer (NLB). ELB is a classic load balancer suitable for applications that were built within the EC2-Classic network. ALB operates at the request level and is best suited for load balancing of HTTP and HTTPS traffic, while NLB operates at the connection level and is ideal for load balancing of TCP traffic where extreme performance is required. These load balancers distribute incoming application traffic across multiple targets, such as Amazon EC2 instances, containers, and IP addresses, in multiple Availability Zones, increasing the availability and fault tolerance of your applications.
Auto Scaling
AWS Auto Scaling can be used to automatically adjust the number of EC2 instances in response to traffic patterns. This helps ensure that the number of instances scales up during demand spikes to maintain performance, and scales down during demand lulls to minimize costs.
Auto Scaling uses scaling policies to determine when to scale out (add instances) or scale in (remove instances). These policies are based on CloudWatch metrics such as CPU utilization, network traffic, or even custom metrics that you define.
Content Delivery Network (CDN)
AWS CloudFront can be used to deliver your content, like data, videos, applications, and APIs, to your customers globally with low latency and high transfer speeds. CDNs help improve performance by caching content at the edge locations closest to your users, reducing the latency of requests for content.
Database
Amazon RDS (Relational Database Service) can be used for relational databases. It’s easy to set up, operate, and scale. Amazon RDS includes several database engines such as Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle Database, and SQL Server. Amazon Aurora is a MySQL and PostgreSQL-compatible relational database built for the cloud, that combines the performance and availability of traditional enterprise databases with the simplicity and cost-effectiveness of open source databases. For NoSQL databases, Amazon DynamoDB can be used. It provides fast and predictable performance with seamless scalability.
Storage
Amazon S3 (Simple Storage Service) can be used for object storage which offers industry-leading scalability, data availability, security, and performance. S3 automatically scales to high request rates. For example, your application can achieve at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix in a bucket. There are no limits to the number of prefixes in a bucket. You can increase your read or write performance by parallelizing reads and writes across multiple prefixes.
Amazon S3 Transfer Acceleration enables fast, easy, and secure transfers of files over long distances between your client and your Amazon S3 bucket. It works by carrying HTTP and HTTPS traffic over a highly optimized, congestion-resistant network path.
Serverless Architecture
AWS Lambda allows you to run your code without provisioning or managing servers. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app. Lambda automatically scales your applications in response to incoming request traffic. You don’t need to set up any scaling configurations or manage any servers. Lambda runs your code concurrently and processes each trigger individually, scaling precisely with the size of the workload.
Microservices
AWS ECS (Elastic Container Service) or EKS (Elastic Kubernetes Service) can be used to run Docker-based microservices. This allows you to scale and deploy services independently.
Caching
Amazon ElastiCache offers fully managed Redis and Memcached. It can be used to enhance the performance of data-intensive apps and improve the response time of your applications by retrieving data that doesn’t change frequently from in-memory caches. Caches help alleviate read load from your databases which are often a major bottleneck in achieving and maintaining performance.
Messaging Systems
AWS provides messaging services like SQS (Simple Queue Service) and SNS (Simple Notification Service) which can be used to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work.
Monitoring and Logging
You can use Amazon CloudWatch to collect and track metrics, monitor log files, and respond to system-wide performance changes.
Security
AWS Identity and Access Management (IAM) can be used to manage access to your AWS resources. AWS Shield provides managed DDoS protection and AWS WAF (Web Application Firewall) provides application-level protection.