Cloud architecture patterns
The same few designs come up whichever cloud you use. Each pattern says when it applies, what tells you it is missing, the steps to put it in place, and the lessons and labs across our cloud, certification and system design courses where you practise it.
Least privilege, checked by simulation before it ships
When to use it: You are giving a person, a workload or a pipeline access to cloud resources.
Signals
- Policies with wildcard actions or resources, such as "*"
- Long-lived access keys in code, CI settings or laptops
- Nobody can say which permissions a role actually uses
Steps
- Write down the job the identity does, then grant only the actions and resources that job needs.
- Prefer roles with short-lived credentials over long-lived keys, and remove keys that have not been used.
- Add conditions where they narrow access further, such as a required tag or source.
- Ask the policy engine to judge the document before you attach it, using a simulator or a dry run.
- Review regularly and remove permissions the role has not used.
Where it is taught
- Policies and Least Privilege — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- Asking the engine to judge both documents — Lab, Cloud Infrastructure & Hyperscaler Architecture
- The Danger of Long-Lived Keys — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- Policy Simulation and Dry Runs — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- What the engine says about that document — Lab, Cloud Infrastructure & Hyperscaler Architecture
Spread across zones, and know what fails over by itself
When to use it: A service must keep running when one data centre, or one availability zone, goes down.
Signals
- Every instance of a tier sits in the same zone
- A load balancer that reports healthy while users get errors
- Nobody has tested what happens when a zone is lost
Steps
- Run each tier in at least two availability zones, behind a load balancer that spans them.
- Know, tier by tier, which parts fail over on their own (a managed database's standby, a load balancer) and which need you.
- Make health checks test what users need, not only that a port is open.
- Check what crossing a zone or region costs in latency and data transfer before you spread chatty components.
- Practise losing a zone, and write down what happened.
Where it is taught
- Regions, Availability Zones and Reliability — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- High Availability Across Tiers: What Fails Over By Itself? — Lesson, AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- The drill, replayed — Lab, AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Healthy, Said The Load Balancer: Azure DNS And Load Balancing — Lesson, Microsoft Azure Administrator (AZ-104) Exam Prep
- Thursday, replayed with both probes — Lab, Microsoft Azure Administrator (AZ-104) Exam Prep
Private by default: a public edge, and private subnets behind it
When to use it: You are laying out the network for an application and its data.
Signals
- Databases or internal services with public IP addresses
- Security groups or firewall rules open to 0.0.0.0/0 on management ports
- Traffic to other cloud services leaving over the public internet
Steps
- Put only the entry point, such as a load balancer, in public subnets; keep application and data tiers private.
- Give private subnets outbound access through NAT only if they need it.
- Allow traffic between tiers by reference to the tier's security group, not by wide address ranges.
- Reach managed services through private endpoints, and administer machines through a bastion or a managed session service rather than open SSH.
- Read the route tables to prove which subnets are public: a subnet is public because its route table sends traffic to an internet gateway.
Where it is taught
- Subnets: Public and Private — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- Classify a route table without the cloud — Lab, Cloud Infrastructure & Hyperscaler Architecture
- Security Groups and Firewalls — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- No Public Doors: Bastion, Service Endpoints And Private Endpoints — Lesson, Microsoft Azure Administrator (AZ-104) Exam Prep
- Cloud Network Isolation: VPC Peering & PrivateLink — Lesson, Cloud Security & DevSecOps Engineering
Scale on a signal, before the demand arrives
When to use it: Load changes through the day or the week, or arrives in sudden spikes.
Signals
- Capacity that arrives after the peak has already hurt users
- Servers sized for the peak and idle the rest of the time
- A queue or latency that grows every time traffic rises
Steps
- Pick a signal that rises before users suffer, such as requests per instance or queue depth, rather than CPU alone.
- Set a minimum and a maximum, so scaling cannot drop below safety or run up an unbounded bill.
- Account for how long new capacity takes to become ready, and schedule it ahead of known peaks.
- Keep instances stateless, so any one of them can be added or removed.
- Replay a real busy period against the policy and check capacity arrived in time.
Where it is taught
- Auto Scaling Groups — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- Scaling Patterns: When Capacity Arrives Too Late — Lesson, AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- The drill, four ways — Lab, AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Elasticity and Scale — Lesson, Cloud Infrastructure & Hyperscaler Architecture
- Last Monday, with autoscale — Lab, Microsoft Azure Administrator (AZ-104) Exam Prep
Backups you have restored: design to an RTO and an RPO
When to use it: You need to recover from data loss, corruption, deletion or the loss of a region.
Signals
- Backups that have never been restored
- No agreed answer to how much data may be lost (RPO) or how long recovery may take (RTO)
- Backups that an attacker, or a mistake, could delete along with the data
Steps
- Agree the recovery point objective and recovery time objective for each system with its owners.
- Choose the cheapest approach that meets both, from backup and restore up to a warm standby.
- Protect backups from deletion: a separate account or vault, and retention locks where you can.
- Restore regularly into a clean environment and time it against the RTO.
- Write the restore steps down where people can find them during an outage.
Where it is taught
- Recovery To RTO And RPO: Backups Nobody Could Delete — Lesson, AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- The cheapest approach that meets both — Lab, AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Restore It From Last Tuesday: Backup Vaults, Policies And Site Recovery — Lesson, Microsoft Azure Administrator (AZ-104) Exam Prep
- Which points exist today? — Lab, Microsoft Azure Administrator (AZ-104) Exam Prep
- etcd Snapshots And A Tested Restore — Lesson, Kubernetes: Production-Grade Container Orchestration
Absorb load with a cache in front and a queue behind
When to use it: A backend is slow or expensive to call, or work arrives in bursts faster than it can be done.
Signals
- The same reads hitting the database over and over
- A cold cache, or many keys expiring together, that floods the origin
- Requests waiting on work that could happen later
Steps
- Cache reads close to where they are needed, and decide how each cached item is invalidated.
- Protect the origin from stampedes: let one request refill a missing key while the others wait, and add jitter to expiry times.
- Move work that does not need an immediate answer onto a queue, and acknowledge the request straight away.
- Bound the queue and watch its depth: by Little's law, waiting time grows with the work in the queue.
- Make the consumers idempotent, so a message delivered twice does no harm.
Where it is taught
- Caching Strategies And Topology — Lesson, Scalable System Design & Distributed Architecture
- Cache Invalidation And The Three Hard Cases — Lesson, Scalable System Design & Distributed Architecture
- The stampede, and what single flight is worth — Lab, Scalable System Design & Distributed Architecture
- Queues, Events And Asynchronous Work — Lesson, Scalable System Design & Distributed Architecture
- Change one constant and the queue stops being a buffer — Lab, Scalable System Design & Distributed Architecture
Timeouts, retries with jitter, and circuit breakers
When to use it: One service calls another over the network, and a slow dependency must not take the caller down with it.
Signals
- One slow dependency making every caller slow, all the way to the user
- Retry storms that arrive in waves after an outage
- Threads or connections used up while waiting on a dependency that has stopped answering
Steps
- Give every network call a timeout, budgeted from the user's deadline inwards.
- Retry only what is safe to retry, with exponential backoff and random jitter, and a limit.
- Put a circuit breaker on calls to a dependency that is failing, so callers fail fast while it recovers.
- Shed load you cannot serve in time, keeping the work that matters most.
- Test the failure paths on purpose, by slowing or stopping a dependency.
Where it is taught
- Timeouts, And Avoiding Cascaded Failure — Lesson, Scalable System Design & Distributed Architecture
- Retries, Backoff And Jitter — Lesson, Scalable System Design & Distributed Architecture
- Jitter separates the retry wave — Lab, Scalable System Design & Distributed Architecture
- The breaker changes the shape of failure — Lab, Scalable System Design & Distributed Architecture
- A shedder keeps the useful work alive — Lab, Scalable System Design & Distributed Architecture