Reliability, Monitoring and Cost

Introduction

Getting an app running in the cloud is the start. Keeping it available, noticing when something is wrong, and not overspending are the daily work of cloud engineers.

Reliability building blocks

Hardware fails and traffic spikes, so design for it:

• Load balancer: spreads requests across several servers, and stops sending traffic to unhealthy ones using health checks.
• Multiple availability zones: run copies in at least two separate data centres, so one failure does not take you down.
• Auto scaling: add or remove servers automatically as load changes, based on measures such as CPU use or request count. It saves money and absorbs spikes.
• Stateless servers: keep user data in databases or caches, not on one server's disk, so any server can be replaced.
• Managed services with built-in redundancy where possible.
• Backups with tested restores.

Monitoring: the three pillars

You cannot fix what you cannot see. Collect:

• Metrics: numbers over time, such as CPU, memory, requests per second, error rate and latency.
• Logs: records of events, ideally structured (JSON) and collected in one searchable place.
• Traces: the path of one request through many services, to find where time goes.

The key service-level numbers are often called the golden signals: latency, traffic, errors and saturation (how full resources are).

Alerts that help

• Alert on symptoms users feel, such as high error rates or slow responses, rather than on every small internal change.
• Every alert should be actionable. If nobody needs to do anything, it is noise.
• Too many alerts cause alert fatigue, and real problems get ignored.
• Link alerts to runbooks, short guides on what to check and do.
• Use dashboards for the big picture, and keep one for each service.

Service level objectives

A service level objective (SLO) is a target, such as "99.9 percent of requests succeed in under 500 milliseconds each month". The leftover tolerance is the error budget. It helps teams balance reliability against shipping new features, and agree what "good enough" means.

Cost management

Cloud bills can surprise you. Practical controls:

1. Budgets and alerts at the account level, so you are told early.
2. Tag resources with owner, project and environment, so costs can be attributed.
3. Right-size: use monitoring to find oversized machines and shrink them.
4. Switch off development and test resources outside working hours, and delete forgotten ones.
5. Use the right pricing model: on-demand for flexible needs, committed use or savings plans for steady workloads, and spot (spare capacity at a discount, which can be taken away) for interruptible jobs.
6. Storage tiers and lifecycle rules that move old data to cheaper classes or delete it.
7. Watch data transfer costs, particularly traffic leaving the cloud or crossing regions.
8. Review the bill monthly, and treat cost as a design factor from the start.

This discipline is known as FinOps, a partnership of engineering, finance and business.

Try it yourself

For a website you know, list the four golden signals you would watch, two alerts you would create, and three cost controls you would apply on day one.

Summary

• Reliability comes from load balancing, multiple zones, auto scaling and stateless design.
• Monitor metrics, logs and traces, and create few, actionable alerts based on user-facing symptoms.
• Control cost with budgets, tags, right-sizing, schedules and the right pricing model.