Monitoring with CloudWatch

What CloudWatch is#

CloudWatch is AWS's built-in monitoring and observability service — it collects metrics, stores logs, and can trigger alarms and automated actions, for both AWS's own services and anything you choose to publish to it yourself. Nearly every AWS service sends metrics to CloudWatch automatically with zero configuration: an EC2 instance's CPU utilization, an RDS instance's connection count, a Lambda function's invocation count and error rate all show up without you writing any monitoring code at all.

Three pieces fit together:

• Metrics — numeric data points over time (CPU utilization, request count, queue depth), automatically collected for most AWS services and optionally published by your own application code too.
• Alarms — a rule that watches a metric and changes state (into ALARM) when it crosses a threshold you define, and can trigger a notification or an automated action when it does.
• Log groups — a place logs are collected and retained, organized into log streams within a group (typically one stream per instance, container, or Lambda invocation context).

Metrics#

Every metric belongs to a namespace (AWS/EC2, AWS/RDS, AWS/Lambda, or a custom one you define) and has dimensions that identify exactly what it's measuring (which instance, which function, which queue). Viewing an EC2 instance's CPU utilization for the last hour via the CLI:

aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
  --start-time 2026-08-25T00:00:00Z \
  --end-time 2026-08-25T12:00:00Z \
  --period 300 \
  --statistics Average Maximum

--period 300 groups data points into 5-minute buckets; --statistics Average Maximum requests both the average and the peak CPU utilization within each bucket. Applications can publish their own custom metrics the same way infrastructure does automatically:

aws cloudwatch put-metric-data \
  --namespace MyApp/Orders \
  --metric-name OrdersProcessed \
  --value 1 \
  --unit Count

A handful of custom metrics like this — orders processed, checkout failures, queue processing time — is often far more useful for understanding whether an application is healthy than infrastructure metrics alone, which only tell you the server is running, not that it's doing its job correctly.

Alarms#

An alarm watches one metric and fires when it breaches a threshold for a configured number of consecutive evaluation periods. Here's an alarm that fires when average CPU utilization stays above 80% for three consecutive 5-minute periods (15 minutes sustained):

aws cloudwatch put-metric-alarm \
  --alarm-name high-cpu-web-server \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
  --statistic Average \
  --period 300 \
  --evaluation-periods 3 \
  --threshold 80 \
  --comparison-operator GreaterThanThreshold \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alerts

--evaluation-periods 3 is deliberately not 1 — a single 5-minute spike is normal and expected; three consecutive periods above threshold is a much stronger signal of a genuine, sustained problem rather than noise. --alarm-actions points at an SNS (Simple Notification Service) topic, which is the usual next hop — SNS can fan that single alarm out to an email, a Slack webhook, a paging system like PagerDuty, or even trigger an Auto Scaling action directly.

Log groups#

Application and infrastructure logs (an EC2 instance's application logs via the CloudWatch agent, a Lambda function's console.log output, an ECS container's stdout) are collected into log groups, each holding many log streams:

Log Group: /aws/lambda/generate-thumbnail
 ├── Log Stream: 2026/08/25/[$LATEST]a1b2c3d4...
 ├── Log Stream: 2026/08/25/[$LATEST]e5f6g7h8...
 └── Log Stream: 2026/08/25/[$LATEST]i9j0k1l2...

Lambda creates and writes to a log group automatically for every function with zero setup — every console.log in the thumbnail function from the previous page lands in /aws/lambda/generate-thumbnail without any extra configuration. For EC2, the CloudWatch agent has to be installed and configured to ship a chosen set of log files off the instance:

aws logs create-log-group --log-group-name /myapp/web-server
aws logs put-retention-policy \
  --log-group-name /myapp/web-server \
  --retention-in-days 30

Searching logs directly from the CLI, filtering for errors in the last hour:

aws logs filter-log-events \
  --log-group-name /aws/lambda/generate-thumbnail \
  --filter-pattern "ERROR" \
  --start-time $(date -d '1 hour ago' +%s000)

put-retention-policy deserves a specific callout: log groups have no expiration by default — logs are retained forever until you explicitly set a retention period, which quietly accumulates storage cost for logs nobody will ever read again.

Common mistakes#

• Never setting a log retention policy — CloudWatch Logs defaults to keeping everything indefinitely, and forgotten log groups from long-decommissioned services are a genuinely common, easy-to-miss source of ongoing cost.
• Setting an alarm's evaluation-periods to 1 for a naturally noisy metric — a single brief spike triggers a page for something that resolved itself before anyone could act on it, training the team to ignore alerts.
• Relying only on infrastructure metrics (CPU, memory) and never publishing application-level custom metrics — a server can look perfectly healthy by every infrastructure metric while the application it's running is actively failing every request.
• Creating an alarm with no --alarm-actions at all — the alarm state changes correctly, but nobody is ever notified unless someone happens to check the CloudWatch console.