API Gateway & Resilience

Spring Cloud Gateway#

External clients shouldn't need to know about (or call) every individual internal service directly — that couples clients to internal topology, and makes cross-cutting concerns (authentication, rate limiting, logging) something every service has to implement separately. Spring Cloud Gateway sits in front of the whole system as a single entry point and routes incoming requests to the right internal service.

spring:
  cloud:
    gateway:
      routes:
        - id: orders-route
          uri: lb://orders-service   # "lb://" -- resolve via the service registry, load-balanced across instances
          predicates:
            - Path=/api/orders/**
        - id: payments-route
          uri: lb://payments-service
          predicates:
            - Path=/api/payments/**

Every request to /api/orders/** is routed to a healthy instance of orders-service, resolved dynamically through service discovery (lb://) rather than a fixed address — tying directly back to the discovery mechanism from the previous page. Filters can be attached per route for cross-cutting concerns without touching the downstream services themselves:

        - id: orders-route
          uri: lb://orders-service
          predicates:
            - Path=/api/orders/**
          filters:
            - StripPrefix=1
            - AddRequestHeader=X-Gateway-Source, api-gateway

Resilience4j: circuit breakers and retries#

A gateway routes traffic correctly, but it doesn't protect a caller from a downstream service that's slow or failing — that's what Resilience4j provides, implementing the circuit breaker pattern (see the Microservices vs Monolith tutorial in the System Design track for the concept itself) as a set of annotations that wrap a method call:

@Service
public class OrdersClient {

    private final RestClient restClient;

    public OrdersClient(RestClient.Builder builder) {
        this.restClient = builder.baseUrl("http://orders-service").build();
    }

    @CircuitBreaker(name = "ordersService", fallbackMethod = "fallbackOrders")
    @Retry(name = "ordersService")
    public List<Order> getOrdersForCustomer(Long customerId) {
        return restClient.get()
            .uri("/api/orders?customerId={id}", customerId)
            .retrieve()
            .body(new ParameterizedTypeReference<List<Order>>() {});
    }

    // signature must match the original method, plus a Throwable parameter
    private List<Order> fallbackOrders(Long customerId, Throwable throwable) {
        return List.of(); // degrade gracefully instead of propagating the failure to the caller
    }
}

resilience4j:
  circuitbreaker:
    instances:
      ordersService:
        sliding-window-size: 10
        failure-rate-threshold: 50       # trip open once 50% of the last 10 calls failed
        wait-duration-in-open-state: 10s # stay open (fail fast) for 10s before testing recovery
  retry:
    instances:
      ordersService:
        max-attempts: 3
        wait-duration: 500ms

When the failure rate crosses the configured threshold, the circuit breaker trips open — further calls fail immediately via fallbackOrders, without even attempting the network call, for the configured cooldown period. After that, it moves to half-open and lets a limited number of test calls through; if those succeed, the circuit closes again and normal traffic resumes.

CLOSED (normal) --[failure rate > threshold]--> OPEN (fail fast, fallback runs)
   ^                                                     |
   |                                          [wait-duration-in-open-state elapses]
   |                                                     v
   +---------[test calls succeed]----------- HALF-OPEN (try a few real calls)

@Retry and @CircuitBreaker are often combined, but deliberately: retries handle brief, transient failures (a single dropped packet), while the circuit breaker handles sustained failure by refusing to keep trying at all — combining them without a circuit breaker risks a "retry storm," where every caller's retries pile additional load onto an already-struggling service.

Common mistakes#

• Configuring @Retry without a @CircuitBreaker alongside it on a call to a genuinely struggling service — retries alone can make an outage worse by adding load instead of backing off.
• Setting the circuit breaker's failure-rate threshold too low, tripping open on ordinary, brief blips instead of sustained failure.
• Forgetting a fallback method (or giving it a mismatched signature) — @CircuitBreaker requires the fallback's parameters to match the original method's, plus a trailing Throwable, and a mismatch fails at runtime.
• Routing every request through the gateway but skipping authentication/rate-limiting there, then re-implementing those cross-cutting concerns separately inside each downstream service anyway.