Show List
Tutorials > Microservices > Resiliency Patterns

Resiliency Patterns

In a microservices architecture, when one Spring Boot microservice calls another, it's crucial to implement resiliency patterns to handle potential failures gracefully and improve the overall reliability of the system. Here are several resiliency patterns you can use and examples of how to implement them in Spring Boot:

Retry Pattern

The Retry Pattern is a resiliency pattern that involves making repeated attempts to perform an operation that may fail before giving up and declaring it as a failure. This pattern can be useful in handling transient failures, such as network issues or temporary unavailability of a service. In Spring Boot, you can implement the Retry Pattern using the Spring Retry library. Below, I'll explain the Retry Pattern and provide a code example using Spring Retry.

  1. Spring Retry Dependency:

    To use the Spring Retry library, you need to include it as a dependency in your Spring Boot project. Add the following dependency to your pom.xml or build.gradle file:

    Maven:


    <dependency>
    <groupId>org.springframework.retry</groupId>
    <artifactId>spring-retry</artifactId>
</dependency>

    Gradle:


    implementation 'org.springframework.retry:spring-retry'

    Code Example:

    Let's create a simple Spring Boot application that demonstrates the Retry Pattern. In this example, we'll create a service method that simulates making an unreliable network call and retry it in case of failure.


    import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.retry.annotation.EnableRetry;
import org.springframework.retry.annotation.Retryable;
import org.springframework.stereotype.Service;

@SpringBootApplication
@EnableRetry
public class RetryPatternExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(RetryPatternExampleApplication.class, args);
    }
}

@Service
class MyService {
    private int callCounter = 0;

    @Retryable(maxAttempts = 3, value = {RuntimeException.class})
    public void makeUnreliableNetworkCall() {
        callCounter++;
        System.out.println("Attempting network call, attempt " + callCounter);
        if (callCounter < 3) {
            throw new RuntimeException("Network call failed");
        } else {
            System.out.println("Network call succeeded.");
        }
    }
}

    In this example:

    1. We enable the Spring Retry framework by annotating the Spring Boot application class with @EnableRetry.

    2. We create a MyService class with a method makeUnreliableNetworkCall. We annotate this method with @Retryable. The @Retryable annotation specifies that the method can be retried in case of a RuntimeException up to three times (maxAttempts = 3).

    3. Inside the makeUnreliableNetworkCall method, we simulate an unreliable network call. If the call fails (as determined by the callCounter variable), we throw a RuntimeException. The Retry Pattern will automatically retry this method up to three times before giving up.

    When you run this Spring Boot application, you'll see the output that shows the method being retried up to three times:


    Attempting network call, attempt 1
Attempting network call, attempt 2
Network call succeeded.

    After the third attempt, the network call succeeds, and the method completes without an exception.

    The Retry Pattern helps improve the reliability of your application by handling transient failures gracefully. You can customize the number of retry attempts, the types of exceptions to retry on, and other parameters to fit your specific use case.

    Circuit Breaker Pattern

    The Circuit Breaker Pattern is a resiliency pattern used in software development to handle faults and failures gracefully, especially in distributed systems. It prevents continuous attempts to access a service that is likely to fail or respond very slowly, which could lead to performance degradation and resource exhaustion. Instead, the pattern allows the system to "open the circuit" and stop trying to access the problematic service temporarily, returning predefined fallback responses when needed.

    The Circuit Breaker Pattern is often implemented with a state machine that transitions between three states:

    1. Closed: In the closed state, the circuit breaker allows service requests to pass through. The system monitors the service for failures, and if a predefined failure threshold is reached, it transitions to the open state.

    2. Open: In the open state, the circuit breaker prevents service requests from being executed, and predefined fallback responses are returned immediately. The circuit breaker periodically allows a limited number of requests to pass through to check if the service has recovered.

    3. Half-Open: After some time in the open state, the circuit breaker transitions to the half-open state, allowing a limited number of test requests to pass through. If these test requests succeed, the circuit breaker transitions back to the closed state. If they fail, it remains in the open state.

    Here's a code example of implementing the Circuit Breaker Pattern in Spring Boot using the Hystrix library:

    Hystrix Dependency:

    To use Hystrix in a Spring Boot project, you need to include the Hystrix dependency:

    Maven:


    <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>

    Gradle:


    implementation 'org.springframework.cloud:spring-cloud-starter-netflix-hystrix'

    Code Example:

    In this example, we'll create a simple Spring Boot service that simulates making unreliable network calls using Hystrix for the Circuit Breaker Pattern.


    import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.hystrix.HystrixCommands;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
@EnableCircuitBreaker
public class CircuitBreakerExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(CircuitBreakerExampleApplication.class, args);
    }
}

@RestController
class MyController {

    @GetMapping("/callUnreliableService")
    public String callUnreliableService() {
        return HystrixCommands
            .from(MyHystrixCommand.class)
            .execute();
    }
}

class MyHystrixCommand extends HystrixCommands.SimpleCommand<String> {

    protected MyHystrixCommand() {
        super(MyHystrixCommand.class);
    }

    @Override
    protected String run() {
        // Simulate an unreliable network call
        if (Math.random() < 0.8) {
            throw new RuntimeException("Network call failed");
        } else {
            return "Network call succeeded";
        }
    }

    @Override
    protected String getFallback() {
        return "Fallback response";
    }
}

    In this example:

    • We enable the Circuit Breaker Pattern by annotating the Spring Boot application class with @EnableCircuitBreaker.

    • We create a simple REST endpoint /callUnreliableService that invokes a method protected by a Hystrix command.

    • The MyHystrixCommand class extends HystrixCommands.SimpleCommand<String>. It simulates an unreliable network call in the run method. If the network call fails (as determined by Math.random()), a RuntimeException is thrown, which triggers the circuit breaker to open.

    • The getFallback method specifies the fallback response to be returned when the circuit is open.

    When you run this Spring Boot application, you can access the /callUnreliableService endpoint, and you'll observe that it returns the fallback response when the circuit breaker is open.

    The Circuit Breaker Pattern, implemented using Hystrix or similar libraries, helps improve the resilience and reliability of your microservices by preventing continuous attempts to access failing services and providing fallback responses when necessary.

    Timeouts Pattern

    The Timeouts Pattern is a resiliency pattern used in software development to handle network calls or operations that take longer than expected to complete. It helps prevent a system from waiting indefinitely for a response and allows it to proceed with predefined fallback actions when the expected response time is exceeded. The Timeout Pattern is essential for ensuring that your application remains responsive and resilient in the face of slow or unresponsive services.

    Here's a code example of implementing the Timeout Pattern in Spring Boot:

    Code Example:

    In this example, we'll create a simple Spring Boot service that simulates making a network call that might take longer to respond. We'll use Java's CompletableFuture and ExecutorService to set a timeout for the network call.


    import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

@SpringBootApplication
public class TimeoutPatternExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(TimeoutPatternExampleApplication.class, args);
    }
}

@RestController
class MyController {
    private final ExecutorService executor = Executors.newFixedThreadPool(3);

    @GetMapping("/makeNetworkCall")
    public ResponseEntity<String> makeNetworkCall() {
        CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
            // Simulate a long-running network call
            try {
                TimeUnit.SECONDS.sleep(5);
                return "Network call response";
            } catch (InterruptedException e) {
                // Handle the interruption
                return "Network call interrupted";
            }
        }, executor);

        try {
            // Wait for the network call to complete with a timeout
            String result = future.get(3, TimeUnit.SECONDS);
            return ResponseEntity.ok(result);
        } catch (Exception e) {
            // Handle the timeout or other exceptions
            return ResponseEntity.ok("Network call timed out");
        }
    }
}

    In this example:

    • We create a Spring Boot application and a REST endpoint /makeNetworkCall that simulates a network call that might take up to 5 seconds to complete.

    • Inside the endpoint method, we use CompletableFuture to execute the network call asynchronously in a separate thread using an ExecutorService. This allows us to set a timeout for the network call.

    • We wait for the completion of the network call using future.get(3, TimeUnit.SECONDS), specifying a timeout of 3 seconds. If the network call takes longer than 3 seconds, it will throw a TimeoutException.

    • We catch the TimeoutException or other exceptions and return an appropriate response to indicate a timeout or other failure.

    This example demonstrates how to implement the Timeout Pattern by setting a maximum response time for a network call and handling timeouts gracefully. You can adjust the timeout value as needed to suit your specific requirements.

    Bulkhead Pattern

    The Bulkhead Pattern is a resiliency pattern used in software development to isolate different parts of a system or service to prevent faults or failures in one part from affecting other parts. The name "bulkhead" is inspired by the compartments in a ship, which are designed to keep water from flooding the entire vessel if one compartment is breached. In a software context, this pattern helps improve system reliability by limiting the impact of failures to specific sections of the system.

    The Bulkhead Pattern is commonly used for limiting resource consumption and avoiding resource exhaustion. For example, in a microservices architecture, you can isolate the thread pool used for external network calls to prevent a misbehaving service from consuming all the available threads and causing thread starvation in other parts of the system.

    Here's a code example of implementing the Bulkhead Pattern in Spring Boot using Hystrix:

    Hystrix Dependency:

    To use Hystrix in a Spring Boot project, you need to include the Hystrix dependency:

    Maven:


    <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>

    Gradle:


    implementation 'org.springframework.cloud:spring-cloud-starter-netflix-hystrix'

    Code Example:

    In this example, we'll create a simple Spring Boot service that simulates making network calls using Hystrix for implementing the Bulkhead Pattern. We'll isolate the thread pool used for these network calls.


    import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.hystrix.HystrixThreadPoolProperties;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
@EnableCircuitBreaker
public class BulkheadPatternExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(BulkheadPatternExampleApplication.class, args);
    }
}

@RestController
class MyController {

    @GetMapping("/callUnreliableService1")
    public String callUnreliableService1() {
        return performNetworkCall("Service 1");
    }

    @GetMapping("/callUnreliableService2")
    public String callUnreliableService2() {
        return performNetworkCall("Service 2");
    }

    private String performNetworkCall(String serviceName) {
        return "Network call to " + serviceName + " succeeded";
    }
}

    In this example:

    • We enable the Bulkhead Pattern by annotating the Spring Boot application class with @EnableCircuitBreaker.

    • We create two REST endpoints, /callUnreliableService1 and /callUnreliableService2, each simulating a network call. In a real application, these endpoints could be making actual network calls to external services.

    • The key part of implementing the Bulkhead Pattern is configuring separate thread pools for these endpoints. By default, Hystrix uses a global thread pool for all commands. However, we can configure separate thread pools for different parts of the system using properties like HystrixThreadPoolProperties.

    By isolating the thread pools for different services or operations, the Bulkhead Pattern prevents one service or operation from affecting the performance and resource availability of others. If one part of the system is heavily loaded or misbehaving, it doesn't impact the performance of the rest of the system. This separation and isolation help ensure that the overall system remains reliable and responsive.

    Fallback pattern

    The Fallback Pattern is a resiliency pattern used in software development to provide alternative behaviors or responses when an operation or service encounters a failure or cannot fulfill a request. This pattern is particularly important in distributed systems, where services may become unavailable or experience issues. When a service or operation fails, the fallback pattern allows the system to gracefully handle the failure by providing a predefined fallback response, reducing the impact on the user or downstream services.

    Here's a code example of implementing the Fallback Pattern in a Spring Boot application:

    Code Example:

    In this example, we'll create a simple Spring Boot service that simulates making a network call. If the network call fails, we'll provide a fallback response.


    import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
public class FallbackPatternExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(FallbackPatternExampleApplication.class, args);
    }
}

@RestController
class MyController {

    @GetMapping("/callUnreliableService")
    public String callUnreliableService() {								
        try {								
            // Simulate a network call that may fail								
            if (Math.random() < 0.8) {								
                throw new RuntimeException("Network call failed");								
            }								
            return "Network call succeeded";								
        } catch (Exception e) {								
            // Handle the exception and provide a fallback response								
            return "Fallback response: " + e.getMessage();								
        }								
    }								
}

    In this example:

    • We create a Spring Boot application with a single REST endpoint /callUnreliableService.

    • Inside the endpoint method we simulate a network call that may fail by using Math.random() to generate random failures. If the network call fails and an exception is thrown we catch the exception and provide a predefined fallback response. The fallback response is a message that includes the error message from the exception.

    • If the network call succeeds the result is returned as "Network call succeeded."

    The Fallback Pattern is an essential pattern for ensuring that your application can gracefully handle failures and provide a reasonable response when things go wrong. In a real-world scenario the fallback response can be customized based on the specific failure scenario and the requirements of your application. This pattern helps improve the resilience and user experience of your system.

    Rate Limiting pattern

    The Rate Limiting Pattern is a resiliency pattern used in software development to control the rate at which certain operations or requests are processed or served. It is commonly used to protect services from being overwhelmed by too many requests which can lead to performance degradation or service disruption. Rate limiting helps ensure that services remain responsive and available even during periods of high demand.

    Here's a code example of implementing the Rate Limiting Pattern in a Spring Boot application:

    Code Example:

    In this example we'll create a simple Spring Boot service with rate limiting applied to an endpoint using Spring Cloud Gateway a common component for building API gateways and handling rate limiting.

    Step 1: Create a Spring Boot Application with Spring Cloud Gateway


    import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;								
								
@SpringBootApplication								
public class RateLimitingPatternExampleApplication {								
								
    public static void main(String[] args) {								
        SpringApplication.run(RateLimitingPatternExampleApplication.class	 args);							
    }								
}

    Step 2: Configure Rate Limiting in application.properties or application.yml

    In your application properties (application.properties or application.yml) you can configure rate limiting rules for specific routes using Spring Cloud Gateway properties:


    spring:							
  cloud:								
    gateway:								
      routes:								
        - id: rate_limit_route								
          uri: http://example.com  # Replace with your target service URL								
          predicates:								
            - Path=/api/some-endpoint								
          filters:								
            - name: RequestRateLimiter								
              args:								
                redis-rate-limiter.replenishRate: 1								
                redis-rate-limiter.burstCapacity: 3

    In this configuration:

    • id: The unique identifier for the route.

    • uri: The target service URL.

    • predicates: Conditions to match the route.

    • filters: Filters to apply to the route including the RequestRateLimiter filter.

    • redis-rate-limiter.replenishRate: The rate at which tokens are replenished. In this example 1 token is replenished every second.

    • redis-rate-limiter.burstCapacity: The maximum number of tokens that can be available at any time. In this example the maximum burst capacity is 3 tokens.

    Step 3: Create a REST Endpoint

    The endpoint you want to apply rate limiting to should be configured in your gateway properties as shown in the configuration. In this example the rate limiting is applied to requests matching the /api/some-endpoint path. You can create the corresponding endpoint in your downstream service.

    Step 4: Run and Test the Application

    Start the Spring Boot application and send requests to the rate-limited endpoint. The rate limiting filter will control the rate of incoming requests according to the configured rules.

    This example demonstrates how to implement the Rate Limiting Pattern using Spring Cloud Gateway to control the rate of incoming requests to a specific endpoint. Rate limiting can be customized to fit your application's requirements and it helps protect your services from overloading and ensures fair resource allocation during high demand periods.



Next: Quiz - Microservices - 1

    Leave a Comment

  • captcha text