AI 接口调用超时拖垮整个服务——超时+熔断+降级三件套 - 服务端开发博客

一、引言

凌晨三点，监控告警突然响起：服务响应时间从 200ms 飙升到 30s，线程池满了，所有接口都超时了。

排查发现，问题出在一个调用大模型 API 的接口上——平时响应时间 2s，那天因为服务商限流排队，响应时间直接飙到 60s。由于没有设置超时，所有请求都卡在等待 AI 响应上，线程池很快耗尽，整个服务瘫痪。

今天我要分享一套完整的解决方案：超时控制 + 熔断保护 + 降级策略，确保外部 API 故障不会拖垮你的服务。

二、问题分析

2.1 故障链路

┌─────────────────────────────────────────────────────────────────────┐
│                        故障传播链路                                  │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  AI API 服务商限流                                                   │
│         ↓                                                          │
│  AI API 响应时间 2s → 60s                                           │
│         ↓                                                          │
│  调用方没有设置超时                                                  │
│         ↓                                                          │
│  请求线程被阻塞，等待响应                                             │
│         ↓                                                          │
│  线程池队列填满                                                      │
│         ↓                                                          │
│  新请求被拒绝（线程池耗尽）                                           │
│         ↓                                                          │
│  整个服务不可用                                                      │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

2.2 根因总结

问题	影响
未设置超时	请求无限等待，线程被永久占用
未设置熔断	故障持续时，所有请求都去调用故障 API
未设置降级	没有兜底方案，服务完全不可用
线程池配置不合理	少量慢请求就耗尽线程

三、技术方案

3.1 技术栈

组件	版本	作用
Java	21	运行环境
Spring Boot	3.2.x	应用框架
Spring RestClient	6.1+	HTTP 客户端（带超时控制）
Resilience4j	2.0+	熔断、限流、降级
Redis	7.x	缓存降级数据

3.2 整体架构

┌─────────────────────────────────────────────────────────────────────┐
│                        完整架构                                     │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  用户请求                                                            │
│     ↓                                                               │
│  ┌──────────────┐                                                   │
│  │ 线程池控制    │ ← 防止线程耗尽                                     │
│  └──────┬───────┘                                                   │
│         ↓                                                           │
│  ┌──────────────┐                                                   │
│  │  RestClient  │ ← 超时控制（connect=5s, read=30s）                  │
│  └──────┬───────┘                                                   │
│         ↓                                                           │
│  ┌──────────────┐                                                   │
│  │ 熔断器        │ ← Resilience4j（失败率>50%熔断）                   │
│  └──────┬───────┘                                                   │
│         ↓                                                           │
│  ┌──────────────┐                                                   │
│  │ 降级策略      │ ← 缓存答案 / 友好提示                              │
│  └──────────────┘                                                   │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

四、项目初始化

4.1 pom.xml 依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
         http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>ai-timeout-circuitbreaker</artifactId>
    <version>1.0.0</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.5</version>
        <relativePath/>
    </parent>

    <properties>
        <java.version>21</java.version>
        <resilience4j.version>2.2.0</resilience4j.version>
    </properties>

    <dependencies>
        <!-- Spring Boot Web -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- Spring Boot Data Redis -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>

        <!-- Resilience4j Circuit Breaker -->
        <dependency>
            <groupId>io.github.resilience4j</groupId>
            <artifactId>resilience4j-spring-boot3</artifactId>
            <version>${resilience4j.version}</version>
        </dependency>

        <!-- Resilience4j Annotation Support -->
        <dependency>
            <groupId>io.github.resilience4j</groupId>
            <artifactId>resilience4j-annotations</artifactId>
            <version>${resilience4j.version}</version>
        </dependency>

        <!-- Lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <!-- Test -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

4.2 application.yml 配置

server:
  port: 8080

spring:
  application:
    name: ai-timeout-circuitbreaker

  # Redis 配置
  data:
    redis:
      host: localhost
      port: 6379
      timeout: 5000ms

# 线程池配置
app:
  thread-pool:
    core-size: 10
    max-size: 20
    queue-capacity: 100
    keep-alive-seconds: 60

# Resilience4j 配置
resilience4j:
  circuitbreaker:
    instances:
      aiApiCircuitBreaker:
        # 失败率阈值，超过此值熔断器打开
        failure-rate-threshold: 50
        # 慢调用率阈值，超过此值熔断器打开
        slow-call-rate-threshold: 50
        # 慢调用判定阈值，超过此时间视为慢调用
        slow-call-duration-threshold: 10s
        # 熔断器打开后，等待多久进入半开状态
        wait-duration-in-open-state: 30s
        # 半开状态下，允许通过的请求数
        permitted-number-of-calls-in-half-open-state: 5
        # 滑动窗口大小（用于统计失败率）
        sliding-window-size: 100
        # 滑动窗口类型：COUNT_BASED 或 TIME_BASED
        sliding-window-type: COUNT_BASED
        # 记录所有异常，还是只记录业务异常
        record-exceptions:
          - java.util.concurrent.TimeoutException
          - java.net.SocketTimeoutException
          - org.springframework.web.client.RestClientException

# AI API 配置
ai:
  api:
    base-url: https://api.openai.com/v1
    api-key: ${OPENAI_API_KEY}
    timeout:
      connect: 5s
      read: 30s

五、线程池配置

5.1 线程池配置类

package com.example.ai.config;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;
import java.util.concurrent.ThreadPoolExecutor;

@Configuration
@EnableAsync
public class ThreadPoolConfig {

    @Value("${app.thread-pool.core-size:10}")
    private int coreSize;

    @Value("${app.thread-pool.max-size:20}")
    private int maxSize;

    @Value("${app.thread-pool.queue-capacity:100}")
    private int queueCapacity;

    @Value("${app.thread-pool.keep-alive-seconds:60}")
    private int keepAliveSeconds;

    @Bean(name = "aiApiExecutor")
    public Executor aiApiExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(coreSize);
        executor.setMaxPoolSize(maxSize);
        executor.setQueueCapacity(queueCapacity);
        executor.setKeepAliveSeconds(keepAliveSeconds);
        executor.setThreadNamePrefix("ai-api-");
        
        // 拒绝策略：由调用线程执行（背压机制）
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        
        executor.initialize();
        return executor;
    }
}

5.2 线程池参数说明

参数	配置值	说明
core-size	10	核心线程数，始终保持存活
max-size	20	最大线程数，队列满时创建
queue-capacity	100	队列容量，超过后创建新线程
keep-alive-seconds	60	非核心线程空闲存活时间
rejected-handler	CallerRunsPolicy	队列和线程池都满时，由调用线程执行

💡 关键设计：使用 CallerRunsPolicy 拒绝策略，当线程池和队列都满时，新请求会在调用线程中执行，形成天然的背压机制，防止请求丢失。

六、RestClient 超时配置

6.1 RestClient 配置类

package com.example.ai.config;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.web.client.RestClientCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.client.RestClient;

import java.time.Duration;

@Configuration
public class RestClientConfig {

    @Value("${ai.api.timeout.connect:5s}")
    private Duration connectTimeout;

    @Value("${ai.api.timeout.read:30s}")
    private Duration readTimeout;

    @Value("${ai.api.base-url}")
    private String aiApiBaseUrl;

    @Value("${ai.api.api-key}")
    private String aiApiKey;

    @Bean
    public RestClientCustomizer restClientCustomizer() {
        return restClientBuilder -> restClientBuilder
            .requestFactory(() -> {
                SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
                factory.setConnectTimeout((int) connectTimeout.toMillis());
                factory.setReadTimeout((int) readTimeout.toMillis());
                return factory;
            });
    }

    @Bean("aiApiRestClient")
    public RestClient aiApiRestClient(RestClient.Builder builder) {
        return builder
            .baseUrl(aiApiBaseUrl)
            .defaultHeader("Authorization", "Bearer " + aiApiKey)
            .defaultHeader("Content-Type", "application/json")
            .build();
    }
}

6.2 超时配置说明

参数	配置值	说明
connectTimeout	5s	连接建立超时，超过则抛出 ConnectTimeoutException
readTimeout	30s	读取响应超时，超过则抛出 SocketTimeoutException

⚠️ 重要提示：readTimeout 应根据业务场景设置，AI API 通常响应较慢，建议设置为 30-60s，但不要设置过长（如 60s+），否则线程池仍可能被耗尽。

七、Resilience4j 熔断配置

7.1 熔断机制原理

熔断器状态转换：
┌─────────────────────────────────────────────────────────────┐
│                                                             │
│   关闭状态（Closed）                                         │
│   ↓ 失败率 > 阈值                                            │
│   打开状态（Open）                                          │
│   ↓ waitDurationInOpenState 时间过去                        │
│   半开状态（Half-Open）                                      │
│   ↓ 测试请求成功                                             │
│   关闭状态（Closed）                                         │
│   ↓ 测试请求失败                                             │
│   打开状态（Open）                                          │
│                                                             │
└─────────────────────────────────────────────────────────────┘

7.2 熔断配置详解

resilience4j:
  circuitbreaker:
    instances:
      aiApiCircuitBreaker:
        # 失败率阈值：超过 50% 时打开熔断器
        failure-rate-threshold: 50
        
        # 慢调用率阈值：超过 50% 时打开熔断器
        slow-call-rate-threshold: 50
        
        # 慢调用判定：超过 10s 视为慢调用
        slow-call-duration-threshold: 10s
        
        # 打开状态持续时间：30s 后进入半开状态
        wait-duration-in-open-state: 30s
        
        # 半开状态允许的请求数：5 个
        permitted-number-of-calls-in-half-open-state: 5
        
        # 滑动窗口大小：最近 100 个请求
        sliding-window-size: 100
        
        # 滑动窗口类型：基于计数
        sliding-window-type: COUNT_BASED
        
        # 记录的异常类型
        record-exceptions:
          - java.util.concurrent.TimeoutException
          - java.net.SocketTimeoutException
          - org.springframework.web.client.RestClientException

八、降级策略实现

8.1 AI API 服务

package com.example.ai.service;

import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

import java.time.Duration;
import java.util.Map;

@Slf4j
@Service
public class AIService {

    private final RestClient aiApiRestClient;
    private final StringRedisTemplate redisTemplate;

    public AIService(@Qualifier("aiApiRestClient") RestClient aiApiRestClient,
                     StringRedisTemplate redisTemplate) {
        this.aiApiRestClient = aiApiRestClient;
        this.redisTemplate = redisTemplate;
    }

    @CircuitBreaker(name = "aiApiCircuitBreaker", fallbackMethod = "fallback")
    public String askAI(String question) {
        log.info("Calling AI API for question: {}", question);

        // 构建请求体
        Map<String, Object> request = Map.of(
            "model", "gpt-4o",
            "messages", List.of(Map.of(
                "role", "user",
                "content", question
            )),
            "max_tokens", 1000
        );

        // 调用 AI API
        String response = aiApiRestClient.post()
            .uri("/chat/completions")
            .body(request)
            .retrieve()
            .body(String.class);

        log.info("AI API response received");
        return response;
    }

    /**
     * 降级方法：当熔断器打开或调用失败时执行
     */
    public String fallback(String question, Throwable throwable) {
        log.warn("AI API call failed, executing fallback. Error: {}", throwable.getMessage());

        // 策略一：尝试从缓存获取答案
        String cachedAnswer = redisTemplate.opsForValue().get("ai:cache:" + question);
        if (cachedAnswer != null) {
            log.info("Returning cached answer for question: {}", question);
            return cachedAnswer;
        }

        // 策略二：返回友好提示
        return buildFriendlyResponse(question, throwable);
    }

    /**
     * 构建友好的降级响应
     */
    private String buildFriendlyResponse(String question, Throwable throwable) {
        String errorType = getErrorType(throwable);
        
        return String.format("""
            {
                "status": "degraded",
                "error_type": "%s",
                "message": "AI 服务暂时繁忙，请稍后再试。",
                "question": "%s",
                "suggestion": "您可以尝试简化问题，或者等待几分钟后重试。"
            }
            """, errorType, question);
    }

    /**
     * 判断错误类型
     */
    private String getErrorType(Throwable throwable) {
        if (throwable instanceof java.util.concurrent.TimeoutException) {
            return "timeout";
        } else if (throwable instanceof java.net.SocketTimeoutException) {
            return "socket_timeout";
        } else if (throwable instanceof org.springframework.web.client.RestClientException) {
            return "http_error";
        } else {
            return "circuit_breaker_open";
        }
    }

    /**
     * 缓存 AI 答案
     */
    public void cacheAnswer(String question, String answer, Duration ttl) {
        redisTemplate.opsForValue().set("ai:cache:" + question, answer, ttl);
    }
}

8.2 缓存降级策略

package com.example.ai.service;

import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;

import java.time.Duration;

@Slf4j
@Service
public class AICacheService {

    private final AIService aiService;

    public AICacheService(AIService aiService) {
        this.aiService = aiService;
    }

    /**
     * 带缓存的 AI 查询
     * 优先从缓存获取，缓存不存在时调用 AI API
     */
    public String askWithCache(String question) {
        String result = aiService.askAI(question);
        
        // 如果结果不是降级响应，缓存答案
        if (!result.contains("\"status\": \"degraded\"")) {
            aiService.cacheAnswer(question, result, Duration.ofHours(1));
        }
        
        return result;
    }
}

九、异步调用与线程池隔离

9.1 异步服务

package com.example.ai.service;

import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;

import java.util.concurrent.CompletableFuture;

@Slf4j
@Service
public class AsyncAIService {

    private final AICacheService aiCacheService;

    public AsyncAIService(AICacheService aiCacheService) {
        this.aiCacheService = aiCacheService;
    }

    @Async("aiApiExecutor")
    public CompletableFuture<String> askAsync(String question) {
        log.info("Async AI call started for question: {}", question);
        String result = aiCacheService.askWithCache(question);
        log.info("Async AI call completed for question: {}", question);
        return CompletableFuture.completedFuture(result);
    }
}

9.2 控制器

package com.example.ai.controller;

import lombok.extern.slf4j.Slf4j;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import com.example.ai.service.AsyncAIService;
import com.example.ai.service.AICacheService;

import java.util.Map;
import java.util.concurrent.CompletableFuture;

@Slf4j
@RestController
@RequestMapping("/api/ai")
public class AIController {

    private final AICacheService aiCacheService;
    private final AsyncAIService asyncAIService;

    public AIController(AICacheService aiCacheService, AsyncAIService asyncAIService) {
        this.aiCacheService = aiCacheService;
        this.asyncAIService = asyncAIService;
    }

    @PostMapping("/ask")
    public ResponseEntity<Map<String, Object>> ask(@RequestBody Map<String, String> request) {
        String question = request.get("question");
        
        log.info("Received AI request: {}", question);
        
        String result = aiCacheService.askWithCache(question);
        
        return ResponseEntity.ok(Map.of(
            "question", question,
            "answer", result,
            "timestamp", System.currentTimeMillis()
        ));
    }

    @PostMapping("/ask-async")
    public CompletableFuture<ResponseEntity<Map<String, Object>>> askAsync(
            @RequestBody Map<String, String> request) {
        String question = request.get("question");
        
        log.info("Received async AI request: {}", question);
        
        return asyncAIService.askAsync(question)
            .thenApply(result -> ResponseEntity.ok(Map.of(
                "question", question,
                "answer", result,
                "timestamp", System.currentTimeMillis()
            )));
    }
}

十、请求流程图

10.1 正常流程

用户请求 → Controller → AICacheService → AIService.askAI()
                                             ↓
                                    RestClient 调用 AI API
                                             ↓
                                    返回正常响应
                                             ↓
                                    缓存结果（可选）
                                             ↓
                                    返回给用户

10.2 超时流程

用户请求 → Controller → AICacheService → AIService.askAI()
                                             ↓
                                    RestClient 调用 AI API
                                             ↓
                                    超过 readTimeout（30s）
                                             ↓
                                    抛出 SocketTimeoutException
                                             ↓
                                    熔断器记录失败
                                             ↓
                                    fallback() 执行
                                             ↓
                                    返回缓存答案或友好提示

10.3 熔断流程

用户请求 → Controller → AICacheService → AIService.askAI()
                                             ↓
                                    熔断器状态为 OPEN
                                             ↓
                                    直接执行 fallback()
                                             ↓
                                    返回缓存答案或友好提示
                                             ↓
                                    waitDurationInOpenState（30s）后
                                             ↓
                                    进入 HALF_OPEN 状态
                                             ↓
                                    允许少量测试请求

十一、监控与告警

11.1 熔断器监控

package com.example.ai.controller;

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.HashMap;
import java.util.Map;

@Slf4j
@RestController
@RequestMapping("/api/monitor")
public class MonitorController {

    private final CircuitBreakerRegistry circuitBreakerRegistry;

    public MonitorController(CircuitBreakerRegistry circuitBreakerRegistry) {
        this.circuitBreakerRegistry = circuitBreakerRegistry;
    }

    @GetMapping("/circuit-breaker")
    public ResponseEntity<Map<String, Object>> getCircuitBreakerStatus() {
        CircuitBreaker circuitBreaker = circuitBreakerRegistry
            .circuitBreaker("aiApiCircuitBreaker");

        Map<String, Object> status = new HashMap<>();
        status.put("name", "aiApiCircuitBreaker");
        status.put("state", circuitBreaker.getState().name());
        status.put("failureRate", circuitBreaker.getMetrics().getFailureRate());
        status.put("slowCallRate", circuitBreaker.getMetrics().getSlowCallRate());
        status.put("bufferedCalls", circuitBreaker.getMetrics().getNumberOfBufferedCalls());
        status.put("failedCalls", circuitBreaker.getMetrics().getNumberOfFailedCalls());
        status.put("slowCalls", circuitBreaker.getMetrics().getNumberOfSlowCalls());

        return ResponseEntity.ok(status);
    }
}

11.2 告警配置建议

告警规则：
┌─────────────────────────────────────────────────────┐
│ 指标                    │ 阈值          │ 动作        │
├─────────────────────────────────────────────────────┤
│ 熔断器状态              │ OPEN          │ 发送告警     │
│ 失败率                  │ > 50%         │ 发送告警     │
│ 慢调用率                │ > 50%         │ 发送告警     │
│ 线程池使用率            │ > 80%         │ 发送告警     │
│ API 响应时间 P99        │ > 30s         │ 发送告警     │
│ 降级响应率              │ > 30%         │ 发送告警     │
└─────────────────────────────────────────────────────┘

十二、完整请求示例

12.1 正常请求

curl -X POST http://localhost:8080/api/ai/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "什么是 Spring Boot？"}'

响应：

{
  "question": "什么是 Spring Boot？",
  "answer": "{\"choices\": [{\"message\": {\"role\": \"assistant\", \"content\": \"Spring Boot 是一个用于构建生产级 Spring 应用的框架...\"}}]}",
  "timestamp": 1699999999999
}

12.2 降级请求（熔断器打开时）

curl -X POST http://localhost:8080/api/ai/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "什么是微服务？"}'

响应：

{
  "question": "什么是微服务？",
  "answer": "{\"status\": \"degraded\", \"error_type\": \"circuit_breaker_open\", \"message\": \"AI 服务暂时繁忙，请稍后再试。\", \"question\": \"什么是微服务？\", \"suggestion\": \"您可以尝试简化问题，或者等待几分钟后重试。\"}",
  "timestamp": 1699999999999
}

12.3 监控查询

curl http://localhost:8080/api/monitor/circuit-breaker

响应：

{
  "name": "aiApiCircuitBreaker",
  "state": "CLOSED",
  "failureRate": 0.0,
  "slowCallRate": 0.0,
  "bufferedCalls": 0,
  "failedCalls": 0,
  "slowCalls": 0
}

十三、总结

13.1 三件套核心价值

组件	作用	配置要点
超时控制	防止请求无限等待	connectTimeout=5s, readTimeout=30s
熔断保护	故障时自动切断调用	failureRateThreshold=50%, waitDuration=30s
降级策略	提供兜底方案	缓存答案 + 友好提示

13.2 线程池配置要点

线程池配置原则：
1. 核心线程数 = CPU 核数 × 2（IO 密集型）
2. 最大线程数 = 核心线程数 × 2
3. 队列容量 = 最大线程数 × 5
4. 拒绝策略 = CallerRunsPolicy（背压机制）
5. 线程名前缀 = 便于排查问题

13.3 生产环境建议

生产环境 checklist：
✅ 设置合理的超时时间（connect=5s, read=30s）
✅ 配置熔断器（失败率阈值、慢调用阈值）
✅ 实现降级策略（缓存 + 友好提示）
✅ 配置独立线程池隔离 AI 调用
✅ 添加监控和告警
✅ 定期压测验证
✅ 准备熔断演练计划

💡 互动话题：你在调用外部 API 时遇到过哪些故障？是如何处理的？欢迎在评论区分享你的经验！