httpclient的几个重要参数，及httpclient连接池的重要参数说明

httpclient的两个重要的参数maxPerRoute及MaxTotal

httpclient的连接池3个参数

HTTP请求时connectionRequestTimeout 、connectionTimeout、socketTimeout三个超时时间的含义
1.connectionRequestTimout：指从连接池获取连接的timeout

2.connetionTimeout：指客户端和服务器建立连接的timeout， 

就是http请求的三个阶段，一：建立连接；二：数据传送；三，断开连接。超时后会ConnectionTimeOutException

3.socketTimeout：指客户端和服务器建立连接后，客户端从服务器读取数据的timeout，超出后会抛出SocketTimeOutException

httpclient封装了java中进行http网络请求的底层实现，是一个被广泛使用的组件。

httpclient是支持池化机制的，这两个参数maxPerRoute及MaxTotal就是表示池化设置的。

服务之间发送http请求常用的有Apache的Fluent以及spring的restTemplate等。对httpclient进行封装的有：Apache的Fluent、es的restHighLevelClient、spring的restTemplate等。

以ES的restHighLevelClient为例说明

    /**
     * Creates a new {@link RestClient} based on the provided configuration.
     */
    public RestClient build() {
        if (failureListener == null) {
            failureListener = new RestClient.FailureListener();
        }
        CloseableHttpAsyncClient httpClient = AccessController.doPrivileged(new PrivilegedAction<CloseableHttpAsyncClient>() {
            @Override
            public CloseableHttpAsyncClient run() {
                return createHttpClient();
            }
        });
        RestClient restClient = new RestClient(httpClient, defaultHeaders, nodes,
                pathPrefix, failureListener, nodeSelector, strictDeprecationMode);
        httpClient.start();
        return restClient;
    }
    private CloseableHttpAsyncClient createHttpClient() {
        //default timeouts are all infinite
        RequestConfig.Builder requestConfigBuilder = RequestConfig.custom()
                .setConnectTimeout(DEFAULT_CONNECT_TIMEOUT_MILLIS)
                .setSocketTimeout(DEFAULT_SOCKET_TIMEOUT_MILLIS);
        if (requestConfigCallback != null) {
            requestConfigBuilder = requestConfigCallback.customizeRequestConfig(requestConfigBuilder);
        }

        try {
            HttpAsyncClientBuilder httpClientBuilder = HttpAsyncClientBuilder.create().setDefaultRequestConfig(requestConfigBuilder.build())
                //default settings for connection pooling may be too constraining
                .setMaxConnPerRoute(DEFAULT_MAX_CONN_PER_ROUTE).setMaxConnTotal(DEFAULT_MAX_CONN_TOTAL)
                .setSSLContext(SSLContext.getDefault())
                .setTargetAuthenticationStrategy(new PersistentCredentialsAuthenticationStrategy());
            if (httpClientConfigCallback != null) {
                httpClientBuilder = httpClientConfigCallback.customizeHttpClient(httpClientBuilder);
            }

            final HttpAsyncClientBuilder finalBuilder = httpClientBuilder;
            return AccessController.doPrivileged(new PrivilegedAction<CloseableHttpAsyncClient>() {
                @Override
                public CloseableHttpAsyncClient run() {
                    return finalBuilder.build();
                }
            });
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException("could not create the default ssl context", e);
        }
    }

例子2：Apache的Fluent，其Executor类

/**
 * An Executor for fluent requests
 * <p/>
 * A {@link PoolingHttpClientConnectionManager} with maximum 100 connections per route and
 * a total maximum of 200 connections is used internally.
 */
//最大100 connections per route 以及 最大200个 connection

CONNMGR = new PoolingHttpClientConnectionManager(sfr);

CONNMGR.setDefaultMaxPerRoute(100);

CONNMGR.setMaxTotal(200);

CLIENT = HttpClientBuilder.create().setConnectionManager(CONNMGR).build();

maxPerRoute及MaxTotal参数含义

maxPerRoute及MaxTotal这两个参数的含义是什么呢？

下面用测试代码说明一下

测试端

public class HttpFluentUtil {
    private Logger logger = LoggerFactory.getLogger(HttpFluentUtil.class);
    private final static int MaxPerRoute = 2;
    private final static int MaxTotal = 4;
    final static PoolingHttpClientConnectionManager CONNMGR;
    final static HttpClient CLIENT;
    final static Executor EXECUTOR;
    static {
        LayeredConnectionSocketFactory ssl = null;
        try {
            ssl = SSLConnectionSocketFactory.getSystemSocketFactory();
        } catch (final SSLInitializationException ex) {
            final SSLContext sslcontext;
            try {
                sslcontext = SSLContext.getInstance(SSLConnectionSocketFactory.TLS);
                sslcontext.init(null, null, null);
                ssl = new SSLConnectionSocketFactory(sslcontext);
            } catch (final SecurityException ignore) {
            } catch (final KeyManagementException ignore) {
            } catch (final NoSuchAlgorithmException ignore) {
            }
        }
        final Registry<ConnectionSocketFactory> sfr = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("http", PlainConnectionSocketFactory.getSocketFactory())
                .register("https", ssl != null ? ssl : SSLConnectionSocketFactory.getSocketFactory()).build();
        CONNMGR = new PoolingHttpClientConnectionManager(sfr);
        CONNMGR.setDefaultMaxPerRoute(MaxPerRoute);
        CONNMGR.setMaxTotal(MaxTotal);
        CLIENT = HttpClientBuilder.create().setConnectionManager(CONNMGR).build();
        EXECUTOR = Executor.newInstance(CLIENT);
    }
    public static String Get(String uri, int connectTimeout, int socketTimeout) throws IOException {
        return EXECUTOR.execute(Request.Get(uri).connectTimeout(connectTimeout).socketTimeout(socketTimeout))
                .returnContent().asString();
    }
    public static String Post(String uri, StringEntity stringEntity, int connectTimeout, int socketTimeout)
            throws IOException {
        return EXECUTOR.execute(Request.Post(uri).socketTimeout(socketTimeout)
                .addHeader("Content-Type", "application/json").body(stringEntity)).returnContent().asString();
    }
    public static void main(String[] args) {
        HttpUtil httpUtil = new HttpUtil();
        String url = "http://localhost:9064/app/test"; // 服务端sleep 5秒再返回
        for (int i = 0; i < 5; i++) { // MaxPerRoute若设置为2，则5线程分3组返回（2、2、1），共15秒
            new Thread(new Runnable() {
                @Override
                public void run() {
                    try {
                        String result = HttpFluentUtil.Get(url, 2000, 2000);
                        System.out.println(result);
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            }).start();
        }
    }
}

服务器端

很简单的springmvc

@GetMapping(value="test")
public String test() throws InterruptedException {
    Thread.sleep(1000);
    return "1";
}

测试1：测试端MaxPerRoute=5 MaxTotal=4

服务器端结果

 

可以看到先接收4个请求，处理完成后，再接收下一次剩余的1个请求。即其一次最多接收MaxTotal次请求

测试2：测试端MaxPerRoute=2 MaxTotal=5

服务器端结果

 

可以看到接收2个请求，2个请求，1个请求，即说明maxPerRoute意思是某一个服务每次能并行接收的请求数量。

什么场景下要设置？

知道了两个参数的含义，那么在什么情况下要对这两个参数进行设置呢？

比如说下面的场景

服务1要通过Fluent调用服务2的接口。服务1发送了400个请求，但由于Fluent默认只支持maxPerRoute=100，MaxTotal=200，比如接口执行时间为500ms，由于maxPerRoute=100，所以要分为100,100,100,100分四批来执行，全部执行完成需要2000ms。而如果maxPerRoute设置为400，全部执行完需要500ms。在这种情况下（提供并发能力时）就要对这两个参数进行设置了。

设置的方法

1、Apache Fluent可以使用上面测试的HttpFluentUtil工具类来执行请求

2、RestTemplate类似使用下面的方式

@Bean
public HttpClient httpClient() {
   Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
            .register("http", PlainConnectionSocketFactory.getSocketFactory())
            .register("https", SSLConnectionSocketFactory.getSocketFactory())
            .build();
    PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(registry);
    connectionManager.setMaxTotal(restTemplateProperties.getMaxTotal());
    connectionManager.setDefaultMaxPerRoute(restTemplateProperties.getDefaultMaxPerRoute());
    connectionManager.setValidateAfterInactivity(restTemplateProperties.getValidateAfterInactivity());
    RequestConfig requestConfig = RequestConfig.custom()
            .setSocketTimeout(restTemplateProperties.getSocketTimeout())
            .setConnectTimeout(restTemplateProperties.getConnectTimeout())
            .setConnectionRequestTimeout(restTemplateProperties.getConnectionRequestTimeout())
            .build();
    return HttpClientBuilder.create()
            .setDefaultRequestConfig(requestConfig)
            .setConnectionManager(connectionManager)
            .build();
   
}
@Bean
public ClientHttpRequestFactory httpRequestFactory() {
    return new HttpComponentsClientHttpRequestFactory(httpClient());
}
@Bean
public RestTemplate restTemplate() {
    return new RestTemplate(httpRequestFactory());
}

其中RestTemplateProperties通过配置文件来配置

max-total
default-max-per-route
connect-timeout  获取连接超时
connection-request-timeout  请求超时
socket-timeout  读超时

3、ES的restHighLevelClient设置

    @Bean
    public RestHighLevelClient restHighLevelClient(){
        //解析hostlist配置信息
        String[] split = hostlist.split(",");
        //创建HttpHost数组，其中存放es主机和端口的配置信息
        HttpHost[] httpHostArray = new HttpHost[split.length];
        for(int i=0;i<split.length;i++){
            String item = split[i];
            httpHostArray[i] = new HttpHost(item.split(":")[0], Integer.parseInt(item.split(":")[1]), "http");
        }
        //创建RestHighLevelClient客户端
        //return new RestHighLevelClient(RestClient.builder(httpHostArray));//.setMaxRetryTimeoutMillis(5 * 60 * 1000)); //超时时间设为5分钟);
        RestClientBuilder builder = RestClient.builder(httpHostArray);

        builder.setRequestConfigCallback(requestConfigBuilder -> {
            requestConfigBuilder.setConnectTimeout(connectTimeoutMillis);
            requestConfigBuilder.setSocketTimeout(socketTimeoutMillis);
            requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeoutMillis);
            return requestConfigBuilder;
        });

        builder.setHttpClientConfigCallback(httpClientBuilder -> {
            httpClientBuilder.setMaxConnTotal(maxConnectTotal);
            httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
            return httpClientBuilder;
        });


        return new RestHighLevelClient(builder);
        
    }

里面的5个变量，通过spring的@Value注入，这里省略。

总结：

max-total：连接池里的最大连接数
default-max-per-route：某一个/每服务每次能并行接收的请求数量
connect-timeout  从连接池里获取连接超时时间
connection-request-timeout  请求超时时间
socket-timeout  读超时时间

参考：https://blog.csdn.net/u013905744/java/article/details/94714696