我们的业务代码里有需要用爬虫爬取商品的图片地址,然后在转到我们的服务器里的过程,中间当然少不了下载图片的过程,最近目标网站有些改变,就是之前http前缀的图片地址部分改成了https,然后就造成了一个问题,图片下载不了,查看log,发现一个java的SSLException.
异常是: java.lang.RuntimeException:Received fatal alert: protocol_version
异常的根源是:javax.net.ssl.SSLException: Received fatal alert: protocol_version.
----------------------------------------------------------------------------------------------------------------
原因是: https的请求需要加上ssl的验证,但是代码里一般只用到http,即使用到https也只是换了个前缀,并没有带上ssl验证的过程。
源代码:
private static final PoolingHttpClientConnectionManager HTTP_CLIENT_CONNECTION_MANAGER; private static final CloseableHttpClient HTTP_CLIENT; static { HTTP_CLIENT_CONNECTION_MANAGER = new PoolingHttpClientConnectionManager(RegistryBuilder .<ConnectionSocketFactory> create().register("http", PlainConnectionSocketFactory.getSocketFactory()) .register("https", SSLConnectionSocketFactory.getSocketFactory()).build()); HTTP_CLIENT_CONNECTION_MANAGER.setDefaultMaxPerRoute(100); HTTP_CLIENT_CONNECTION_MANAGER.setMaxTotal(200); RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(60000).setConnectTimeout(60000) .setSocketTimeout(60000).build(); HTTP_CLIENT = HttpClientBuilder.create().setConnectionManager(HTTP_CLIENT_CONNECTION_MANAGER) .setDefaultRequestConfig(requestConfig).build(); }
HttpGet httpGet = new HttpGet(url + (StringUtils.contains(url, "?") ? "&" : "?")
+ EntityUtils.toString(new UrlEncodedFormEntity(nameValuePairs, "UTF-8")));
CloseableHttpResponse httpResponse = HTTP_CLIENT.execute(httpGet);
----------------------------------------------------------------------------------------------------------------
解决方法:加上SSL的验证,加上协议
修改后可用的代码:
private static final PoolingHttpClientConnectionManager HTTP_CLIENT_CONNECTION_MANAGER; private static final CloseableHttpClient HTTP_CLIENT; static { SSLContext ctx = SSLContexts.createSystemDefault(); SSLConnectionSocketFactory fac = new SSLConnectionSocketFactory(ctx, new String[]{"SSLv2Hello", "TLSv1.2"}, null, SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER); HTTP_CLIENT_CONNECTION_MANAGER = new PoolingHttpClientConnectionManager(RegistryBuilder .<ConnectionSocketFactory> create().register("http", PlainConnectionSocketFactory.getSocketFactory()) .register("https", fac).build()); HTTP_CLIENT_CONNECTION_MANAGER.setDefaultMaxPerRoute(100); HTTP_CLIENT_CONNECTION_MANAGER.setMaxTotal(200); RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(60000).setConnectTimeout(60000) .setSocketTimeout(60000).build(); HTTP_CLIENT = HttpClientBuilder.create().setConnectionManager(HTTP_CLIENT_CONNECTION_MANAGER) .setDefaultRequestConfig(requestConfig).build(); }
HttpGet httpGet = new HttpGet(url + (StringUtils.contains(url, "?") ? "&" : "?")
+ EntityUtils.toString(new UrlEncodedFormEntity(nameValuePairs, "UTF-8")));
CloseableHttpResponse httpResponse = HTTP_CLIENT.execute(httpGet);
希望能帮到你。。。