zoukankan      html  css  js  c++  java
  • java HttpClicent网络请求与解析

      HttpClicent是Apache下的一个子项目,文档齐全,详见官网:http://hc.apache.org/httpcomponents-client-4.5.x/quickstart.html

      这里列出一些常见用法:

        (1)使用步骤:

          1. 创建HttpClient对象。

          2. 创建请求方法的实例,并指定请求URL。如果需要发送GET请求,创建HttpGet对象;如果需要发送POST请求,创建HttpPost对象。

          3. 如果需要发送请求参数,可调用HttpGet、HttpPost共同的setParams(HetpParams params)方法来添加请求参数;对于HttpPost对象而言,也可调用setEntity(HttpEntity entity)方法来设置请求参数。

          4. 调用HttpClient对象的execute(HttpUriRequest request)发送请求,该方法返回一个HttpResponse。

          5. 调用HttpResponse的getAllHeaders()、getHeaders(String name)等方法可获取服务器的响应头;调用HttpResponse的getEntity()方法可获取HttpEntity对象,该对象包装了服务器的响应内容。程序可通过该对象获取服务器的响应内容。

          6. 释放连接。无论执行方法是否成功,都必须释放连接

        (2)官网示例:CloseableHttpClient httpclient = HttpClients.createDefault();HttpGet httpGet = new HttpGet("http://targethost/homepage");

    CloseableHttpResponse response1 = httpclient.execute(httpGet);
    // The underlying HTTP connection is still held by the response object
    // to allow the response content to be streamed directly from the network socket.
    // In order to ensure correct deallocation of system resources
    // the user MUST call CloseableHttpResponse#close() from a finally clause.
    // Please note that if response content is not fully consumed the underlying
    // connection cannot be safely re-used and will be shut down and discarded
    // by the connection manager. 
    try {
        System.out.println(response1.getStatusLine());
        HttpEntity entity1 = response1.getEntity();
        // do something useful with the response body
        // and ensure it is fully consumed
        EntityUtils.consume(entity1);
    } finally {
        response1.close();
    }
    
    HttpPost httpPost = new HttpPost("http://targethost/login");
    List <NameValuePair> nvps = new ArrayList <NameValuePair>();
    nvps.add(new BasicNameValuePair("username", "vip"));
    nvps.add(new BasicNameValuePair("password", "secret"));
    httpPost.setEntity(new UrlEncodedFormEntity(nvps));
    CloseableHttpResponse response2 = httpclient.execute(httpPost);
    
    try {
        System.out.println(response2.getStatusLine());
        HttpEntity entity2 = response2.getEntity();
        // do something useful with the response body
        // and ensure it is fully consumed
        EntityUtils.consume(entity2);
    } finally {
        response2.close();
    }
      (3)解析结果:
        1.返回时JSON:
              List<NameValuePair> nvps = new ArrayList<NameValuePair>();
                nvps.add(new BasicNameValuePair("loginname","ss"));
                nvps.add(new BasicNameValuePair("password", "s123"));
                hp.setEntity(new UrlEncodedFormEntity(nvps));
                HttpResponse res = client.execute(hp);
                html = EntityUtils.toString(res.getEntity());

            JSONObject obj = JSON.parseObject(html);//这里用了fastjson,把String类型的html转换成JSON对象,若有多个,可转换成数组
            Iterator it = obj.keySet().iterator();
            while(it.hasNext()){
              String key = it.next().toString();
              if(key.equals("Success")){
              html = obj.get(key).toString();
              }
            }

        2.返回html:  

            HttpGet hg = new HttpGet(course_list_url);
                HttpResponse res = client.execute(hg);
                html = EntityUtils.toString(res.getEntity());
                Document doc = Jsoup.parse(html);//这里用了Jsoup抓取需要的html代码
                Element content = doc.getElementById("col-sm-5");
                Elements links = content.getElementsByTag("a");
                for (Element link : links) {
                  String linkHref = link.attr("href");
                  String linkText = link.text();
                }
                String courseId = "";
                Elements courseIds = doc.select("div.overflow-h>a[href]");
                Elements courseTimes = doc.select("h3.heading-xs>a");

        (4)其他关于HttoClicent相关例子:

      1 package com.test;
      2 
      3 import java.io.File;
      4 import java.io.FileInputStream;
      5 import java.io.IOException;
      6 import java.io.UnsupportedEncodingException;
      7 import java.security.KeyManagementException;
      8 import java.security.KeyStore;
      9 import java.security.KeyStoreException;
     10 import java.security.NoSuchAlgorithmException;
     11 import java.security.cert.CertificateException;
     12 import java.util.ArrayList;
     13 import java.util.List;
     14 
     15 import javax.net.ssl.SSLContext;
     16 
     17 import org.apache.http.HttpEntity;
     18 import org.apache.http.NameValuePair;
     19 import org.apache.http.ParseException;
     20 import org.apache.http.client.ClientProtocolException;
     21 import org.apache.http.client.entity.UrlEncodedFormEntity;
     22 import org.apache.http.client.methods.CloseableHttpResponse;
     23 import org.apache.http.client.methods.HttpGet;
     24 import org.apache.http.client.methods.HttpPost;
     25 import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
     26 import org.apache.http.conn.ssl.SSLContexts;
     27 import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
     28 import org.apache.http.entity.ContentType;
     29 import org.apache.http.entity.mime.MultipartEntityBuilder;
     30 import org.apache.http.entity.mime.content.FileBody;
     31 import org.apache.http.entity.mime.content.StringBody;
     32 import org.apache.http.impl.client.CloseableHttpClient;
     33 import org.apache.http.impl.client.HttpClients;
     34 import org.apache.http.message.BasicNameValuePair;
     35 import org.apache.http.util.EntityUtils;
     36 import org.junit.Test;
     37 
     38 public class HttpClientTest {
     39 
     40     @Test
     41     public void jUnitTest() {
     42         get();
     43     }
     44 
     45     /**
     46      * HttpClient连接SSL
     47      */
     48     public void ssl() {
     49         CloseableHttpClient httpclient = null;
     50         try {
     51             KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
     52             FileInputStream instream = new FileInputStream(new File("d:\tomcat.keystore"));
     53             try {
     54                 // 加载keyStore d:\tomcat.keystore  
     55                 trustStore.load(instream, "123456".toCharArray());
     56             } catch (CertificateException e) {
     57                 e.printStackTrace();
     58             } finally {
     59                 try {
     60                     instream.close();
     61                 } catch (Exception ignore) {
     62                 }
     63             }
     64             // 相信自己的CA和所有自签名的证书
     65             SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(trustStore, new TrustSelfSignedStrategy()).build();
     66             // 只允许使用TLSv1协议
     67             SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslcontext, new String[] { "TLSv1" }, null,
     68                     SSLConnectionSocketFactory.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER);
     69             httpclient = HttpClients.custom().setSSLSocketFactory(sslsf).build();
     70             // 创建http请求(get方式)
     71             HttpGet httpget = new HttpGet("https://localhost:8443/myDemo/Ajax/serivceJ.action");
     72             System.out.println("executing request" + httpget.getRequestLine());
     73             CloseableHttpResponse response = httpclient.execute(httpget);
     74             try {
     75                 HttpEntity entity = response.getEntity();
     76                 System.out.println("----------------------------------------");
     77                 System.out.println(response.getStatusLine());
     78                 if (entity != null) {
     79                     System.out.println("Response content length: " + entity.getContentLength());
     80                     System.out.println(EntityUtils.toString(entity));
     81                     EntityUtils.consume(entity);
     82                 }
     83             } finally {
     84                 response.close();
     85             }
     86         } catch (ParseException e) {
     87             e.printStackTrace();
     88         } catch (IOException e) {
     89             e.printStackTrace();
     90         } catch (KeyManagementException e) {
     91             e.printStackTrace();
     92         } catch (NoSuchAlgorithmException e) {
     93             e.printStackTrace();
     94         } catch (KeyStoreException e) {
     95             e.printStackTrace();
     96         } finally {
     97             if (httpclient != null) {
     98                 try {
     99                     httpclient.close();
    100                 } catch (IOException e) {
    101                     e.printStackTrace();
    102                 }
    103             }
    104         }
    105     }
    106 
    107     /**
    108      * post方式提交表单(模拟用户登录请求)
    109      */
    110     public void postForm() {
    111         // 创建默认的httpClient实例.  
    112         CloseableHttpClient httpclient = HttpClients.createDefault();
    113         // 创建httppost  
    114         HttpPost httppost = new HttpPost("http://localhost:8080/myDemo/Ajax/serivceJ.action");
    115         // 创建参数队列  
    116         List<namevaluepair> formparams = new ArrayList<namevaluepair>();
    117         formparams.add(new BasicNameValuePair("username", "admin"));
    118         formparams.add(new BasicNameValuePair("password", "123456"));
    119         UrlEncodedFormEntity uefEntity;
    120         try {
    121             uefEntity = new UrlEncodedFormEntity(formparams, "UTF-8");
    122             httppost.setEntity(uefEntity);
    123             System.out.println("executing request " + httppost.getURI());
    124             CloseableHttpResponse response = httpclient.execute(httppost);
    125             try {
    126                 HttpEntity entity = response.getEntity();
    127                 if (entity != null) {
    128                     System.out.println("--------------------------------------");
    129                     System.out.println("Response content: " + EntityUtils.toString(entity, "UTF-8"));
    130                     System.out.println("--------------------------------------");
    131                 }
    132             } finally {
    133                 response.close();
    134             }
    135         } catch (ClientProtocolException e) {
    136             e.printStackTrace();
    137         } catch (UnsupportedEncodingException e1) {
    138             e1.printStackTrace();
    139         } catch (IOException e) {
    140             e.printStackTrace();
    141         } finally {
    142             // 关闭连接,释放资源  
    143             try {
    144                 httpclient.close();
    145             } catch (IOException e) {
    146                 e.printStackTrace();
    147             }
    148         }
    149     }
    150 
    151     /**
    152      * 发送 post请求访问本地应用并根据传递参数不同返回不同结果
    153      */
    154     public void post() {
    155         // 创建默认的httpClient实例.  
    156         CloseableHttpClient httpclient = HttpClients.createDefault();
    157         // 创建httppost  
    158         HttpPost httppost = new HttpPost("http://localhost:8080/myDemo/Ajax/serivceJ.action");
    159         // 创建参数队列  
    160         List<namevaluepair> formparams = new ArrayList<namevaluepair>();
    161         formparams.add(new BasicNameValuePair("type", "house"));
    162         UrlEncodedFormEntity uefEntity;
    163         try {
    164             uefEntity = new UrlEncodedFormEntity(formparams, "UTF-8");
    165             httppost.setEntity(uefEntity);
    166             System.out.println("executing request " + httppost.getURI());
    167             CloseableHttpResponse response = httpclient.execute(httppost);
    168             try {
    169                 HttpEntity entity = response.getEntity();
    170                 if (entity != null) {
    171                     System.out.println("--------------------------------------");
    172                     System.out.println("Response content: " + EntityUtils.toString(entity, "UTF-8"));
    173                     System.out.println("--------------------------------------");
    174                 }
    175             } finally {
    176                 response.close();
    177             }
    178         } catch (ClientProtocolException e) {
    179             e.printStackTrace();
    180         } catch (UnsupportedEncodingException e1) {
    181             e1.printStackTrace();
    182         } catch (IOException e) {
    183             e.printStackTrace();
    184         } finally {
    185             // 关闭连接,释放资源  
    186             try {
    187                 httpclient.close();
    188             } catch (IOException e) {
    189                 e.printStackTrace();
    190             }
    191         }
    192     }
    193 
    194     /**
    195      * 发送 get请求
    196      */
    197     public void get() {
    198         CloseableHttpClient httpclient = HttpClients.createDefault();
    199         try {
    200             // 创建httpget.  
    201             HttpGet httpget = new HttpGet("http://www.baidu.com/");
    202             System.out.println("executing request " + httpget.getURI());
    203             // 执行get请求.  
    204             CloseableHttpResponse response = httpclient.execute(httpget);
    205             try {
    206                 // 获取响应实体  
    207                 HttpEntity entity = response.getEntity();
    208                 System.out.println("--------------------------------------");
    209                 // 打印响应状态  
    210                 System.out.println(response.getStatusLine());
    211                 if (entity != null) {
    212                     // 打印响应内容长度  
    213                     System.out.println("Response content length: " + entity.getContentLength());
    214                     // 打印响应内容  
    215                     System.out.println("Response content: " + EntityUtils.toString(entity));
    216                 }
    217                 System.out.println("------------------------------------");
    218             } finally {
    219                 response.close();
    220             }
    221         } catch (ClientProtocolException e) {
    222             e.printStackTrace();
    223         } catch (ParseException e) {
    224             e.printStackTrace();
    225         } catch (IOException e) {
    226             e.printStackTrace();
    227         } finally {
    228             // 关闭连接,释放资源  
    229             try {
    230                 httpclient.close();
    231             } catch (IOException e) {
    232                 e.printStackTrace();
    233             }
    234         }
    235     }
    236 
    237     /**
    238      * 上传文件
    239      */
    240     public void upload() {
    241         CloseableHttpClient httpclient = HttpClients.createDefault();
    242         try {
    243             HttpPost httppost = new HttpPost("http://localhost:8080/myDemo/Ajax/serivceFile.action");
    244 
    245             FileBody bin = new FileBody(new File("F:\image\sendpix0.jpg"));
    246             StringBody comment = new StringBody("A binary file of some kind", ContentType.TEXT_PLAIN);
    247 
    248             HttpEntity reqEntity = MultipartEntityBuilder.create().addPart("bin", bin).addPart("comment", comment).build();
    249 
    250             httppost.setEntity(reqEntity);
    251 
    252             System.out.println("executing request " + httppost.getRequestLine());
    253             CloseableHttpResponse response = httpclient.execute(httppost);
    254             try {
    255                 System.out.println("----------------------------------------");
    256                 System.out.println(response.getStatusLine());
    257                 HttpEntity resEntity = response.getEntity();
    258                 if (resEntity != null) {
    259                     System.out.println("Response content length: " + resEntity.getContentLength());
    260                 }
    261                 EntityUtils.consume(resEntity);
    262             } finally {
    263                 response.close();
    264             }
    265         } catch (ClientProtocolException e) {
    266             e.printStackTrace();
    267         } catch (IOException e) {
    268             e.printStackTrace();
    269         } finally {
    270             try {
    271                 httpclient.close();
    272             } catch (IOException e) {
    273                 e.printStackTrace();
    274             }
    275         }
    276     }
    277 }</namevaluepair></namevaluepair></namevaluepair></namevaluepair>
    
    
  • 相关阅读:
    DRUPAL-PSA-CORE-2014-005 && CVE-2014-3704 Drupal 7.31 SQL Injection Vulnerability /includes/database/database.inc Analysis
    WDCP(WDlinux Control Panel) mysql/add_user.php、mysql/add_db.php Authentication Loss
    Penetration Testing、Security Testing、Automation Testing
    Tomcat Server Configuration Automation Reinforcement
    Xcon2014 && Geekpwn2014
    phpMyadmin /scripts/setup.php Remote Code Injection && Execution CVE-2009-1151
    Linux System Log Collection、Log Integration、Log Analysis System Building Learning
    The Linux Process Principle,NameSpace, PID、TID、PGID、PPID、SID、TID、TTY
    Windows Management Instrumentation WMI Security Technology Learning
    IIS FTP Server Anonymous Writeable Reinforcement, WEBDAV Anonymous Writeable Reinforcement(undone)
  • 原文地址:https://www.cnblogs.com/nightswatch/p/4777406.html
Copyright © 2011-2022 走看看