浏览器与服务器交互原理以及用java模拟浏览器操作v

浏览器应用服务器JavaPHPApache 

* 1,在HTTP的WEB应用中, 应用客户端和服务器之间的状态是通过Session来维持的, 而Session的本质就是Cookie, 
* 简单的讲,当浏览器向服务器发送Http请求的时候, HTTP服务器会产生一个SessionID,这个SessionID就唯一的标识了一个客户端到服务器的请求会话过程. 
* 就如同一次会议开始时,主办方给每位到场的嘉宾一个临时的编号胸牌一样, 可以通过这个编号记录每个嘉宾(客户端)的活动(请求状态). 
* 为了保持这个状态, 当服务端向客户端回应的时候,会附带Cookie信息,当然,Cookie里面就包含了SessionID 
* 客户端在执行一系列操作时向服务端发送请求时,也会带上这个SessionID, 一般来说,Session也是一个URL QueryParameter ,就是说,session可以以Key-Value的形式通过URL传递 
* 比如,http://www.51etest.com/dede/login.php?PHPSESSIONID=7dg3dsf19SDf73wqc32fdsf 
* 一般而言,浏览器会自动把此Session信息放入Header报文体中进行传递. 
* 如果浏览器不支持Cookie,那么,浏览器会自动把SessionID附加到URL中去. 
* 
* 2,在这个例子中,以登陆这个功能点进行讲解. 
* 首先,我们登陆的页面是http://www.51etest.com/dede, 我们第一次访问这个页面后,可以从服务器过来的Http Response报文中的Header中找出服务器与浏览器向关联的数据 — Cookie, 
* 而且Session的值也在Cookie中. 于是,我们可以通过分析Set-Cookie这个Header中的参数的值,找到Seesion的Key-Value段. 
* 然后,我们再向服务器发送请求,请求URL为:post@@http://www.51etest.com/dede/login.php@@userid=admin&pwd=tidus2005&gotopage=/dede/&dopost=login
* 服务器验证登陆成功了, 并且在此次会话变量中增加了我们登陆成功的标识. 
* 
* 3,增加一个广告定义 
* 增加一个广告定义其实就是一个添加数据的过程,无非是我们把我们要添加的数据通过参数的形式告诉指定url页面,页面获取后添加到数据库去而已. 
* 此url地址为: 
* post@@http://www.51etest.com/dede/ad_add.php@@dopost=save&tagname=test&typeid=0&adname=test&starttime=2008-05-29 
* 因为这个页面会先判断我是否登陆 
* 而判断的依据，前面讲了，就是根据我请求时的SessionID找到指定的Session数据区中是否存在我的登陆信息，
* 所以我当然要把访问登陆页面时获取的SessionID原封不动的再发回去 
* 相当于对服务器说，这是我刚刚来时，你发我的临时身份证，我现在可以形势我的权利。 
* 
* 这就是整个Java后台登陆网站，然后添加数据的过程。 

/** 
*  
*/ 
package sky.dong.test; 

import java.io.BufferedReader; 
import java.io.InputStreamReader; 

import org.apache.commons.httpclient.Cookie; 
import org.apache.commons.httpclient.Header; 
import org.apache.commons.httpclient.HttpClient; 
import org.apache.commons.httpclient.NameValuePair; 
import org.apache.commons.httpclient.cookie.CookiePolicy; 
import org.apache.commons.httpclient.methods.PostMethod; 
import org.apache.commons.httpclient.params.HttpMethodParams; 

/** 
* @author 核弹头 
* Email:happyman_dong@sina.com 版权所有 盗版必究 
* @since 2009-8-11 
* @version 1.0 
*/ 
public class HttpLoginTest { 

    public static void main(String[] args) { 
        String url = "http://discuzdemo.c88.53dns.com/logging.php?action=login&loginsubmit=yes&floatlogin=yes";//论坛的登陆页面 
        String url2="http://discuzdemo.c88.53dns.com/post.php?infloat=yes&action=newthread&fid=2&extra=&topicsubmit=yes&inajax=1";//论坛的发贴页面 
        HttpClient httpClient = new HttpClient(); 
        //httpClient.getHostConfiguration().setProxy("222.247.62.195", 8080); 
        httpClient.getParams().setCookiePolicy( 
                CookiePolicy.BROWSER_COMPATIBILITY); 
        PostMethod postMethod = new PostMethod(url); 
        PostMethod postMethod2 = new PostMethod(url2); 
        NameValuePair[] data = { 
                new NameValuePair("username", "123"), 
                new NameValuePair("referer", 
                        "http://discuzdemo.c88.53dns.com/index.php"), 
                new NameValuePair("password", "123"), 
                new NameValuePair("loginfield", "username"), 
                new NameValuePair("questionid", "0"), 
                new NameValuePair("formhash", "fc922ca7") }; 
        postMethod.setRequestHeader("Referer", 
                "http://discuzdemo.c88.53dns.com/index.php"); 
        postMethod.setRequestHeader("Host", "discuzdemo.c88.53dns.com"); 
        // postMethod.setRequestHeader("Connection", "keep-alive"); 
        // postMethod.setRequestHeader("Cookie", "jbu_oldtopics=D123D; 
        // jbu_fid2=1249912623; smile=1D1; jbu_onlineusernum=2; 
        // jbu_sid=amveZM"); 
        postMethod 
                .setRequestHeader( 
                        "User-Agent", 
                        "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2"); 
        postMethod 
                .setRequestHeader("Accept", 
                        "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"); 
        // postMethod.setRequestHeader("Accept-Encoding", "gzip,deflate"); 
        // postMethod.setRequestHeader("Accept-Language", "zh-cn"); 
        // postMethod.setRequestHeader("Accept-Charset", 
        // "GB2312,utf-8;q=0.7,*;q=0.7"); 
        postMethod.setRequestBody(data); 
        try { 
            httpClient.executeMethod(postMethod); 
            StringBuffer response = new StringBuffer(); 
            BufferedReader reader = new BufferedReader(new InputStreamReader( 
                    postMethod.getResponseBodyAsStream(), "gb2312"));//以gb2312编码方式打印从服务器端返回的请求 
            String line; 
            while ((line = reader.readLine()) != null) { 
                response.append(line).append( 
                        System.getProperty("line.separator")); 
            } 
            reader.close(); 
            Header header = postMethod.getResponseHeader("Set-Cookie"); 
            Cookie[] cookies=httpClient.getState().getCookies();//取出登陆成功后，服务器返回的cookies信息，里面保存了服务器端给的“临时证” 
            String tmpcookies=""; 
            for(Cookie c:cookies){ 
                tmpcookies=tmpcookies+c.toString()+";"; 
                System.out.println(c); 
            } 
            System.out.println(tmpcookies); 
//            System.out.println(header.getValue()); 
            System.out.println(response); 
            NameValuePair[] data2 = { 
                    new NameValuePair("subject", "测试自动发贴"), 
                    new NameValuePair("message", 
                            "能否发贴成功呢？测试一下就知道了"), 
                    new NameValuePair("updateswfattach", "0"), 
                    new NameValuePair("wysiwyg", "0"), 
                    new NameValuePair("checkbox", "0"), 
                    new NameValuePair("handlekey", "newthread"), 
                    new NameValuePair("formhash", "885493ec") }; 
            postMethod2.setRequestHeader("cookie",tmpcookies);//将“临时证明”放入下一次的发贴请求操作中 
            postMethod2.getParams().setParameter(HttpMethodParams.HTTP_CONTENT_CHARSET, "gbk");//因为发贴时候有中文，设置一下请求编码 
            postMethod2.setRequestHeader("Referer", 
                    "http://discuzdemo.c88.53dns.com/forumdisplay.php?fid=4"); 
            postMethod2.setRequestHeader("Host", "discuzdemo.c88.53dns.com"); 
            // postMethod.setRequestHeader("Connection", "keep-alive"); 
            // postMethod.setRequestHeader("Cookie", "jbu_oldtopics=D123D; 
            // jbu_fid2=1249912623; smile=1D1; jbu_onlineusernum=2; 
            // jbu_sid=amveZM"); 
            postMethod2 
                    .setRequestHeader( 
                            "User-Agent", 
                            "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2");
            postMethod2 
                    .setRequestHeader("Accept", 
                            "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");//以上操作是模拟浏览器的操作，使用服务器混淆 
             
            postMethod2.setRequestBody(data2); 
            httpClient.executeMethod(postMethod2); 
            StringBuffer response1 = new StringBuffer(); 
            BufferedReader reader1 = new BufferedReader(new InputStreamReader( 
                    postMethod2.getResponseBodyAsStream(), "gb2312")); 
            String line1; 
            while ((line1 = reader1.readLine()) != null) { 
                response1.append(line1).append( 
                        System.getProperty("line.separator")); 
            } 
            reader1.close(); 
            System.out.println(response1); 
        } catch (Exception e) { 
            System.out.println(e.getMessage()); 
            // TODO: handle exception 
        } finally { 
            postMethod.releaseConnection(); 
            postMethod2.releaseConnection(); 
        } 

    } 

} 
飛奔嘅蝸牛 
@Strong Yuan 
浏览器与服务器交互原理以及用java模拟浏览器操作 
Posted by strong on 2011/04/11 Leave a comment (0) Go to comments 

* 1,在HTTP的WEB应用中, 应用客户端和服务器之间的状态是通过Session来维持的, 而Session的本质就是Cookie, 
* 简单的讲,当浏览器向服务器发送Http请求的时候, HTTP服务器会产生一个SessionID,这个SessionID就唯一的标识了一个客户端到服务器的请求会话过程. 
* 就如同一次会议开始时,主办方给每位到场的嘉宾一个临时的编号胸牌一样, 可以通过这个编号记录每个嘉宾(客户端)的活动(请求状态). 
* 为了保持这个状态, 当服务端向客户端回应的时候,会附带Cookie信息,当然,Cookie里面就包含了SessionID 
* 客户端在执行一系列操作时向服务端发送请求时,也会带上这个SessionID, 一般来说,Session也是一个URL QueryParameter ,就是说,session可以以Key-Value的形式通过URL传递 
* 比如,http://www.51etest.com/dede/login.php?PHPSESSIONID=7dg3dsf19SDf73wqc32fdsf 
* 一般而言,浏览器会自动把此Session信息放入Header报文体中进行传递. 
* 如果浏览器不支持Cookie,那么,浏览器会自动把SessionID附加到URL中去. 
* 
* 2,在这个例子中,以登陆这个功能点进行讲解. 
* 首先,我们登陆的页面是http://www.51etest.com/dede, 我们第一次访问这个页面后,可以从服务器过来的Http Response报文中的Header中找出服务器与浏览器向关联的数据 — Cookie, 
* 而且Session的值也在Cookie中. 于是,我们可以通过分析Set-Cookie这个Header中的参数的值,找到Seesion的Key-Value段. 
* 然后,我们再向服务器发送请求,请求URL为:post@@http://www.51etest.com/dede/login.php@@userid=admin&pwd=tidus2005&gotopage=/dede/&dopost=login
* 服务器验证登陆成功了, 并且在此次会话变量中增加了我们登陆成功的标识. 
* 
* 3,增加一个广告定义 
* 增加一个广告定义其实就是一个添加数据的过程,无非是我们把我们要添加的数据通过参数的形式告诉指定url页面,页面获取后添加到数据库去而已. 
* 此url地址为: 
* post@@http://www.51etest.com/dede/ad_add.php@@dopost=save&tagname=test&typeid=0&adname=test&starttime=2008-05-29 
* 因为这个页面会先判断我是否登陆 
* 而判断的依据，前面讲了，就是根据我请求时的SessionID找到指定的Session数据区中是否存在我的登陆信息，
* 所以我当然要把访问登陆页面时获取的SessionID原封不动的再发回去 
* 相当于对服务器说，这是我刚刚来时，你发我的临时身份证，我现在可以形势我的权利。 
* 
* 这就是整个Java后台登陆网站，然后添加数据的过程。 


/** 
*  
*/ 
package sky.dong.test; 

import java.io.BufferedReader; 
import java.io.InputStreamReader; 

import org.apache.commons.httpclient.Cookie; 
import org.apache.commons.httpclient.Header; 
import org.apache.commons.httpclient.HttpClient; 
import org.apache.commons.httpclient.NameValuePair; 
import org.apache.commons.httpclient.cookie.CookiePolicy; 
import org.apache.commons.httpclient.methods.PostMethod; 
import org.apache.commons.httpclient.params.HttpMethodParams; 

/** 
* @author 核弹头 
* Email:happyman_dong@sina.com 版权所有 盗版必究 
* @since 2009-8-11 
* @version 1.0 
*/ 
public class HttpLoginTest { 

    public static void main(String[] args) { 
        String url = "http://discuzdemo.c88.53dns.com/logging.php?action=login&loginsubmit=yes&floatlogin=yes";//论坛的登陆页面 
        String url2="http://discuzdemo.c88.53dns.com/post.php?infloat=yes&action=newthread&fid=2&extra=&topicsubmit=yes&inajax=1";//论坛的发贴页面 
        HttpClient httpClient = new HttpClient(); 
        //httpClient.getHostConfiguration().setProxy("222.247.62.195", 8080); 
        httpClient.getParams().setCookiePolicy( 
                CookiePolicy.BROWSER_COMPATIBILITY); 
        PostMethod postMethod = new PostMethod(url); 
        PostMethod postMethod2 = new PostMethod(url2); 
        NameValuePair[] data = { 
                new NameValuePair("username", "123"), 
                new NameValuePair("referer", 
                        "http://discuzdemo.c88.53dns.com/index.php"), 
                new NameValuePair("password", "123"), 
                new NameValuePair("loginfield", "username"), 
                new NameValuePair("questionid", "0"), 
                new NameValuePair("formhash", "fc922ca7") }; 
        postMethod.setRequestHeader("Referer", 
                "http://discuzdemo.c88.53dns.com/index.php"); 
        postMethod.setRequestHeader("Host", "discuzdemo.c88.53dns.com"); 
        // postMethod.setRequestHeader("Connection", "keep-alive"); 
        // postMethod.setRequestHeader("Cookie", "jbu_oldtopics=D123D; 
        // jbu_fid2=1249912623; smile=1D1; jbu_onlineusernum=2; 
        // jbu_sid=amveZM"); 
        postMethod 
                .setRequestHeader( 
                        "User-Agent", 
                        "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2"); 
        postMethod 
                .setRequestHeader("Accept", 
                        "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"); 
        // postMethod.setRequestHeader("Accept-Encoding", "gzip,deflate"); 
        // postMethod.setRequestHeader("Accept-Language", "zh-cn"); 
        // postMethod.setRequestHeader("Accept-Charset", 
        // "GB2312,utf-8;q=0.7,*;q=0.7"); 
        postMethod.setRequestBody(data); 
        try { 
            httpClient.executeMethod(postMethod); 
            StringBuffer response = new StringBuffer(); 
            BufferedReader reader = new BufferedReader(new InputStreamReader( 
                    postMethod.getResponseBodyAsStream(), "gb2312"));//以gb2312编码方式打印从服务器端返回的请求 
            String line; 
            while ((line = reader.readLine()) != null) { 
                response.append(line).append( 
                        System.getProperty("line.separator")); 
            } 
            reader.close(); 
            Header header = postMethod.getResponseHeader("Set-Cookie"); 
            Cookie[] cookies=httpClient.getState().getCookies();//取出登陆成功后，服务器返回的cookies信息，里面保存了服务器端给的“临时证” 
            String tmpcookies=""; 
            for(Cookie c:cookies){ 
                tmpcookies=tmpcookies+c.toString()+";"; 
                System.out.println(c); 
            } 
            System.out.println(tmpcookies); 
//            System.out.println(header.getValue()); 
            System.out.println(response); 
            NameValuePair[] data2 = { 
                    new NameValuePair("subject", "测试自动发贴"), 
                    new NameValuePair("message", 
                            "能否发贴成功呢？测试一下就知道了"), 
                    new NameValuePair("updateswfattach", "0"), 
                    new NameValuePair("wysiwyg", "0"), 
                    new NameValuePair("checkbox", "0"), 
                    new NameValuePair("handlekey", "newthread"), 
                    new NameValuePair("formhash", "885493ec") }; 
            postMethod2.setRequestHeader("cookie",tmpcookies);//将“临时证明”放入下一次的发贴请求操作中 
            postMethod2.getParams().setParameter(HttpMethodParams.HTTP_CONTENT_CHARSET, "gbk");//因为发贴时候有中文，设置一下请求编码 
            postMethod2.setRequestHeader("Referer", 
                    "http://discuzdemo.c88.53dns.com/forumdisplay.php?fid=4"); 
            postMethod2.setRequestHeader("Host", "discuzdemo.c88.53dns.com"); 
            // postMethod.setRequestHeader("Connection", "keep-alive"); 
            // postMethod.setRequestHeader("Cookie", "jbu_oldtopics=D123D; 
            // jbu_fid2=1249912623; smile=1D1; jbu_onlineusernum=2; 
            // jbu_sid=amveZM"); 
            postMethod2 
                    .setRequestHeader( 
                            "User-Agent", 
                            "Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2");
            postMethod2 
                    .setRequestHeader("Accept", 
                            "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");//以上操作是模拟浏览器的操作，使用服务器混淆 
             
            postMethod2.setRequestBody(data2); 
            httpClient.executeMethod(postMethod2); 
            StringBuffer response1 = new StringBuffer(); 
            BufferedReader reader1 = new BufferedReader(new InputStreamReader( 
                    postMethod2.getResponseBodyAsStream(), "gb2312")); 
            String line1; 
            while ((line1 = reader1.readLine()) != null) { 
                response1.append(line1).append( 
                        System.getProperty("line.separator")); 
            } 
            reader1.close(); 
            System.out.println(response1); 
        } catch (Exception e) { 
            System.out.println(e.getMessage()); 
            // TODO: handle exception 
        } finally { 
            postMethod.releaseConnection(); 
            postMethod2.releaseConnection(); 
        } 

    } 

}