python 爬虫 ~ 查看收发包的情况

DebugLog

　　可以用来查看收发包的情况，比较有意思，现特意记录下来：

Sample:

import urllib2
httpHandler = urllib2.HTTPHandler(debuglevel=1)
httpsHandler = urllib2.HTTPSHandler(debuglevel=1)
opener = urllib2.build_opener(httpHandler, httpsHandler)
urllib2.install_opener(opener)
response = urllib2.urlopen('http://www.baidu.com')

运行结果：

send: 'GET http://www.baidu.com HTTP/1.1
Accept-Encoding: identity
Host: www.baidu.com
Connection: close
User-Agent: Python-urllib/2.7

'
reply: 'HTTP/1.1 200 
'
header: Content-Length: 111470
header: Bdqid: 0xac52fd1000001301
header: X-Powered-By: HPHP
header: Set-Cookie: BAIDUID=C79A798AE88AE217892800990B29F9AF:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com
header: Set-Cookie: BIDUPSID=C79A798AE88AE217892800990B29F9AF; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com
header: Set-Cookie: PSTM=1506586983; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com
header: Set-Cookie: BDSVRTM=0; path=/
header: Set-Cookie: BD_HOME=0; path=/
header: Set-Cookie: H_PS_PSSID=1442_21084_17001_22072; path=/; domain=.baidu.com
header: Expires: Thu, 28 Sep 2017 08:22:20 GMT
header: Bduserid: 0
header: Server: BWS/1.1
header: Cxy_All: baidu+93815e78c13d58fab2103ab02b94c0d4
header: Date: Thu, 28 Sep 2017 08:23:03 GMT
header: P3P: CP=" OTI DSP COR IVA OUR IND COM "
header: Content-Type: text/html; charset=utf-8
header: Bdpagetype: 1
header: X-Ua-Compatible: IE=Edge,chrome=1

  

References:

　　Python爬虫入门（4）：Urllib库的高级用法