zoukankan      html  css  js  c++  java
  • Requests库练习

    实例一:京东商品页面爬取
    import requests
    url = "http://item.jd.com/2967929.html"
    try:
    r = requests.get(url)
    r.raise_for_status()
    r.encoding = r.apparent_encoding
    print(r.text[:1000])
    except:
    print("爬取失败")
    实例二:亚马逊商品页面爬取
    import requests
    url = "https://www.amazon.cn/gp/product/B01M8L5Z3Y"
    try:
    kv = {'user_agent':'Mozilla/5.0'}
    r = requests.get(url,headers=kv)
    r.raise_for_status()
    r,encoding = r.apparent_encoding
    print(r.text[1000:2000])
    except:
    print("爬取失败")
    实例三:百度/360搜索关键字提交
    import requests
    keyword = "Python"
    try:
    kv = {'q':keyword}
    r = requests.get("http://www.so.com/s",params = kv)
    print(r.request.url)
    r.raise_for_status()
    print(len(r.text))
    except:
    print("爬取失败")
    实例四:网络图片的爬取和存储
    import requests
    import os
    url = ""
    root = "D://pics//"
    path = root + url.split('/')[-1]
    try:
    if not os.path.exists(root):
    os.mkdir(root)
    if not os.path.exists(path):
    r = requests.get(url)
    with open(path,'wb') as f :
    f.write(r.content)
    f.close()
    print("sucess")
    else:
    print("cunzai")
    except:
    print("爬取失败")
    实例五:IP地址归属地的自动查询
    import requests
    url = "http://m.ip138.com/ip.asp?ip="
    try:
    r = requests.get(url+'202.204.80.112')
    r.raise_for_status()
    r.encoding = r.apparent_encoding
    print(r.text[-500:])
    except:
    print("爬取失败")
  • 相关阅读:
    线段树入门总结
    从零基础学三分查找
    Codeforces Beta Round #1 A,B,C
    isupper()函数
    matlab字符串操作总结
    hdu 4873 ZCC Loves Intersection(大数+概率)
    设计模式入门之桥接模式Bridge
    有关UIWebView的SSL总结
    vmware虚拟机上linux操作系统进行tty1~tty6切换方法和具体步骤
    Python BeautifulSoup4 使用指南
  • 原文地址:https://www.cnblogs.com/lskai/p/11910912.html
Copyright © 2011-2022 走看看