zoukankan      html  css  js  c++  java
  • python爬虫:将数据保存到本地

    一.python语句存储

    1.with open()语句

    1 with open(name,mode,encoding) as file:
    2   file.write()

    name:包含文件名称的字符串; 
    mode:决定了打开文件的模式,只读/写入/追加等; 
    encoding:表示我们要写入数据的编码,一般为 utf-8 或者 gbk ; 
    file:表示我们在代码中对文件的命名。

    2.w:只写模式,如果没有文件则自动创建

    1 f.write("{} {} {} {}
    ".format(title,price,scrible,pic))

    3.例子

    1)

    1 with open('a.txt','wb') as f:
    2     for tag in soup.find_all('div',class_='service-item-pic'):
    3         a_url = tag.find('a').get('href')
    4         f.write(a_url)
    5         f.write('
    ')

    2)

    1 for tag in soup.find_all('div',class_='service-item-pic'):
    2     with open('3.txt', 'a') as f:             
    3         a_url = tag.find('a').get('href')
    4         f.write(a_url)
    5         f.write('
    ')

    二.保存图片

    1.方法

    首先用Beautiful Soup结合正则表达式的方式来提取所有链接:

    1 links = soup.find_all('img', "origin_image zh-lightbox-thumb",src=re.compile(r'.jpg$'))

    提取出所有链接后,使用request.urlretrieve来将所有链接保存到本地

    2.例子

     1 import time
     2 from urllib import request
     3 from bs4 import BeautifulSoup
     4 import re
     5 url = r'https://www.zhihu.com/question/355015346/answer/892031308'
     6 headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'}
     7 page = request.Request(url, headers=headers)
     8 page_info = request.urlopen(page).read().decode('utf-8')
     9 soup = BeautifulSoup(page_info, 'html.parser')
    10 links = soup.find_all('img', "origin_image zh-lightbox-thumb",src=re.compile(r'.jpg$'))
    11 local_path = r'C:UsersAdministratorDesktop'
    12 for link in links:
    13     print(link.attrs['src'])
    14     request.urlretrieve(link.attrs['src'], local_path+r'\%s.jpg' % time.time())

     

  • 相关阅读:
    截图软件FastStone
    java中继承thread类的其他类的start()方法与run()方法
    .net连接MySql 出错
    题解【Codeforces1139C】Edgy Trees
    题解【Codeforces580C】Kefa and park
    题解【Codeforces1234D】Distinct Characters Queries
    题解【洛谷P4025】[PA2014]Bohater
    题解【洛谷P1445】[Violet]樱花
    题解【洛谷P2516】[HAOI2010]最长公共子序列
    题解【洛谷P3275】[SCOI2011]糖果
  • 原文地址:https://www.cnblogs.com/huanghuangwei/p/11866925.html
Copyright © 2011-2022 走看看