zoukankan      html  css  js  c++  java
  • 正则表达式爬取猫眼电影

    正则表达式爬取猫眼电影Top100

    import requests
    import re,json
    from multiprocessing import Pool
    
    def get_one_page(url):
        response = requests.get(url)
        return response.text
    
    def parse_one_page(html):
        pattern = re.compile('<dd>.*?board-index.*?>(d+)</i>.*?data-src="(.*?)".*?name"><a'
                             +'.*?>(.*?)</a>.*?star">(.*?)</p>.*?releasetime">(.*?)</p>'
                              +'.*?integer">(.*?)</i>.*?fraction">(.*?)</i>.*?</dd>',re.S)
    
        items = re.findall(pattern,html)
        for item in items:
            yield {
                'index':item[0],
                'image':item[1],
                'title':item[2],
                'actor':item[3].strip()[3:],
                'time':item[4].strip()[5:],
                'score':item[5]+item[6]
            }
    
    def write_to_file(content):
        with open('maoyan.txt','a',encoding='utf-8') as f:
            f.write(json.dumps(content,ensure_ascii=False) + '
    ')
            f.close()
    
    def main(offset):
        url = 'http://maoyan.com/board/4?offset=' + str(offset)
        html = get_one_page(url)
        for item in parse_one_page(html):
            write_to_file(item)
    
    if __name__ == '__main__':
        # for i in range(10):
        #     main(i*10)
        #使用进程池
        pool = Pool()
        pool.map(main,[i*10 for i in range(10)])
     
     
     
  • 相关阅读:
    书单
    parzen 窗的matlab实现
    Oracle-07
    图片加载不出来
    省市区三表建立及数据插入
    a链接与button按钮混用
    省市区三级联动,获取地区编号,JSON格式
    纯JS省市区三级联动
    JDK与JRE的区别
    web-03-jQuery
  • 原文地址:https://www.cnblogs.com/gaidy/p/12096797.html
Copyright © 2011-2022 走看看