zoukankan      html  css  js  c++  java
  • redis使用日志(二) 数据存储到redis

    一段简短的代码,来展示如何把爬取内容写到redis里面:

     1 #! /usr/bin/env python
     2 # -*- coding=utf-8 -*-
     3 import requests
     4 import json
     5 import re
     6 import sys
     7 import redis
     8 reload(sys)
     9 sys.setdefaultencoding("utf-8")
    10 classinfo = []
    11 f = open('info.txt','w')
    12 num = 0
    13 class RedisTT(object):
    14     def __init__(self):
    15         self.keyName = 'DouBan'
    16         self.host = 'localhost'
    17         self.port = '6379'
    18         self.re = redis.Redis(host=self.host,port=self.port)
    19     def insertRedis(self,jsonStr):#存入到redis中
    20         self.re.lpush(self.keyName,jsonStr)
    21 class Douban(object):
    22     def write(self,htm):
    23         titl = re.findall('data-tit(.*?)data-enough',htm.text,re.S)
    24         for each in titl:
    25             #print each
    26             info = {}
    27             #print each
    28             info['title'] = re.search('le="(.*?)"',each,re.S).group(1)
    29             info['year'] = re.search('data-release="(.*?)" data',each,re.S).group(1)
    30             info['Rating']= re.findall('data-rate="(.*?)" data-star',each,re.S)[0]
    31             info['time'] = re.findall('data-duration="(.*?)" data-re',each,re.S)[0]
    32             info['reg'] = re.findall('data-region="(.*?)" data-dir',each,re.S)[0]
    33             info['act'] = re.findall('data-actors="(.*?)" data-in',each,re.S)[0]
    34             RedisTT().insertRedis(json.dumps(info))
    35     def getremen(self):
    36         url = 'http://movie.douban.com/'
    37         html = requests.get(url)
    38         html.encoding = 'utf-8'
    39         # print html.text
    40         self.write(html)
    41 if __name__ == "__main__":
    42     Douban().getremen()

     结果如下:

  • 相关阅读:
    HttpClientUtil的工具类请求三方API
    Linux下使用docker搭建ftp服务器
    Springboot2.0.4整合Mybatisplus
    Springboot前后端分离项目,配置跨域
    Nokia S40 系统配置
    "Your profile could not be opened correctly" Google Chromium Browser 错误纠正
    CLR AppDomain
    iTunes下载提速
    Which Programming Language?
    linux下制作软盘镜像文件
  • 原文地址:https://www.cnblogs.com/lovychen/p/5643086.html
Copyright © 2011-2022 走看看