zoukankan      html  css  js  c++  java
  • scrapy 管道里面使用mysql插入数据库 python操作mysql

    # -*- coding: utf-8 -*-
    
    # Define your item pipelines here
    #
    # Don't forget to add your pipeline to the ITEM_PIPELINES setting
    # See: https://docs.scrapy.org/en/latest/topics/item-pipeline.html
    import pymysql
    
    
    class QuotesPipeline(object):
    
        def __init__(self):
            self.conn = pymysql.connect(host='xxxxxxx', user='root', passwd='xxxxxxx', db='spider', charset='utf8')
            self.cur = self.conn.cursor()
    
        def open_spider(self,spider):
            print('spider start')
    
        def process_item(self, item, spider):
    
            try:
                import time
                # 格式化成2016-03-20 11:45:39形式
                now = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
                sql = "insert ignore  into test(author,content,created_at) values('{}','{}','{}')".format(item['author'],item['content'],now)
                reCount = self.cur.execute(sql)
                self.conn.commit()
            except Exception as  e:
                print(str(e))
    
            return item

    pip install pymsql 

  • 相关阅读:
    简单状态机
    c语言状态机
    存储公司
    正确跑步
    好好做自己能做的
    I2C学习
    es6 generator函数
    es6 for of 循环
    es6 proxy代理
    es6 Symbol类型
  • 原文地址:https://www.cnblogs.com/brady-wang/p/11792393.html
Copyright © 2011-2022 走看看