zoukankan      html  css  js  c++  java
  • scrapy数据存入mongodb

    存入mongodb的pipelines文件是这样子写的

    from openpyxl import Workbook
    from scrapy.conf import settings
    import pymongo
    class UsergentPipeline(object):
        def __init__(self):
            host=settings['MONGODB_HOST']
            port=settings['MONGODB_PORT']
            dbname=settings['MONGODB_DBNAME']#数据库名
            client=pymongo.MongoClient(host=host,port=port)
            tdb=client[dbname]
            self.port=tdb[settings['MONGODB_DOCNAME']]#表名
        def process_item(self, item, spider):
            agentinfo=dict(item)
            self.port.insert(agentinfo)
            return item

    settings文件应该这样子写:

    MONGODB_HOST = '192.168.90.9'
    MONGODB_PORT = 27017
    MONGODB_DBNAME = 'Flight'
    MONGODB_DOCNAME = 't_user_gent'
  • 相关阅读:
    Web应用网络模型
    Http协议
    Array数组标准库
    Array数组基础
    javascript--Object
    javascript--Function
    letCode-3
    面试前的准备
    面试常见问题
    面试经验总结
  • 原文地址:https://www.cnblogs.com/caicaihong/p/5944621.html
Copyright © 2011-2022 走看看