zoukankan      html  css  js  c++  java
  • Python访问MongoDB,并且转换成Dataframe

    #!/usr/bin/env python
    # -*- coding: utf-8 -*-
    # @Time    : 2018/7/13 11:10
    # @Author  : baoshan
    # @Site    : 
    # @File    : pandans_pymongo.py
    # @Software: PyCharm Community Edition
    
    import pymongo
    import pandas as pd
    
    
    def _connect_mongo(host, port, username, password, db):
        """ A util for making a connection to mongo. """
        if username and password:
            mongo_uri = "mongodb://%s:%s@%s:%s/%s" % (username, password, host, port, db)
            conn = pymongo.MongoClient(mongo_uri)
        else:
            conn = pymongo.MongoClient(host, port)
    
        return conn[db]
    
    
    def read_mongo(db, collection, query={}, host='test43', port=27017, username=None, password=None, no_id=True):
        """ Read from Mongo and Store into DataFrame. """
    
        # Connect to MongoDB
        db = _connect_mongo(host=host, port=port, username=username, password=password, db=db)
    
        # Make a query to the specific DB and Collection
        cursor = db[collection].find(query).limit(10)
    
        # Expand the cursor and construct the DataFrame
        df = pd.DataFrame(list(cursor))
        df.to_csv("abc.csv", encoding="utf_8_sig")  # 处理中文乱码问题
    
        if no_id:
            del df['_id']
    
        return df
    
    
    read_mongo(db='service', collection='trace_log_regular', query={}, host='xxx', port=27017, username="xxx", password="xxx")

     小结:

    1. 解决了pymongo访问MongoDB的问题

    2. 解决了查询的数据转成dataframe的问题

    3. 解决了dataframe写入到csv的问题

    4. 解决了中文乱码问题。

    谢谢

    【参考】https://stackoverflow.com/questions/16249736/how-to-import-data-from-mongodb-to-pandas

  • 相关阅读:
    几个数之和----数组刷题
    单调栈刷题
    腾讯金融科技凉经
    mysql刷题
    链表类题目常用方法
    阿里云一面凉经
    腾讯TEG一面凉经
    腾讯软件开发-后台开发实习生-一面凉经
    剑指 Offer 19. 正则表达式匹配
    剑指 Offer 20. 表示数值的字符串
  • 原文地址:https://www.cnblogs.com/zhzhang/p/9305920.html
Copyright © 2011-2022 走看看