Python中读取文件的方式有许多种，往往数据量大的情况令人苦恼。之前一篇博客https://www.cnblogs.com/xiaolan-Lin/p/12014686.html我说会发大文件读取的后续。
那么后续来啦！
1 import pandas as pd 2 # 采用块读取功能，读取csv文件 3 path1 = "./data\tianchi_mobile_recommend_train_user.csv" 4 5 file1 = open(path1) 6 7 user_data = pd.read_csv(path1, sep=',', iterator=True) 8 9 loop = True 10 chunkSize = 1000 11 chunks1 = [] 12 while loop: 13 try: 14 chunk1 = user_data.get_chunk(chunkSize) 15 chunks1.append(chunk1) 16 except StopIteration: 17 loop = False 18 user_data = pd.concat(chunks1, ignore_index=True)