zoukankan      html  css  js  c++  java
  • python pandas使用一些协程

    import pandas as pd
    
    def coroutine(func):
    	"""装饰器:向前执行到第一个`yield`表达式,预激`func`"""
    	@wraps(func)
    	def primer(*args,**kwargs):
    		gen = func(*args,**kwargs)
    		next(gen)
    		return gen
    	primer.__name__ = func.__name__
    	primer.__dict__ = func.__dict__
    	primer.__doc__  = func.__doc__
    	return primer
    
    @coroutine
    def getd():
    	grouped=pd.DataFrame()
    	while True:
    		chunk=yield grouped
    		if chunk is None:
    			break
    		#chunk['cishu']=1
    		chunkgrouped=chunk.groupby('somekey',as_index=False).sum()
    		newchunk=pd.concat([grouped,chunkgrouped],ignore_index=True)
    		grouped=newchunk.groupby('somekey', as_index=False).sum()
    	return grouped
    
    cor=getd()
    chunks=pd.read_csv(path,low_memory=False,dtype='object',chunksize=10)
    for chunk in chunks:
    	cor.send(chunk)
    try:
    	cor.send(None)
    except StopIteration as exc:
    	result = exc.value
    	print(result)
    

      

  • 相关阅读:
    (14) go 结构体
    (13) go map
    (12) go make初始化
    (11)go 数组和切片
    (10) go 错误
    (9) go 时间日期
    (8)go 字符串
    (7) go 函数
    (6) go 流程控制
    (5) go 格式化输入输出 类型转换
  • 原文地址:https://www.cnblogs.com/mahailuo/p/10209740.html
Copyright © 2011-2022 走看看