python 操作excel工具及xlsxwriter处理数据量大的方法

前言

- 在我们的web项目中一些图表的预览页面常常伴随着数据导出至excel的功能。在python中我们常常使用xlrd、xlwt、xlutils、openpyxl、xlsxwriter来达成我们的需求。

方法简介

一.xlsxwriter 只写 (推荐)

注：xlsxwriter 只能写，不能读，但功能强大，可进行添加图表等复杂操作，详见https://xlsxwriter.readthedocs.io/

1.本地生成(数据量大时建议保存在本地)

简单的例子

import xlsxwriter as xw
#新建excel，这样在本地会生成这个文件，下文介绍与写入内存中
workbook  = xw.Workbook('myexcel.xlsx')
#新建工作薄
worksheet = workbook.add_worksheet()
#写入数据
worksheet.wirte('A1',1)
#关闭保存
workbook.close()

插入图表的例子

import xlsxwriter
 
def get_chart(series):
    chart = workbook.add_chart({'type': 'line'})
    for ses in series:
        name = ses["name"]
        values = ses["values"]
        chart.add_series({
            'name': name,
            'categories': 'A2:A10',
            'values':values
        })
    chart.set_size({'width': 600, 'height': 250})
    return chart
 
if __name__ == '__main__':
    workbook = xlsxwriter.Workbook('xxxx.xlsx')
    worksheet = workbook.add_worksheet("工作簿1")
    headings = ['日期', '平均值']
    worksheet.write_row('A1', headings)
    index=0
    for row in range(1,10):
        for com in [0,1]:
            worksheet.write(row,com,index)
            index+=1
    series = [{"name":"平均值","values":"B2:B10"}]
    chart = get_chart(series)
    chart.set_title ({'name': '每日页面分享数据'}) 
    worksheet.insert_chart('H7', chart)
    workbook.close()

2.内存生成(web中常用直接返回响应数据)

python3 中操作需 借助 BytesIO， Python2 需借助 StringIO

场景 python3 + django 实现excel导出

from io import BytesIO

import xlsxwriter as xw
from django.http import FileResponse, HttpResponse


def export(request, *arg, **kwargs):
    
    out_put = BytesIO()
    #新建excel
    workbook  = xw.Workbook(out_put)
    #新建工作薄
    worksheet = workbook.add_worksheet()
    #写入数据
    worksheet.wirte('A1',1)
    #关闭保存
    workbook.close()
    # make 响应对象
    response = HttpResponse()
    response['Content-Type'] = "application/octet-stream"
    disposition = 'attachment;filename={}.xlsx'.format(escape_uri_path(file_name))
    response['Content-Disposition'] = disposition
    response.write(out_put.getvalue())
    
    return response

3.xlsxwriter处理大数据

注：默认情况下，XlsxWriter将所有单元数据保存在内存中。当格式化与数据分开应用时，这将允许将来使用功能。这样做的结果是XlsxWriter会消耗大量内存，并且在创建大文件时可能会耗尽内存。

解决办法：

- 我们可以通过Workbook() 的 constant_memory 属性来几乎完全减少内存使用量

workbook = xlsxwriter.Workbook(filename, {'constant_memory': True})

源码：

当设置为True时源码解释        
    # Write out the worksheet data as a single row with cells.
    # This method is used when constant_memory is on. A single
    # row is written and the data table is reset. That way only
    # one row of data is kept in memory at any one time. We don't
    # write span data in the optimized case since it is optional.

官网解释：
# Ok. With 'constant_memory' you must write data in row by column order.
for row in range(0, row_max):
    for col in range(0, col_max):
        worksheet.write(row, col, some_data)

# Not ok. With 'constant_memory' this will only write the first column of data.
for col in range(0, col_max):
    for row in range(0, row_max):
        worksheet.write(row, col, some_data)

缺点：

无法在写入单元数据后利用任何新功能来操纵单元数据。当前，该add_table()方法不适用于此模式，merge_range()并且set_row()仅适用于当前行。

二.openpyxl可以对excel文件进行读写操作

示例1：

from openpyxl import Workbook
from openpyxl import load_workbook
from openpyxl.writer.excel import ExcelWriter 
 
workbook_ = load_workbook(u"test1477881109469.xlsx")
sheetnames =workbook_.get_sheet_names() #获得表单名字
print sheetnames
sheet = workbook_.get_sheet_by_name(sheetnames[0])
print sheet.cell(row=3,column=3).value
sheet['A1'] = '47'
workbook_.save(u"test1477881109469_new.xlsx") 
wb = Workbook()
ws = wb.active
ws['A1'] = 4
wb.save("test.xlsx")

示例2

import openpyxl
# 新建文件
workbook = openpyxl.Workbook()
# 写入文件
sheet = workbook.activesheet['A1']='A1'
# 保存文件
workbook.save('test.xlsx')

三.xlrd 主要是用来读取excel文件

import xlrd
data = xlrd.open_workbook('abcd.xls') # 打开xls文件
table = data.sheets()[0] # 打开第一张表
nrows = table.nrows # 获取表的行数
for i in range(nrows): # 循环逐行打印
    if i == 0:# 跳过第一行
        continue
    print (table.row_values(i)[:13]) # 取前十三列

四.xlutils结合xlrd可以达到修改excel文件目的

import xlrd
from xlutils.copy import copy
workbook = xlrd.open_workbook('xxxxx.xls')
workbooknew = copy(workbook)
ws = workbooknew.get_sheet(0)
ws.write(3, 0, 'changed!')
workbooknew.save('xxxx.xls')

五.xlwt主要是用来写excel文件

import xlwt
wbk = xlwt.Workbook()
sheet = wbk.add_sheet('sheet 1')
sheet.write(0,1,'test text')#第0行第一列写入内容
wbk.save('xxxx.xls')