1、数据量较少时 用npoi
网上比较多,不在多说,但是有限制只能读到65000条左右
2、大数据量时采用了OleDb 20W+
(1)具体做法是首先链接一个excel
public static void OleDBHelperStart(string filePath) { string fileType = System.IO.Path.GetExtension(filePath); // if (string.IsNullOrEmpty(fileType)) return null; if (fileType == ".xls") connStr = "Provider=Microsoft.Jet.OLEDB.4.0;" + "Data Source=" + filePath + ";" + ";Extended Properties=\"Excel 8.0;HDR=NO;IMEX=1\""; else connStr = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source=" + filePath + ";" + ";Extended Properties=\"Excel 12.0;HDR=NO;IMEX=1\""; // 初始化连接,并打开 conn = new OleDbConnection(connStr); conn.Open(); }
(2)读取n<65000条数据到DataTable中
/// <summary> /// 直接读取excel到datatable /// </summary> /// <param name="filePath"></param> /// <param name="j">第几次循环</param> /// <param name="leaveflag">剩余标志</param> /// <returns></returns> public static DataTable StartTable(int j, out int leaveflag) { leaveflag = 0; string sql_F = "Select * FROM [{0}]"; DataTable dataTable = new DataTable(); try { da = new OleDbDataAdapter(); da.SelectCommand = new OleDbCommand(String.Format(sql_F, "Sheet1$A" + j * maxNum + ":BF" + ((j + 1) * maxNum-1)), conn); da.Fill(dataTable); if (dataTable.Rows.Count < maxNum-1) leaveflag = 1; } catch (Exception ex) { } finally { // 关闭连接 if (conn.State == ConnectionState.Open) { da.Dispose(); } } da.Dispose(); return dataTable; }
这样就可以一次循环处理了
本地测试,21W,数据读取后还进行一次遍历,批量插入数据库 总耗时 340s