zoukankan      html  css  js  c++  java
  • mysql 批量插入数据过多的解决方法

    使用场景:

    测试时需要插入100w的数据,跑sql脚本插入非常慢。

    存储过程如下:

    //DELIMITER
    DROP PROCEDURE if EXISTS createAmountCount;
    create PROCEDURE createAmountCount()
    BEGIN
    DECLARE i int;
    set i=0;
    drop table  if exists person ;
    create table person(  
    id int not null auto_increment,  
    name varchar(40) not null, 
    age int, 
    primary key(id)  
    )engine=innodb charset=gb2312;
    while i<10000 
    DO
    insert into person (name, age) values(CONCAT('usercode',i));
    set i=i+1;
    end while;
    END;
    //DELIMITER
    CALL createAmountCount();

    解决方案:

    1. 使用sql 生成100w数据到txt文件中。

    public static void main(String[] args) throws IOException {
            // TODO Auto-generated method stub
            BufferedWriter writer = new BufferedWriter(new FileWriter(new File("D:/driver/data.txt"), true));
            for(int i=0;i<1000000;i++){
                if(i%10==0){
                    writer.write("赵"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }if(i%10==1){
                    writer.write("钱"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }
                if(i%10==2){
                    writer.write("孙"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }if(i%10==3){
                    writer.write("李"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }
                if(i%10==4){
                    writer.write("郑"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }if(i%10==5){
                    writer.write("吴"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }
                if(i%10==6){
                    writer.write("周"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }if(i%10==7){
                    writer.write("王"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }
                if(i%10==8){
                    writer.write("张"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }if(i%10==9){
                    writer.write("刘"+(i/10)+"	"+ (int)(Math.random()*100)+"
    ");
                }
            }
            writer.close();
        }

    2. 数据库中将数据导入表中:

    create table person(  
    id int not null auto_increment,  
    name varchar(40) not null, 
    age int, 
    primary key(id)  
    )engine=innodb charset=gb2312;
    
    load data local infile 'D:/driver/data.txt' 
    into table person(name,age);
    
    select count(*) from person;

    耗费时间:

    [SQL]load data local infile 'D:/driver/data.txt' 
    into table person(name,age);
    受影响的行: 1000000
    时间: 10.067s

    本方案的缺点:

    1.权限。 非admin用户没有导入。

    2. 写文件代码。对测试人员来说,写文件不仅仅限于java,可以使用任何语言实现之。

  • 相关阅读:
    September 17th 2016 Week 38th Saturday
    【2016-09-16】UbuntuServer14.04或更高版本安装问题记录
    September 16th 2016 Week 38th Friday
    September 11th 2016 Week 38th Sunday
    September 12th 2016 Week 38th Monday
    September 10th 2016 Week 37th Saturday
    September 9th 2016 Week 37th Friday
    c++暂停
    八皇后问题
    ( 转转)Android初级开发第九讲--Intent最全用法(打开文件跳转页面等)
  • 原文地址:https://www.cnblogs.com/davidwang456/p/4460309.html
Copyright © 2011-2022 走看看