zoukankan      html  css  js  c++  java
  • JPA中以HibernatePersistence为provider的批量插入问题

    为什么要批量插入

    要插入10000条数据,如果不批量插入的话,那么我们执行的sql语句将是10000条insert

    insert into member (group_id, user_id, role, extend) values (101, 100, 3, NULL)
    insert into member (group_id, user_id, role, extend) values (101, 101, 3, NULL)
    insert into member (group_id, user_id, role, extend) values (101, 102, 3, NULL)
    insert into member (group_id, user_id, role, extend) values (101, 103, 3, NULL)

    ......

    如果是批量插入,一批次插入50条,那么10000条数据将生成200条insert语句

    insert into member (group_id, user_id, role, extend) VALUES (101, 100, 3, NULL), (101, 101, 3, NULL), ..., (101, 149, 3, NULL)

    insert into member (group_id, user_id, role, extend) VALUES (101, 100, 3, NULL), (101, 101, 3, NULL), ..., (101, 149, 3, NULL)

    好了,以上是jdbc级别的批量插入,换言之,我们需要JPA提供批量插入功能

    JPA批量插入

    搜索了Stackoverflow,似乎JPA本身并没有对批量插入的支持

    http://stackoverflow.com/questions/7440397/how-to-do-the-batch-insert-in-jpa

    Hibernate批量插入

    因为使用Hibernate作为JPA的provider实现,因此我们可以使用Hibernate提供的批量插入功能

    查阅hibernate文档

    http://docs.jboss.org/hibernate/core/3.3/reference/en/html/batch.html

    (1) 首先要在hibernate配置文件中配置属性

    <property name="hibernate.jdbc.batch_size" value="20" /> <!-- value文档建议在10-50间 -->

    (2) 批量插入代码示例

    1 for ( int i=0; i<100000; i++ ) {
    2     Customer customer = new Customer(.....);
    3     session.save(customer);
    4     if ( i % 20 == 0 ) { //20, same as the JDBC batch size
    5         //flush a batch of inserts and release memory:
    6         session.flush();
    7         session.clear();
    8     }
    9 }

    注意

    如果批量插入的实体主键为@GeneratedValue(strategy=IDENTITY)类型,则hibernate会透明关闭在jdbc级别的批量插入;因为实际上,每次插入该实体的一个实例,hibernate都需要搜索该实例的主键;想想在代码中,我们persist实例之后,就能够通过get方法获取实例的主键;

    实验结果也说明了这点

    实体定义

     1 @Entity
     2 @Table (name="member")
     3 public class GroupUser {
     4   @Id
     5   @GeneratedValue(strategy=IDENTITY)
     6   @Column(name="id", nullable=true, unique=true)
     7   private Integer id;
     8 
     9   ......
    10 
    11 }

    实验代码

     1 Session session = (Session)em.getDelegate();
     2 session.setFlushMode(FlushMode.MANUAL);
     3 
     4 for (int i = 0; i < userIds.size(); ++i) {
     5   session.save(new GroupUser(groupId, userIds.get(i), role));
     6   if (i % 20 == 0) {
     7     session.flush();
     8     session.clear();
     9   }
    10 }

    日志输出

    Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
    Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
    Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
    Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
    ......

  • 相关阅读:
    IDEA 现有项目连接SVN
    图片预加载,懒加载
    element-ui table中排序 取消表格默认排序问题
    Linux vim编辑命令
    linux mount 硬盘挂载和卸载
    Linux mke2fs 硬盘格式化
    Linux rm 删除文件
    linux 创建和删除目录
    linux 配置IP
    kickstart自动化安装--tftp+nfs+dhcp
  • 原文地址:https://www.cnblogs.com/hzhesi/p/4234247.html
Copyright © 2011-2022 走看看