Spring JDBC 批量操作 数据

通过JDBC 操作数据库 有三种方式  第一种 是 使用 JDBC的JDBC Template ，一种是NamedParameterJdbcTemplate ，一种是 使用 Simple JDBC Insert。

具体的例子 可以 参考下面的 几种 示例。 因为我的 类 是 使用 的 Map  进行传参，而且又不想 使用 占位符 ， 所以变使用了 Simple JDBC Insert

参考文档 ：

1. Spring JDBC 常用批量操作及插入操作  https://m.imooc.com/article/details?article_id=13993

2. Spring - JDBC Batch Update https://www.logicbig.com/tutorials/spring-framework/spring-data-access-with-jdbc/spring-jdbc-batch-update.html

3. jdbctemplate batchupdate 的事务管理 http://www.voidcn.com/article/p-qbkywcbs-te.html

A JDBC batch update is multiple updates using the same database session. That is, we don't have to open connections multiple times.

In our previous example, let's say we want to insert multiple Person objects in the database. Followings are the various ways to do that in Spring.

Using JdbcTemplate

1. batchUpdate(String... sql) : 
   
   

   jdbcTemplate.batchUpdate(
   "insert into PERSON (FIRST_NAME, LAST_NAME, ADDRESS) values ('Dana', 'Whitley', '464 Gorsuch Drive')",
   "insert into PERSON (FIRST_NAME, LAST_NAME, ADDRESS) values ('Robin', 'Cash', '64 Zella Park')"
               );

2. batchUpdate(String sql, List<Object[]> batchArgs) : 
   
   

   jdbcTemplate.batchUpdate(
           "insert into PERSON (FIRST_NAME, LAST_NAME, ADDRESS) values (?, ?, ?)",
           Arrays.asList(new Object[]{"Dana", "Whitley", "464 Gorsuch Drive"},
                         new Object[]{"Robin", "Cash", "64 Zella Park"})
    );

3. batchUpdate(String sql, List <Object[]> batchArgs, int[] argTypes) : 
   
   

   jdbcTemplate.batchUpdate(
           "insert into PERSON (FIRST_NAME, LAST_NAME, ADDRESS) values (?, ?, ?)",
           Arrays.asList(new Object[]{"Dana", "Whitley", "464 Gorsuch Drive"},
                         new Object[]{"Robin", "Cash", "64 Zella Park"}),
                   new int[]{Types.VARCHAR, Types.VARCHAR, Types.VARCHAR}
    );

4. batchUpdate(String sql, BatchPreparedStatementSetter pss) 
   
   

   final List<Person> persons = Arrays.asList(
           Person.create("Dana", "Whitley", "464 Gorsuch Drive"),
           Person.create("Robin", "Cash", "64 Zella Park")
   );
   
   String sql = "insert into Person (first_Name, Last_Name, Address) values (?, ?, ?)";
   
   int[] updateCounts = jdbcTemplate.batchUpdate(sql, new BatchPreparedStatementSetter() {
       @Override
       public void setValues(PreparedStatement ps, int i) throws SQLException {
           ps.setString(1, persons.get(i).getFirstName());
           ps.setString(2, persons.get(i).getLastName());
           ps.setString(3, persons.get(i).getAddress());
       }
   
       @Override
       public int getBatchSize() {
           return persons.size();
       }
   });

5. batchUpdate(String sql, Collection<T> batchArgs, int batchSize, ParameterizedPreparedStatementSetter <T> pss). This method can break the batch updates into serveral smaller batches specified by batchSize. 
   
   

   final List<Person> persons = Arrays.asList(
           Person.create("Dana", "Whitley", "464 Gorsuch Drive"),
           Person.create("Robin", "Cash", "64 Zella Park")
   );
   
   String sql = "insert into Person (first_Name, Last_Name, Address) values (?, ?, ?)";
   
   int[][] updateCounts = jdbcTemplate.batchUpdate(sql, persons, persons.size(),
   
           new ParameterizedPreparedStatementSetter<Person>() {
               @Override
               public void setValues(PreparedStatement ps, Person person) throws SQLException {
                   ps.setString(1, person.getFirstName());
                   ps.setString(2, person.getLastName());
                   ps.setString(3, person.getAddress());
               }
           });

Using NamedParameterJdbcTemplate

1. batchUpdate(String sql, Map <String,?>[] batchValues) 
   
   

   List<Person> persons = Arrays.asList(
           Person.create("Dana", "Whitley", "464 Gorsuch Drive"),
           Person.create("Robin", "Cash", "64 Zella Park")
   );
   
   String sql = "insert into Person (first_Name, Last_Name, Address) " +
           "values (:firstName, :lastName, :address)";
   
   List<Map<String, Object>> batchValues = new ArrayList<>(persons.size());
   for (Person person : persons) {
       batchValues.add(
               new MapSqlParameterSource("firstName", person.getFirstName())
                       .addValue("lastName", person.getLastName())
                       .addValue("address", person.getAddress())
                       .getValues());
   }
   
   int[] updateCounts = namedParamJdbcTemplate.batchUpdate(sql,
                              batchValues.toArray(new Map[persons.size()]));

2. batchUpdate(String sql, SqlParameterSource[] batchArgs) 
   
   

   List<Person> persons = Arrays.asList(
           Person.create("Dana", "Whitley", "464 Gorsuch Drive"),
           Person.create("Robin", "Cash", "64 Zella Park")
   );
   
   String sql = "insert into Person (first_Name, Last_Name, Address) " +
           "values (:firstName, :lastName, :address)";
   
   SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(persons.toArray());
   int[] updateCounts = namedParamJdbcTemplate.batchUpdate(sql, batch);

Using SimpleJdbcInsert

1. executeBatch(Map<String,?>... batch)  对大小写 不敏感 ， 只要 Columns 中有 数值 就可以 
   
   

   List<Map<String, Object>> batchValues = new ArrayList<>(persons.size());
   for (Person person : persons) {
    Map<String, Object> map = new HashMap<>();
     map.put("first_Name", person.getFirstName());
     map.put("last_Name", person.getLastName());
     map.put("address", person.getAddress());
   
    batchValues.add(map);
     }
   
    SimpleJdbcInsert simpleJdbcInsert = new SimpleJdbcInsert(dataSource).withTableName("PERSON");
   
    int[] ints = simpleJdbcInsert.executeBatch(batchValues.toArray(new Map[persons.size()]));

2. public int[] executeBatch(SqlParameterSource... batch). We don't have to specify any column to bean field name mapping. It can figure out the differences like underscores in the table column names. For example bean#firstName is automatically mapped to column FIRST_NAME. 
   
   

   List<Person> persons = Arrays.asList(
           Person.create("Dana", "Whitley", "464 Gorsuch Drive"),
           Person.create("Robin", "Cash", "64 Zella Park")
   );
   
   SimpleJdbcInsert simpleJdbcInsert = new SimpleJdbcInsert(dataSource)
                                             .withTableName("PERSON")
                                             .usingGeneratedKeyColumns("id");
   
   SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(persons.toArray());
   int[] ints = simpleJdbcInsert.executeBatch(batch);

3. 我自己的示例 ，首先我需要 多批次的操作，其次 我已经 封装好了 一个 Map  使用 Java 8 的 流的 方式 将Map 的list 转换 为 Map 数组。

在这里 我就碰到了 这个坑 如果是 传统的 list.toArray() , 那么 其实 转换出来的 是一个 Object 数组， 这样子使用的 方式 是 Bean  命名 的方式。

for ( int i = 0 ; i < list.size() ; i +=batchSize ) {
    final List<Map<String, Object>> batchList = list.subList(i, i + batchSize > list.size() ? list.size() : i + batchSize);
batchList.stream().forEach(putMapValue(batchId, site));
Map[] maps = batchList.stream().toArray(Map[]::new);
SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(maps);
jdbcInsert.executeBatch(batch);
}

private Consumer<Map<String, Object>> putMapValue(String batchId, String site) {
    return map -> {
        map.put("UUID", UUID.randomUUID().toString());
map.put("LAST_BATCH_ID",batchId);
map.put("SITE", site);
};
}

for ( int i = 0 ; i < list.size() ; i +=batchSize ) {
    final List<Map<String, Object>> batchList = list.subList(i, i + batchSize > list.size() ? list.size() : i + batchSize);
batchList.stream().forEach(putMapValue(batchId, site));
Map[] maps = batchList.stream().toArray(Map[]::new);
SqlParameterSource[] batch = SqlParameterSourceUtils.createBatch(maps);
jdbcInsert.executeBatch(batch);
}