zoukankan      html  css  js  c++  java
  • Spring Batch 之 Sample(复合格式文件的读、多文件的写)(七)

          前面关于Spring Batch的文章,讲述了SpringBatch对CSV文件的读写操作对XML文件的操作,以及对固定长格式文件的操作。这些事例,同一个Reader读取的都是相同格式的数据,最终写入一个文件。如果遇到下面这样的数据,并想将学生信息和商品信息分类后写入两个文件,应该如何处理呢?

    student,200001,ZhangSan,18,78
    goodsPNH001011000200.1zhangshana2011/12/18 01:12:36
    student,200002,LiSi,19,79
    goodsPNH001022000300.1zhangshanb2011/12/19 01:12:36
    student,200003,WangWu,20,80
    goodsPNH001033000400.1zhangshanc2011/12/20 01:12:36

      * 以student开头的数据代表学生信息,以goods开头代表商品信息

    这次将和大家一起探讨Spring Batch读取复合格式的数据,然后写入不同的文件的处理方式。

          工程结构如下图:

           applicationContext.xml和log4j.xml前文已经叙述过,在此不做赘述。

          本实例的核心配置文件batch.mxl内容如下:

      1 <?xml version="1.0" encoding="UTF-8"?>
    2 <bean:beans xmlns="http://www.springframework.org/schema/batch"
    3 xmlns:bean="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    4 xmlns:p="http://www.springframework.org/schema/p" xmlns:tx="http://www.springframework.org/schema/tx"
    5 xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context"
    6 xmlns:util="http://www.springframework.org/schema/util"
    7 xsi:schemaLocation="http://www.springframework.org/schema/beans
    8 http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
    9 http://www.springframework.org/schema/tx
    10 http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
    11 http://www.springframework.org/schema/aop
    12 http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
    13 http://www.springframework.org/schema/context
    14 http://www.springframework.org/schema/context/spring-context-2.5.xsd
    15 http://www.springframework.org/schema/batch
    16 http://www.springframework.org/schema/batch/spring-batch-2.1.xsd
    17 http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd">
    18
    19 <bean:import resource="applicationContext.xml" />
    20 <!-- Job的配置信息 -->
    21 <job id="multiTypeSingleFileJob">
    22 <step id="xmlFileReadAndWriterStep">
    23 <tasklet>
    24 <chunk reader="multiTypesItemReader" writer="multiTypesItemWriter"
    25 commit-interval="1">
    26 <streams>
    27 <stream ref="studentWriter" />
    28 <stream ref="goodsWriter" />
    29 </streams>
    30 </chunk>
    31 </tasklet>
    32 </step>
    33 </job>
    34
    35 <!-- 不同格式数据的文件读取 -->
    36 <bean:bean id="multiTypesItemReader"
    37 class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
    38 <bean:property name="resource"
    39 value="file:#{jobParameters['inputFilePath']}" />
    40 <bean:property name="lineMapper">
    41 <bean:bean
    42 class="org.springframework.batch.item.file.mapping.PatternMatchingCompositeLineMapper">
    43 <bean:property name="tokenizers">
    44 <bean:map>
    45 <bean:entry key="student*" value-ref="studentTokenizer" />
    46 <bean:entry key="goods*" value-ref="goodsTokenizer" />
    47 </bean:map>
    48 </bean:property>
    49 <bean:property name="fieldSetMappers">
    50 <bean:map>
    51 <bean:entry key="student*" value-ref="studentFieldSetMapper" />
    52 <bean:entry key="goods*" value-ref="goodsFieldSetMapper" />
    53 </bean:map>
    54 </bean:property>
    55 </bean:bean>
    56 </bean:property>
    57 </bean:bean>
    58 <bean:bean id="studentTokenizer"
    59 class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
    60 <bean:property name="delimiter" value="," />
    61 <bean:property name="names">
    62 <bean:list>
    63 <bean:value>student</bean:value>
    64 <bean:value>ID</bean:value>
    65 <bean:value>name</bean:value>
    66 <bean:value>age</bean:value>
    67 <bean:value>score</bean:value>
    68 </bean:list>
    69 </bean:property>
    70 </bean:bean>
    71 <bean:bean id="studentFieldSetMapper"
    72 class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
    73 <bean:property name="prototypeBeanName" value="student" />
    74 <bean:property name="distanceLimit" value="100" />
    75 </bean:bean>
    76 <!-- 学生Pojo类 -->
    77 <bean:bean id="student"
    78 class="com.wanggc.springbatch.sample.multitypessinglefile.pojo.Student"
    79 scope="prototype" />
    80
    81 <bean:bean id="goodsTokenizer"
    82 class="org.springframework.batch.item.file.transform.FixedLengthTokenizer">
    83 <bean:property name="columns" value="6-13,14-17,18-22,23-32,33-" />
    84 <bean:property name="names"
    85 value="isin,quantity,price,customer,buyDay" />
    86 </bean:bean>
    87 <bean:bean id="goodsFieldSetMapper"
    88 class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
    89 <bean:property name="prototypeBeanName" value="goods" />
    90 </bean:bean>
    91 <!-- 商品Pojo类 -->
    92 <bean:bean id="goods"
    93 class="com.wanggc.springbatch.sample.multitypessinglefile.pojo.Goods"
    94 scope="prototype" />
    95
    96 <bean:bean id="multiTypesItemWriter"
    97 class="com.wanggc.springbatch.sample.multitypessinglefile.MultiItemWriter">
    98 <bean:property name="delegates">
    99 <bean:list>
    100 <bean:ref bean="studentWriter" />
    101 <bean:ref bean="goodsWriter" />
    102 </bean:list>
    103 </bean:property>
    104 </bean:bean>
    105 <!-- 学生信息的写 -->
    106 <bean:bean id="studentWriter"
    107 class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
    108 <bean:property name="resource"
    109 value="file:#{jobParameters['outputFilePathStudent']}" />
    110 <bean:property name="lineAggregator">
    111 <bean:bean
    112 class="org.springframework.batch.item.file.transform.FormatterLineAggregator">
    113 <bean:property name="fieldExtractor">
    114 <bean:bean
    115 class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
    116 <bean:property name="names" value="ID,name,age,score" />
    117 </bean:bean>
    118 </bean:property>
    119 <bean:property name="format" value="%-9s%-9s%3d%-2.0f" />
    120 </bean:bean>
    121 </bean:property>
    122 </bean:bean>
    123 <!-- 商品信息的写 -->
    124 <bean:bean id="goodsWriter"
    125 class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
    126 <bean:property name="resource"
    127 value="file:#{jobParameters['outputFilePathGoods']}" />
    128 <bean:property name="lineAggregator">
    129 <bean:bean
    130 class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
    131 <bean:property name="fieldExtractor">
    132 <bean:bean
    133 class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
    134 <bean:property name="names"
    135 value="isin,quantity,price,customer,buyDay" />
    136 </bean:bean>
    137 </bean:property>
    138 </bean:bean>
    139 </bean:property>
    140 </bean:bean>
    141 </bean:beans>

           21-33行配置了Job的基本信息。  

          36-57行配置了Reader的基本信息。FlatFileItemReader的lineMapper属性使用SpringBatch核心类PatternMatchingCompositeLineMapper的时候,会将读取的记录按照不同的方式映射成我们的Pojo对象。当然首先我们要配置不同的tokenizers(43-48)和fieldSetMappers(49-54),并告诉它当前的记录按照那条原则去解析和映射。如45行所示,我们指定key为student*的时候,用studentTokenizer去解析成fieldset,用studentFieldSetMapper将studentTokenizer解析好的fieldset记录映射成Student对象。我们指定的key,其实也就是student开头的记录,*是通配符。PatternMatchingCompositeLineMapper支持两种通配符:*和?,前者代表多个字符,后者仅代表一个字符。至于student和goods信息如何映射成pojo对象,前面的文章中已经做过详细的介绍,这里就不做赘述了。

          96-104行配置了Writer的基本信息。Writer也是使用代理的方式,学生信息使用106-122行定义的studentWriter按照固定长的格式写入学生信息文件中,商品信息使用124-141行定义的goodsWriter按照CSV的格式写入商品信息文件中。MultiItemWriter的代码很简单,就不做详细解释了。如下:

    package com.wanggc.springbatch.sample.multitypessinglefile;

    import java.util.ArrayList;
    import java.util.List;

    import org.springframework.batch.item.ItemWriter;

    import com.wanggc.springbatch.sample.multitypessinglefile.pojo.Goods;
    import com.wanggc.springbatch.sample.multitypessinglefile.pojo.Student;

    /**
    * 写处理类。
    *
    *
    @author Wanggc
    *
    *
    @param <T>
    */
    @SuppressWarnings("unchecked")
    public class MultiItemWriter<T> implements ItemWriter<T> {
    /** 写代理 */
    private List<ItemWriter<? super T>> delegates;

    public void setDelegates(List<ItemWriter<? super T>> delegates) {
    this.delegates = delegates;
    }

    @Override
    public void write(List<? extends T> items) throws Exception {
    // 学生信息的Writer
    ItemWriter studentWriter = (ItemWriter) delegates.get(0);
    // 商品信息的Writer
    ItemWriter goodsWriter = (ItemWriter) delegates.get(1);
    // 学生信息
    List<Student> studentList = new ArrayList<Student>();
    // 商品信息
    List<Goods> goodsList = new ArrayList<Goods>();
    // 将传过来的信息按照不同的类型添加到不同的List中
    for (int i = 0; i < items.size(); i++) {
    if ("Student".equals(items.get(i).getClass().getSimpleName())) {
    studentList.add((Student) items.get(i));
    } else {
    goodsList.add((Goods) items.get(i));
    }
    }
    // 如果学生List中有数据,就执行学生信息的写
    if (studentList.size() > 0) {
    studentWriter.write(studentList);
    }
    // 如果商品List中有数据,就执行商品信息的写
    if (goodsList.size() > 0) {
    goodsWriter.write(goodsList);
    }
    }
    }

          至此,复合文件的读写操作已经讨论结束了。注意实例没有配置Processor。下面是一些辅助文件的信息。

          student和goods类的信息与前面文章一样,就不再贴出代码了。

          Job启动的代码如下:

    package com.wanggc.springbatch.sample.multitypessinglefile;

    import org.springframework.batch.core.Job;
    import org.springframework.batch.core.JobExecution;
    import org.springframework.batch.core.JobParametersBuilder;
    import org.springframework.batch.core.launch.JobLauncher;
    import org.springframework.context.ApplicationContext;
    import org.springframework.context.support.ClassPathXmlApplicationContext;

    public class Launch {
    public static void main(String[] args) {
    ApplicationContext context = new ClassPathXmlApplicationContext(
    "batch.xml");
    JobLauncher launcher = (JobLauncher) context.getBean("jobLauncher");
    Job job = (Job) context.getBean("multiTypeSingleFileJob");

    try {
    // JOB实行
    JobExecution result = launcher.run(
    job,
    new JobParametersBuilder()
    .addString("inputFilePath",
    "C:\\testData\\multiTypesInput.txt")
    .addString("outputFilePathStudent",
    "C:\\testData\\student.txt")
    .addString("outputFilePathGoods",
    "C:\\testData\\goods.csv")
    .toJobParameters());
    // 运行结果输出
    System.out.println(result.toString());
    } catch (Exception e) {
    e.printStackTrace();
    }
    }
    }

          Input文件内容如下图:

          处理结果的学生信息文件如下图:

         处理结果的商品信息文件如下图:

          Spring Batch对复合格式文件的读写操作就讨论到这里。至此,Spring Batch对文件简单操作的讨论也告一段落,下次将讨论Spring Batch读写DB的操作。

     

    作者:孤旅者
    如果您对本文有意见或者建议,欢迎留言,哪怕是拍砖(^_^)!
    欢迎转载,请注明出处!
    感谢您的阅读,请关注后续博客!
    共享视频教程请访问:JAVA 高级软件工程师视频
  • 相关阅读:
    Swift3 重写一个带占位符的textView
    Swift3 使用系统UIAlertView方法做吐司效果
    Swift3 页面顶部实现拉伸效果代码
    Swift3 倒计时按钮扩展
    iOS 获取当前对象所在的VC
    SpringBoot在IDEA下使用JPA
    hibernate 异常a different object with the same identifier value was already associated with the session
    SpringCloud IDEA 教学 番外篇 后台运行Eureka服务注册中心
    SpringCloud IDEA 教学 (五) 断路器控制台(HystrixDashboard)
    SpringCloud IDEA 教学 (四) 断路器(Hystrix)
  • 原文地址:https://www.cnblogs.com/gulvzhe/p/2291570.html
Copyright © 2011-2022 走看看