zoukankan      html  css  js  c++  java
  • Hadoop自定义输出文件名

    在job页面添加一条代码:

    job.setOutputFormatClass(num.class);
    新建一个类,里面继承 TextOutputFormat 类复写 getDefaultWorkFile 方法.
    return new Path(committer.getWorkPath(), "num");中更改输出文件的名字
    package com.test.map;
    import java.io.IOException;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.mapreduce.RecordWriter;
    import org.apache.hadoop.mapreduce.TaskAttemptContext;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter;
    import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
    public class num extends TextOutputFormat{
        @Override
        public Path getDefaultWorkFile(TaskAttemptContext context, String extension) throws IOException {
            FileOutputCommitter committer = (FileOutputCommitter) getOutputCommitter(context);
            return new Path(committer.getWorkPath(), "num");//getOutputName(context)
        }
    }
  • 相关阅读:
    AC自动机模板
    2013 ACM/ICPC Asia Regional Changsha Online–C (模拟)
    Codeforces126B
    Codeforces182D
    Codeforces149E
    POJ3080
    POJ2752
    HDU4745
    HDU4737
    POJ1226
  • 原文地址:https://www.cnblogs.com/ideaAI/p/13829979.html
Copyright © 2011-2022 走看看