zoukankan      html  css  js  c++  java
  • JobConf

    /**
     * A map/reduce job configuration.
     * 翻译:一个map/reduce作业配置
     * <p><code>JobConf</code> is the primary interface for a user to describe a
     * map-reduce job to the Hadoop framework for execution. The framework tries to
     * faithfully execute the job as-is described by <code>JobConf</code>, however:

     * 翻译:JobConf是用户描述一个Hadoop框架将要执行的map-reduce作业的最基本的接口。框架试图按照JobConf所描述的那样去执行作业,然而:
     * <ol>
     *   <li>
     *   Some configuration parameters might have been marked as
     *   <a href="{@docRoot}/org/apache/hadoop/conf/Configuration.html#FinalParams">
     *   final</a> by administrators and hence cannot be altered.

     *  翻译:一些配置参数可能已经被管理员标记为final,因此不能被改变
     *   </li>
     *   <li>
     *   While some job parameters are straight-forward to set
     *   (e.g. {@link #setNumReduceTasks(int)}), some parameters interact subtly
     *   rest of the framework and/or job-configuration and is relatively more
     *   complex for the user to control finely (e.g. {@link #setNumMapTasks(int)}).

     *   翻译:然而,一些作业参数直接被设置了,比如setNumReduceTasks(int),一些参数和其余的框架或者作业配置进行交互,这些参数相对比较复杂,

                  用户不能很好地控制。
     *   </li>
     * </ol></p>
     *
     * <p><code>JobConf</code> typically specifies the {@link Mapper}, combiner
     * (if any), {@link Partitioner}, {@link Reducer}, {@link InputFormat} and
     * {@link OutputFormat} implementations to be used etc.
     *  翻译:JobConf通常需要指定(说明)将被用到的Mapper、Combiner(如果有的话),Partitioner、Reducer、InputFormat和OutptFormat的实现。

     *
     * <p>Optionally <code>JobConf</code> is used to specify other advanced facets
     * of the job such as <code>Comparator</code>s to be used, files to be put in  
     * the {@link DistributedCache}, whether or not intermediate and/or job outputs
     * are to be compressed (and how), debugability via user-provided scripts
     * ( {@link #setMapDebugScript(String)}/{@link #setReduceDebugScript(String)}),
     * for doing post-processing on task logs, task's stdout, stderr, syslog.
     * and etc.</p>
     * 翻译:JonCOnf可以被用来指定作业将被用到的其他高级配置(高等的方面、高级的方面),比如Comparator。文件被放进DistributedCache,

     * 无论中间输出或者作业输出是否被压缩,它们如何被压缩,通过用户提供脚本来产生的debugability(setMapDebugScript(String)、  setReduceDebugScript(String)),可以后置处理任务日志,任务输出,系统日志等等。
     * <p>Here is an example on how to configure a job via <code>JobConf</code>:</p>

       翻译:这里有一个例子,关于如何使用JobConf来配置一个作业。
     * <p><blockquote><pre>
     *     // Create a new JobConf 翻译:创建一个JobConf
     *     JobConf job = new JobConf(new Configuration(), MyJob.class);
     *     
     *     // Specify various job-specific parameters  翻译:指定一些列作业的具体参数
     *     job.setJobName("myjob");
     *     
     *     FileInputFormat.setInputPaths(job, new Path("in"));
     *     FileOutputFormat.setOutputPath(job, new Path("out"));
     *     
     *     job.setMapperClass(MyJob.MyMapper.class);
     *     job.setCombinerClass(MyJob.MyReducer.class);
     *     job.setReducerClass(MyJob.MyReducer.class);
     *     
     *     job.setInputFormat(SequenceFileInputFormat.class);
     *     job.setOutputFormat(SequenceFileOutputFormat.class);
     * </pre></blockquote></p>
     *
     * @see JobClient
     * @see ClusterStatus
     * @see Tool
     * @see DistributedCache
     */

  • 相关阅读:
    深度学习代码注解(一)—— mnistdeepauto
    只属于你我的共同记忆
    只属于你我的共同记忆
    道教的认识
    道教的认识
    作家、文学大家、大师的艺术风格
    作家、文学大家、大师的艺术风格
    视频、画面、语言、文字与脑海、心灵
    视频、画面、语言、文字与脑海、心灵
    URAL 1963 Kite 四边形求对称轴数
  • 原文地址:https://www.cnblogs.com/mrxiaohe/p/5474211.html
Copyright © 2011-2022 走看看