zoukankan      html  css  js  c++  java
  • JobClient

    /**
     * <code>JobClient</code> is the primary interface for the user-job to interact
     * with the {@link JobTracker}.
     * 翻译:JobClient是用户的作业与JobTracker进行交互的最基本接口
     * <code>JobClient</code> provides facilities to submit jobs, track their
     * progress, access component-tasks' reports/logs, get the Map-Reduce cluster
     * status information etc.
     * 翻译:JobClient提供提交作业的工具,追踪作业的进度,获取component-tasks(合成任务)的日志,获取Map-Reduce集群状态信息等等。
     * <p>The job submission process involves:翻译:作业提交过程包括如下
     * <ol>
     *   <li>
     *   Checking the input and output specifications of the job.翻译:检测作业的输入和输入描述
     *   </li>
     *   <li>
     *   Computing the {@link InputSplit}s for the job.翻译:计算作业的InputSplit
     *   </li>
     *   <li>
     *   Setup the requisite accounting information for the {@link DistributedCache}
     *   of the job, if necessary.

     *   翻译:如果有必要的话,为作业的DistributedCache设置必要的accounting information
     *   </li>
     *   <li>
     *   Copying the job's jar and configuration to the map-reduce system directory
     *   on the distributed file-system.

     *   翻译:拷贝作业的jar文件和配置文件到分布式文件系统里的map-reduce系统目录
     *   </li>
     *   <li>
     *   Submitting the job to the <code>JobTracker</code> and optionally monitoring
     *   it's status.

     *  翻译:提交作业到JobTracker,并选择性的监控它的状态
     *   </li>
     * </ol></p>
     *  
     * Normally the user creates the application, describes various facets of the
     * job via {@link JobConf} and then uses the <code>JobClient</code> to submit
     * the job and monitor its progress.
     * 翻译:通常用户创建应用程序,通过JobConf来描述作业的各个方面,并且用JobClient来提交作业,并监视它的进度
     * <p>Here is an example on how to use <code>JobClient</code>:</p>翻译:这里有一个例子,教你如何使用JobClient
     * <p><blockquote><pre>
     *     // Create a new JobConf 翻译:创建一个JobConf对象
     *     JobConf job = new JobConf(new Configuration(), MyJob.class);
     *     
     *     // Specify various job-specific parameters   翻译:指定各种各样的和作业有关的具体参数
     *     job.setJobName("myjob");
     *     
     *     job.setInputPath(new Path("in"));
     *     job.setOutputPath(new Path("out"));
     *     
     *     job.setMapperClass(MyJob.MyMapper.class);
     *     job.setReducerClass(MyJob.MyReducer.class);
     *
     *     // Submit the job, then poll for progress until the job is complete翻译:提交作业,不停的询问进度,知道作业完成
     *     JobClient.runJob(job);
     * </pre></blockquote></p>
     *
     * <h4 id="JobControl">Job Control</h4>
     *
     * <p>At times clients would chain map-reduce jobs to accomplish complex tasks
     * which cannot be done via a single map-reduce job. This is fairly easy since
     * the output of the job, typically, goes to distributed file-system and that
     * can be used as the input for the next job.</p>
     * 翻译:有时,clients会把许多的map-reduce作业“链”在一起,取完成一些复杂的任务,这些作业是不能通过一个单一的map-reduce作业来完成的。

        这是非常容易实现的,因为作业的输出通常是在分布式文件系统,所以这些在分布式文件系统的输出可以用作下一个作业的输入。
     * <p>However, this also means that the onus on ensuring jobs are complete
     * (success/failure) lies squarely on the clients. In such situations the
     * various job-control options are:

     * 然而,这也意味着,确保作业成功或者失败的重任直接就落在了clients上。在这种情况下,job-control选项如下:
     * <ol>
     *   <li>
     *   {@link #runJob(JobConf)} : submits the job and returns only after
     *   the job has completed.翻译:提交作业,并且只有在作业完成之后返回。
     *   </li>
     *   <li>
     *   {@link #submitJob(JobConf)} : only submits the job, then poll the
     *   returned handle to the {@link RunningJob} to query status and make
     *   scheduling decisions.

     *   翻译:仅提交作业,此时,通过RunningJob(<p>Clients can get hold of <code>RunningJob</code> via the {@link JobClient}
     * and then query the running-job for details such as name, configuration,
     * progress etc.</p> )不停的请求句柄,来查询状态和调度决策
     *   </li>
     *   <li>
     *   {@link JobConf#setJobEndNotificationURI(String)} : setup a notification
     *   on job-completion, thus avoiding polling.

     *   翻译:设置一个作业完成通知,因此就可以避免不停的询问进度
     *   </li>
     * </ol></p>
     *
     * @see JobConf
     * @see ClusterStatus
     * @see Tool
     * @see DistributedCache
     */

  • 相关阅读:
    待续中的图灵机与有限状态机
    barabasilab-networkScience学习笔记6-evolving networks
    pythonyCool-moviepy
    各种方法合成地震图的对应单位
    matla互相关协方差的计算和理解
    matlab运行中出现“Caught "std::exception" Exception message is: Message Catalog MATLAB:builtins was not loaded from the file."
    对拉普拉斯平滑 的认识
    关于小波变换
    随笔学习
    关于滤波频段的选取(近震波形)
  • 原文地址:https://www.cnblogs.com/mrxiaohe/p/5468625.html
Copyright © 2011-2022 走看看