zoukankan      html  css  js  c++  java
  • 从零自学Hadoop(13):Hadoop命令下

    阅读目录

    本文版权归mephisto和博客园共有,欢迎转载,但须保留此段声明,并给出原文链接,谢谢合作。

    文章是哥(mephisto)写的,SourceLink

      上一篇,我们对Hadoop命令中的HDFS Commands进行了简略的列举,下面我们就MapReduce Commands和Yarn Commands进行简略的列举,官网基本都是英文的,所以只能拙略的翻译下,妄大家见谅。

      下面,我们就开始对Hadoop命令下进行讲解。

    MapReduce Commands

    一:介绍

      所有的MapReduce命令通过bin/ mapred脚本调用。不指定参数运行mapred脚本会打印所有命令的描述。

      用法: mapred [SHELL_OPTIONS] COMMAND [GENERIC_OPTIONS] [COMMAND_OPTIONS]

      Hadoop有一个选项解析框架用于解析一般的选项和运行类。

    COMMAND_OPTIONSDescription
    SHELL_OPTIONS The common set of shell options. These are documented on the Hadoop Commands Reference page.
    GENERIC_OPTIONS The common set of options supported by multiple commands. See the Hadoop Commands Reference for more information.
    COMMAND COMMAND_OPTIONS Various commands with their options are described in the following sections. The commands have been grouped into User Commands and Administration Commands.

    User Commands

    一:介绍

      用于Hadoop集群用户命令。

    二:archive

      创建一个Hadoop档案.More information can be found at Hadoop Archives Guide.

    三:classpath

      打印Hadoop jar和所需的libs的class路径。

      用法: mapred classpath

    四:distcp

      递归复制文件或目录. More information can be found at Hadoop DistCp Guide.

    五:job

      MapReduce jobs的命令

      用法: mapred job | [GENERIC_OPTIONS] | [-submit <job-file>] | [-status <job-id>] | [-counter <job-id> <group-name> <counter-name>] | [-kill <job-id>] | [-events <job-id> <from-event-#> <#-of-events>] | [-history [all] <jobOutputDir>] | [-list [all]] | [-kill-task <task-id>] | [-fail-task <task-id>] | [-set-priority <job-id> <priority>]

    六:pipes

      运行一个pipes job.

      用法: mapred pipes [-conf <path>] [-jobconf <key=value>, <key=value>, ...] [-input <path>] [-output <path>] [-jar <jar file>] [-inputformat <class>] [-map <class>] [-partitioner <class>] [-reduce <class>] [-writer <class>] [-program <executable>] [-reduces <num>]

    七:queue

      查看作业队列信息

      用法:  mapred queue [-list] | [-info <job-queue-name> [-showJobs]] | [-showacls]

    八:version

      打印版本。

      用法:  mapred version

    Administration Commands

    一:介绍

      用于Hadoop集群用户命令。

    二:historyserver

        启动JobHistoryServer.

      用法: mapred historyserver

    三:hsadmin

      运行一个MapReduce hsadmin客户端执行jobhistoryserver administrative命令。

      用法:Usage: mapred hsadmin [-refreshUserToGroupsMappings] | [-refreshSuperUserGroupsConfiguration] | [-refreshAdminAcls] | [-refreshLoadedJobCache] | [-refreshLogRetentionSettings] | [-refreshJobRetentionSettings] | [-getGroups [username]] | [-help [cmd]]

    YARN Commands

    一:介绍

      所有的YARN命令通过bin/yarn 脚本调用。不指定参数运行yarn脚本会打印所有命令的描述。

      用法: yarn [--config confdir] COMMAND [--loglevel loglevel] [GENERIC_OPTIONS] [COMMAND_OPTIONS]

      YARN有一个选项解析框架用于解析一般的选项和运行类。

    User Commands

    一:介绍

      用于Hadoop集群用户命令。

    二:application

      打印app报表或kill掉app

      用法: yarn application [options]

    三:applicationattempt

      打印applicationattempt(s) 报表

      用法: yarn applicationattempt [options]

    四:classpath

      打印Hadoop jar和需要的库类路径。

      用法: yarn classpath

    五:container

      打印container(s)报表

      用法: yarn container [options]

    六:jar

      执行一个jar文件。

      用法: yarn jar <jar> [mainClass] args...

    七:logs

      转储容器日志

      用法: yarn logs -applicationId <application ID> [options]

    八:node

      打印node报表

      用法: yarn node [options]

    九:queue

      打印queue信息

      用法: yarn queue [options]

    十:version

      打印hadoop版本。

      用法: yarn version

    Administration Commands用

    一:介绍

      用于Hadoop集群用户命令。

    二:daemonlog

      获取/设置在后台Log的级别

      用法:yarn daemonlog -getlevel <host:httpport> <classname>
           yarn daemonlog -setlevel <host:httpport> <classname> <level>

    三:nodemanager

      启动NodeManager

      用法: yarn nodemanager

    四:proxyserver

      启动web proxy server

      用法: yarn proxyserver

    五:resourcemanager

      启动ResourceManager

      用法: yarn resourcemanager [-format-state-store]

    六:rmadmin

      运行ResourceManager管理客户端

    用法:yarn rmadmin [-refreshQueues]
                   [-refreshNodes]
                   [-refreshUserToGroupsMapping]
                   [-refreshSuperUserGroupsConfiguration]
                   [-refreshAdminAcls]
                   [-refreshServiceAcl]
                   [-getGroups [username]]
                   [-transitionToActive [--forceactive] [--forcemanual] <serviceId>]
                   [-transitionToStandby [--forcemanual] <serviceId>]
                   [-failover [--forcefence] [--forceactive] <serviceId1> <serviceId2>]
                   [-getServiceState <serviceId>]
                   [-checkHealth <serviceId>]
                   [-help [cmd]]

    七:scmadmin

      运行Shared Cache Manager管理客户端

      用法: yarn scmadmin [options]

    八:sharedcachemanager

      启动Shared Cache Manager

      用法: yarn sharedcachemanager

    九:timelineserver

      启动TimeLineServer

      用法: yarn timelineserver

    --------------------------------------------------------------------

      到此,本章节的内容讲述完毕。

    引用

    Apache  MapReduce Commands:http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html

    Apache YARN Commands:http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html

    Apache 1.04 中文:http://hadoop.apache.org/docs/r1.0.4/cn/commands_manual.html

    系列索引

      【源】从零自学Hadoop系列索引

     

    本文版权归mephisto和博客园共有,欢迎转载,但须保留此段声明,并给出原文链接,谢谢合作。

    文章是哥(mephisto)写的,SourceLink

  • 相关阅读:
    pytorch lstm crf 代码理解
    python sys.argv是什么?
    如何用简单易懂的例子解释条件随机场(CRF)模型?它和HMM有什么区别?
    jieba分词工具的使用方法
    手把手教你用Python实现自动特征工程
    命名实体识别视频51cto
    命名实体识别入门教程(必看)
    零基础入门--中文命名实体识别(BiLSTM+CRF模型,含代码)
    自然语言处理深度学习篇-BiLSTM文本挖掘实践 命名实体识别
    导航栏颜色
  • 原文地址:https://www.cnblogs.com/mephisto/p/4870207.html
Copyright © 2011-2022 走看看