zoukankan      html  css  js  c++  java
  • 一脸懵逼学习Storm的搭建--(一个开源的分布式实时计算系统)

    Storm的官方网址:http://storm.apache.org/index.html

    1:集群部署的基本流程(基本套路):
        集群部署的流程:下载安装包、解压安装包、修改配置文件、分发安装包、启动集群;

     1:安装一个zookeeper集群,之前已经部署过,这里省略,贴一下步骤;

    安装配置zooekeeper集群:
            1.1:解压
                tar -zxvf zookeeper-3.4.5.tar.gz
            1.2:修改配置
                cd /home/hadoop/zookeeper-3.4.5/conf/
                cp zoo_sample.cfg zoo.cfg
                vim zoo.cfg
                修改:dataDir=/home/hadoop/zookeeper-3.4.5/tmp
                在最后添加:
                server.1=master:2888:3888
                server.2=slaver1:2888:3888
                server.3=slaver2:2888:3888
                保存退出
                然后创建一个tmp文件夹
                mkdir /home/hadoop/zookeeper-3.4.5/tmp
                再创建一个空文件
                touch /home/hadoop/zookeeper-3.4.5/tmp/myid
                最后向该文件写入ID
                echo 1 > /home/hadoop/zookeeper-3.4.5/tmp/myid
            1.3将配置好的zookeeper拷贝到其他节点:
                scp -r /home/hadoop/zookeeper-3.4.5/ slaver1:/home/hadoop/
                scp -r /home/hadoop/zookeeper-3.4.5/ slaver2:/home/hadoop/
                
                注意:修改slaver1slaver2对应/home/hadoop/zookeeper-3.4.5/tmp/myid内容
                slaver1
                    echo 2 > /home/hadoop/zookeeper-3.4.5/tmp/myid
                slaver2
                    echo 3 > /home/hadoop/zookeeper-3.4.5/tmp/myid

    2、上传storm的安装包,解压缩:

    [root@master hadoop]# tar -zxvf apache-storm-0.9.2-incubating.tar.gz

    3、修改配置文件storm.yaml:

    可以创建一个软连接,方便操作storm:[root@master soft]# ln -s apache-storm-0.9.2-incubating storm

    修改内容如下所示,下面两个配置均可:

    #指定storm使用的zk集群
    storm.zookeeper.servers:
    - "zk01"
    - "zk02"
    - "zk03"
    #指定storm集群中的nimbus节点所在的服务器
    nimbus.host: "storm01"
    #指定nimbus启动JVM最大可用内存大小
    nimbus.childopts: "-Xmx1024m"
    #指定supervisor启动JVM最大可用内存大小
    supervisor.childopts: "-Xmx1024m"
    #指定supervisor节点上,每个worker启动JVM最大可用内存大小
    worker.childopts: "-Xmx768m"
    #指定ui启动JVM最大可用内存大小,ui服务一般与nimbus同在一个节点上。
    ui.childopts: "-Xmx768m"
    #指定supervisor节点上,启动worker时对应的端口号,每个端口对应槽,每个槽位对应一个worker
    supervisor.slots.ports:
    - 6700
    - 6701
    - 6702
    - 6703

     此次配置使用下面的,进行集群安装:

    #所使用的zookeeper集群主机
    storm.zookeeper.servers:
         - "master"
         - "slaver1"
         - "slaver2"
    
    #nimbus所在的主机名
    nimbus.host: "master"

    supervisor.slots.ports
    -6701
    -6702
    -6703
    -6704
    -6705

    然后将master修改好的storm发送到slaver1,slaver2:

    [root@master hadoop]# scp -r apache-storm-0.9.2-incubating/ slaver1:/home/hadoop/

    [root@master hadoop]# scp -r apache-storm-0.9.2-incubating/ slaver2:/home/hadoop/

     

    4:启动storm集群,首先启动你的Zookeeper集群,然后再启动你的storm集群哈。

     启动Zookeeper集群:

    然后启动Storm集群:

     

    启动storm
    在nimbus主机上,在nimbus.host所属的机器上启动 nimbus服务:
      nohup ./storm nimbus 1>/dev/null 2>&1 &    或者使用命令:nohup ./storm nimbus &   
      或者使用命令:./storm nimbus > ../logs/info 2>&1 &  (注释:将启动信息,错误信息,标准信息都打印到如此目录里面)
    在nimbus.host所属的机器上启动ui服务:   nohup ./storm ui 1>/dev/null 2>&1 & 或者使用命令:nohup ./storm ui & 在supervisor主机上,在其它个点击上启动supervisor服务:   nohup ./storm supervisor 1>/dev/null 2>&1 & 或者使用命令:nohup ./storm supervisor &
      或者使用命令:./storm supervisor > ../logs/info 2>&1 (注释:将启动信息,错误信息,标准信息都打印到如此目录里面)

     注意,解释:

      1>/dev/null:代表标准输入到这个目录;

      2>&1:代表标准输出也到这个目录下面;

      &:代表这个是后台运行;

    如下启动storm方便观察,最后一行是卡住不动的哦:

    查看进程如下所示:

    可以启动一下storm的ui查看:

     

    查看一下进程如:

    启动ui以后可以在浏览器访问,如:

    启动supervisor

    [root@slaver1 bin]# ./storm supervisor

     然后可以启动剩下的storm:

    [root@slaver2 bin]# ./storm supervisor

    启动以后可以查看进程jps的启动情况,然后可以去浏览器查看自己http://192.168.3.129:8080/index.html的页面各个启动情况,如supervisor等等。

    5:Storm常用操作命令:

    1:有许多简单且有用的命令可以用来管理拓扑,它们可以提交、杀死、禁用、再平衡拓扑。
        提交任务命令格式:storm jar 【jar路径】 【拓扑包名.拓扑类名】 【拓扑名称】
    2:bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.6.jar storm.starter.WordCountTopology wordcount
        杀死任务命令格式:storm kill 【拓扑名称】 -w 10(执行kill命令时可以通过-w [等待秒数]指定拓扑停用以后的等待时间)
    3:storm kill topology-name -w 10
        停用任务命令格式:storm deactivte  【拓扑名称】
    4:storm deactivte topology-name
    5:我们能够挂起或停用运行中的拓扑。当停用拓扑时,所有已分发的元组都会得到处理,但是spouts的nextTuple方法不会被调用。销毁一个拓扑,可以使用kill命令。它会以一种安全的方式销毁一个拓扑,首先停用拓扑,在等待拓扑消息的时间段内允许拓扑完成当前的数据流。
        启用任务命令格式:storm activate【拓扑名称】
            storm activate topology-name
        重新部署任务命令格式:storm rebalance  【拓扑名称】
            storm rebalance topology-name
            再平衡使你重分配集群任务。这是个很强大的命令。比如,你向一个运行中的集群增加了节点。再平衡命令将会停用拓扑,然后在相应超时时间之后重分配工人,并重启拓扑。

     注意使用storm运行jar包的时候是没有输入输出路径的,区别于hadoop离线分析:

     1 [root@master storm]# bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar storm.starter.WordCountTopology wordcount
     2 Running: /home/hadoop/soft/jdk1.7.0_65/bin/java -client -Dstorm.options= -Dstorm.home=/home/hadoop/soft/apache-storm-0.9.2-incubating -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/hadoop/soft/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/jline-2.11.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar:/home/hadoop/soft/apache-storm-0.9.2-incubating/conf:/home/hadoop/soft/apache-storm-0.9.2-incubating/bin -Dstorm.jar=examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar storm.starter.WordCountTopology wordcount
     3 712  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
     4 772  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar to assigned location: storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar
     5 Start uploading file 'examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar' to 'storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar' (2927299 bytes)
     6 [==================================================] 2927299 / 2927299
     7 File 'examples/storm-starter/storm-starter-topologies-0.9.2-incubating.jar' uploaded to 'storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar' (2927299 bytes)
     8 941  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: storm-local/nimbus/inbox/stormjar-76fccf41-491e-4d61-8a98-4092c8630161.jar
     9 941  [main] INFO  backtype.storm.StormSubmitter - Submitting topology wordcount in distributed mode with conf {"topology.workers":3,"topology.debug":true}
    10 1431 [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: wordcount
    11 [root@master storm]#

    使用storm自带的统计的demo启动起来以后ui如下所示:

    然后点击wordCount以后如下所示,点击下面的按钮是相应的操作

    点击上面的按钮是相应的操作;

    出现如下错误,这里贴一下,出错原因是:/storm/conf/storm.yaml的配置nimbus.host: "master"前面多了一个空格,删除空格即可:

      1 [root@slaver1 bin]# ./storm supervisor 
      2 Exception in thread "main" java.lang.ExceptionInInitializerError
      3     at java.lang.Class.forName0(Native Method)
      4     at java.lang.Class.forName(Class.java:190)
      5     at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
      6     at backtype.storm.config__init.load(Unknown Source)
      7     at backtype.storm.config__init.<clinit>(Unknown Source)
      8     at java.lang.Class.forName0(Native Method)
      9     at java.lang.Class.forName(Class.java:270)
     10     at clojure.lang.RT.loadClassForName(RT.java:2098)
     11     at clojure.lang.RT.load(RT.java:430)
     12     at clojure.lang.RT.load(RT.java:411)
     13     at clojure.core$load$fn__5018.invoke(core.clj:5530)
     14     at clojure.core$load.doInvoke(core.clj:5529)
     15     at clojure.lang.RestFn.invoke(RestFn.java:408)
     16     at clojure.core$load_one.invoke(core.clj:5336)
     17     at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
     18     at clojure.core$load_lib.doInvoke(core.clj:5374)
     19     at clojure.lang.RestFn.applyTo(RestFn.java:142)
     20     at clojure.core$apply.invoke(core.clj:619)
     21     at clojure.core$load_libs.doInvoke(core.clj:5417)
     22     at clojure.lang.RestFn.applyTo(RestFn.java:137)
     23     at clojure.core$apply.invoke(core.clj:621)
     24     at clojure.core$use.doInvoke(core.clj:5507)
     25     at clojure.lang.RestFn.invoke(RestFn.java:408)
     26     at backtype.storm.command.config_value$loading__4910__auto__.invoke(config_value.clj:16)
     27     at backtype.storm.command.config_value__init.load(Unknown Source)
     28     at backtype.storm.command.config_value__init.<clinit>(Unknown Source)
     29     at java.lang.Class.forName0(Native Method)
     30     at java.lang.Class.forName(Class.java:270)
     31     at clojure.lang.RT.loadClassForName(RT.java:2098)
     32     at clojure.lang.RT.load(RT.java:430)
     33     at clojure.lang.RT.load(RT.java:411)
     34     at clojure.core$load$fn__5018.invoke(core.clj:5530)
     35     at clojure.core$load.doInvoke(core.clj:5529)
     36     at clojure.lang.RestFn.invoke(RestFn.java:408)
     37     at clojure.lang.Var.invoke(Var.java:415)
     38     at backtype.storm.command.config_value.<clinit>(Unknown Source)
     39 Caused by: while parsing a block mapping
     40  in 'reader', line 18, column 1:
     41     storm.zookeeper.servers:
     42     ^
     43 expected <block end>, but found BlockMappingStart
     44  in 'reader', line 23, column 2:
     45      nimbus.host: "master"
     46      ^
     47 
     48     at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:570)
     49     at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
     50     at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:143)
     51     at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:230)
     52     at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:159)
     53     at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:122)
     54     at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:105)
     55     at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:120)
     56     at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:481)
     57     at org.yaml.snakeyaml.Yaml.load(Yaml.java:424)
     58     at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:141)
     59     at backtype.storm.utils.Utils.readStormConfig(Utils.java:188)
     60     at backtype.storm.utils.Utils.<clinit>(Utils.java:71)
     61     ... 36 more
     62 Exception in thread "main" java.lang.ExceptionInInitializerError
     63     at java.lang.Class.forName0(Native Method)
     64     at java.lang.Class.forName(Class.java:190)
     65     at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
     66     at backtype.storm.config__init.load(Unknown Source)
     67     at backtype.storm.config__init.<clinit>(Unknown Source)
     68     at java.lang.Class.forName0(Native Method)
     69     at java.lang.Class.forName(Class.java:270)
     70     at clojure.lang.RT.loadClassForName(RT.java:2098)
     71     at clojure.lang.RT.load(RT.java:430)
     72     at clojure.lang.RT.load(RT.java:411)
     73     at clojure.core$load$fn__5018.invoke(core.clj:5530)
     74     at clojure.core$load.doInvoke(core.clj:5529)
     75     at clojure.lang.RestFn.invoke(RestFn.java:408)
     76     at clojure.core$load_one.invoke(core.clj:5336)
     77     at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
     78     at clojure.core$load_lib.doInvoke(core.clj:5374)
     79     at clojure.lang.RestFn.applyTo(RestFn.java:142)
     80     at clojure.core$apply.invoke(core.clj:619)
     81     at clojure.core$load_libs.doInvoke(core.clj:5417)
     82     at clojure.lang.RestFn.applyTo(RestFn.java:137)
     83     at clojure.core$apply.invoke(core.clj:621)
     84     at clojure.core$use.doInvoke(core.clj:5507)
     85     at clojure.lang.RestFn.invoke(RestFn.java:408)
     86     at backtype.storm.command.config_value$loading__4910__auto__.invoke(config_value.clj:16)
     87     at backtype.storm.command.config_value__init.load(Unknown Source)
     88     at backtype.storm.command.config_value__init.<clinit>(Unknown Source)
     89     at java.lang.Class.forName0(Native Method)
     90     at java.lang.Class.forName(Class.java:270)
     91     at clojure.lang.RT.loadClassForName(RT.java:2098)
     92     at clojure.lang.RT.load(RT.java:430)
     93     at clojure.lang.RT.load(RT.java:411)
     94     at clojure.core$load$fn__5018.invoke(core.clj:5530)
     95     at clojure.core$load.doInvoke(core.clj:5529)
     96     at clojure.lang.RestFn.invoke(RestFn.java:408)
     97     at clojure.lang.Var.invoke(Var.java:415)
     98     at backtype.storm.command.config_value.<clinit>(Unknown Source)
     99 Caused by: while parsing a block mapping
    100  in 'reader', line 18, column 1:
    101     storm.zookeeper.servers:
    102     ^
    103 expected <block end>, but found BlockMappingStart
    104  in 'reader', line 23, column 2:
    105      nimbus.host: "master"
    106      ^
    107 
    108     at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:570)
    109     at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
    110     at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:143)
    111     at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:230)
    112     at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:159)
    113     at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:122)
    114     at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:105)
    115     at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:120)
    116     at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:481)
    117     at org.yaml.snakeyaml.Yaml.load(Yaml.java:424)
    118     at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:141)
    119     at backtype.storm.utils.Utils.readStormConfig(Utils.java:188)
    120     at backtype.storm.utils.Utils.<clinit>(Utils.java:71)
    121     ... 36 more
    122 Exception in thread "main" java.lang.ExceptionInInitializerError
    123     at java.lang.Class.forName0(Native Method)
    124     at java.lang.Class.forName(Class.java:190)
    125     at backtype.storm.config$loading__4910__auto__.invoke(config.clj:17)
    126     at backtype.storm.config__init.load(Unknown Source)
    127     at backtype.storm.config__init.<clinit>(Unknown Source)
    128     at java.lang.Class.forName0(Native Method)
    129     at java.lang.Class.forName(Class.java:270)
    130     at clojure.lang.RT.loadClassForName(RT.java:2098)
    131     at clojure.lang.RT.load(RT.java:430)
    132     at clojure.lang.RT.load(RT.java:411)
    133     at clojure.core$load$fn__5018.invoke(core.clj:5530)
    134     at clojure.core$load.doInvoke(core.clj:5529)
    135     at clojure.lang.RestFn.invoke(RestFn.java:408)
    136     at clojure.core$load_one.invoke(core.clj:5336)
    137     at clojure.core$load_lib$fn__4967.invoke(core.clj:5375)
    138     at clojure.core$load_lib.doInvoke(core.clj:5374)
    139     at clojure.lang.RestFn.applyTo(RestFn.java:142)
    140     at clojure.core$apply.invoke(core.clj:619)
    141     at clojure.core$load_libs.doInvoke(core.clj:5417)
    142     at clojure.lang.RestFn.applyTo(RestFn.java:137)
    143     at clojure.core$apply.invoke(core.clj:621)
    144     at clojure.core$use.doInvoke(core.clj:5507)
    145     at clojure.lang.RestFn.invoke(RestFn.java:408)
    146     at backtype.storm.command.config_value$loading__4910__auto__.invoke(config_value.clj:16)
    147     at backtype.storm.command.config_value__init.load(Unknown Source)
    148     at backtype.storm.command.config_value__init.<clinit>(Unknown Source)
    149     at java.lang.Class.forName0(Native Method)
    150     at java.lang.Class.forName(Class.java:270)
    151     at clojure.lang.RT.loadClassForName(RT.java:2098)
    152     at clojure.lang.RT.load(RT.java:430)
    153     at clojure.lang.RT.load(RT.java:411)
    154     at clojure.core$load$fn__5018.invoke(core.clj:5530)
    155     at clojure.core$load.doInvoke(core.clj:5529)
    156     at clojure.lang.RestFn.invoke(RestFn.java:408)
    157     at clojure.lang.Var.invoke(Var.java:415)
    158     at backtype.storm.command.config_value.<clinit>(Unknown Source)
    159 Caused by: while parsing a block mapping
    160  in 'reader', line 18, column 1:
    161     storm.zookeeper.servers:
    162     ^
    163 expected <block end>, but found BlockMappingStart
    164  in 'reader', line 23, column 2:
    165      nimbus.host: "master"
    166      ^
    167 
    168     at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:570)
    169     at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
    170     at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:143)
    171     at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:230)
    172     at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:159)
    173     at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:122)
    174     at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:105)
    175     at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:120)
    176     at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:481)
    177     at org.yaml.snakeyaml.Yaml.load(Yaml.java:424)
    178     at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:141)
    179     at backtype.storm.utils.Utils.readStormConfig(Utils.java:188)
    180     at backtype.storm.utils.Utils.<clinit>(Utils.java:71)
    181     ... 36 more

    开始我用的apache-storm-0.9.2-incubating.tar.gz版本,使用如下官方demo,出现的问题是storm的ui的点击wordcount没有

    Spouts (All time),Bolts (All time),Topology Visualization,Topology Configuration等等这些内容

    解决方法,我是换的新版本解决问题:

    [root@slaver1 storm]# bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.6.jar storm.starter.WordCountTopology wordcount

    6:在Storm的目录的bin目录下面,输入./storm命令可以查看Storm有何命令可使用:

    #列出Topology计算拓补
    [root@slaver1 bin]# ./storm list
    #杀死Topology计算拓补
    [root@slaver1 bin]# ./storm kill Topology这个计算拓补的名称

     Storm的源码下载以及查看,点击Apache/storm即可进入Github进行源码下载:

    7:Idea创建的Maven项目打包,测试,如何操作呢???

    最后打成的jar包就是target里面的jar包:

    运行如下所示:

    1 [root@master bin]# storm jar /home/hadoop/data_hadoop/storm-1.0-SNAPSHOT.jar com.bie.WordCountTopologyMain

    效果如下所示:

    待续......

  • 相关阅读:
    下巴肉和脖子肉怎么减肥
    java中compareTo和compare方法之比较,集合中对象的比较
    easyui中combotree只能选子选项,父级不被选中
    java线程总结(2/5)
    流行的框架与新技术
    Spring官网改版后下载
    prepareStatement与Statement的区别
    jQuery li click失效问题
    Flask 启动报错 error: [Errno 10053]
    [linux]CentOS 7 下安装 RabbitMQ
  • 原文地址:https://www.cnblogs.com/biehongli/p/7762717.html
Copyright © 2011-2022 走看看