zoukankan      html  css  js  c++  java
  • Storm-166:Nimbus HA solution based on Zookeeper

      Nimbus HA feature is quite important for our application running on the storm cluster. So, we've been working on the problem for some time and now a solution seems not that perfect but be enough to apply has comed out.

      Numbus的高性能特征对于运行在Storm上的应用来说是相当的重要。所以,我们已经研究此问题一段时间了,现在我们已经提出了一种解决方案,虽然它可能不太完美。


      1.Nimbus Servers now can register themselves in Zookeeper. They perform a leader election using "InterProcessMutex" interact with Zookeeper to ensure that there is only one nimbus responsible for launching and monitoring topologies.

      1、Nimbus服务器现在已经可以自注册到Zookeeper上。他们利用InterProcessMutex选举算法与Zookeeper通信,以确保只有一个nimbus负责运行和监控拓扑。


      2.Every Nimbus Server is running a timer to compare and find if there are topology codes which are not exists on it's local disk. They would download lcoal missing topology codes from the Nimbus leader through the thrift RPC just like Supervisors do.With this feature, any numbers of Nimbus Server can be launched through out the cluster.

      2、每一个nimbus服务器运行一个计时器以便比较和查找是否有拓扑代码运行在他们的本地磁盘。他们可以从nimbus主节点通过thrift RPC下载本地丢失的拓扑代码,就像supervisor一样。有了这个特性,任何数量的nimbus服务器都可以通过集群启动。


      3.StormSubmitter,Supervisor,Non-leader Nimbus and Storm UI now are able to find and connect to the Nimbus leader via Zookeeper.A Nimbus leadership table is also added to Storm-UI on the main page to show every Nimbus's leader-election state and it's host in addition.

      3、StormSubmitter,Supervisor,Non-leader Nimbus and Storm UI现在可以通过Zookeeper找到并且和nimbus主节点通信。nimbus节点表也被添加到Storm-UI的主页上,以便显示每个nimbus节点的选举状态以及它们的主机地址。


    PS: Some implementation of the Nimbus-Election part has taken @Frostman's solution for reference(link: nathanmarz/storm#422).

    附注:nimbus选举机制的实施部分请参看Frostman的解决方案。



  • 相关阅读:
    [其他]JAVA与C#的Socket通信
    Ext 向Ext.form.ComboBox()中添加列表的分类
    Extjs tree 过滤查询功能
    Extjs TreePanel API详解
    JVM虚拟机21: 1.8中废弃永久代(PermGen)迎来元空间(Metaspace)
    JVM虚拟机20:内存区域详解(Eden Space、Survivor Space、Old Gen、Code Cache和Perm Gen)
    Java虚拟机19:再谈四种引用状态
    Java虚拟机18:Java对象大小、对象内存布局及锁状态变化
    Java虚拟机17:互斥同步、锁优化及synchronized和volatile
    Java虚拟机16:Java内存模型
  • 原文地址:https://www.cnblogs.com/xymqx/p/4403037.html
Copyright © 2011-2022 走看看