zoukankan      html  css  js  c++  java
  • zookeeper生成分布式自增ID

    1. 环境

    zookeeper: 3.6.0 windows
    springboot 2.2.6
    jdk 11

    2. 依赖引入

    <properties>
        <curator.version>4.2.0</curator.version>
    </properties>
    <!-- curator ZK 客户端 -->
    <dependency>
        <groupId>org.apache.curator</groupId>
        <artifactId>curator-framework</artifactId>
        <version>${curator.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.curator</groupId>
        <artifactId>curator-recipes</artifactId>
        <version>${curator.version}</version>
    </dependency>
    

    完整的pom.xml文件如下

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>2.2.6.RELEASE</version>
            <relativePath/> <!-- lookup parent from repository -->
        </parent>
        <groupId>com.example</groupId>
        <artifactId>zookeeper</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>zookeeper</name>
        <description>Demo project for Spring Boot</description>
    
        <properties>
            <java.version>1.8</java.version>
            <curator.version>4.2.0</curator.version>
        </properties>
    
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
                <exclusions>
                    <exclusion>
                        <groupId>org.junit.vintage</groupId>
                        <artifactId>junit-vintage-engine</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
            <!-- curator ZK 客户端 -->
            <dependency>
                <groupId>org.apache.curator</groupId>
                <artifactId>curator-framework</artifactId>
                <version>${curator.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.curator</groupId>
                <artifactId>curator-recipes</artifactId>
                <version>${curator.version}</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
            <dependency>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
                <version>1.18.10</version>
                <scope>provided</scope>
            </dependency>
    
        </dependencies>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    
    </project>
    
    

    2. 配置文件

    配置文件为srcmain esourceszookeeper.properties,存储内容如下:

    # zk host地址
    zk.host=127.0.0.1:2181
    # zk自增存储node
    zk.sequence-path=/news/sequence/
    

    3. 枚举封装

    创建com.example.zookeeper.sequence.ZkSequenceEnum文件,用于定义通过Zk生成自增ID的枚举,在项目中规范要求与物理表名项目,使用与当前项目阶段的枚举如下:

    public enum ZkSequenceEnum {
        AP_LIKES, AP_READ_BEHAVIOR, AP_COLLECTION, AP_USER_FOLLOW, AP_USER_FAN
    }
    
    

    4. 序列封装

    创建com.example.zookeeper.sequence.ZkSequence文件,用于封装程序在运行时每个表对应的自增器,这里主要通过分布式原子自增类(DistributedAtomicLong)实现,注意每500毫秒重试3次后仍然生成失败则返回null,由上层处理,相关实现代码如下:

    public class ZkSequence {
    
        RetryPolicy retryPolicy = new ExponentialBackoffRetry(500, 3);
    
        DistributedAtomicLong distAtomicLong;
    
        public ZkSequence(String sequenceName, CuratorFramework client) {
            distAtomicLong = new DistributedAtomicLong(client, sequenceName, retryPolicy);
        }
    
        /**
         * 生成序列
         *
         * @return
         * @throws Exception
         */
        public Long sequence() throws Exception {
            AtomicValue<Long> sequence = this.distAtomicLong.increment();
            if (sequence.succeeded()) {
                return sequence.postValue();
            } else {
                return null;
            }
        }
    
    }
    

    5. Client封装

    创建com.example.zookeeper.client.ZookeeperClient类,通过PostConstruct注解在内构器之后调用init方法初始化客户端连接,并调用initZkSequence方法初始项目所定义的ZkSequence,并存储在zkSequenceMap集合中,最终提供sequence方法来查询对应zkSequence获取自增ID,相关实现代码如下:

    @Data
    public class ZookeeperClient {
        private static Logger logger = LoggerFactory.getLogger(ZookeeperClient.class);
        private String host;
        private String sequencePath;
    
        // 重试休眠时间
        private final int SLEEP_TIME_MS = 1000;
        // 最大重试1000次
        private final int MAX_RETRIES = 1000;
        //会话超时时间
        private final int SESSION_TIMEOUT = 30 * 1000;
        //连接超时时间
        private final int CONNECTION_TIMEOUT = 3 * 1000;
    
        //创建连接实例
        private CuratorFramework client = null;
        // 序列化集合
        private Map<String, ZkSequence> zkSequence = Maps.newConcurrentMap();
    
        public ZookeeperClient(String host, String sequencePath) {
            this.host = host;
            this.sequencePath = sequencePath;
        }
    
        @PostConstruct
        public void init() throws Exception {
            this.client = CuratorFrameworkFactory.builder()
                    .connectString(this.getHost())
                    .connectionTimeoutMs(CONNECTION_TIMEOUT)
                    .sessionTimeoutMs(SESSION_TIMEOUT)
                    .retryPolicy(new ExponentialBackoffRetry(SLEEP_TIME_MS, MAX_RETRIES)).build();
            this.client.start();
            this.initZkSequence();
        }
    
        public void initZkSequence() {
            ZkSequenceEnum[] list = ZkSequenceEnum.values();
            for (int i = 0; i < list.length; i++) {
                String name = list[i].name();
                String path = this.sequencePath + name;
                ZkSequence seq = new ZkSequence(path, this.client);
                zkSequence.put(name, seq);
            }
        }
    
        /**
         * 生成SEQ
         *
         * @param name
         * @return
         * @throws Exception
         */
        public Long sequence(ZkSequenceEnum name) {
            try {
                ZkSequence seq = zkSequence.get(name.name());
                if (seq != null) {
                    return seq.sequence();
                }
            } catch (Exception e) {
                logger.error("获取[{}]Sequence错误:{}", name, e);
            }
            return null;
        }
    }
    

    注:在这里ZookeeperClient是一个BeanFactoryZkSequence是一个FactoryBean

    6. Config封装

    创建com.example.zookeeper.config.ZkConfig类,用于自动化配置环境文件的导入,和zkClient定义Bean定义,其相关的实现代码如下:

    /**
     * 自动化配置核心数据库的连接配置
     */
    @Data
    @Configuration
    @ConfigurationProperties(prefix="zk")
    @PropertySource("classpath:zookeeper.properties")
    public class ZkConfig {
    
        String host;
        String sequencePath;
    
        /**
         * 这是最快的数据库连接池
         * @return
         */
        @Bean
        public ZookeeperClient zookeeperClient(){
            return new ZookeeperClient(this.host,this.sequencePath);
        }
    
    }
    

    7. Sequences封装

    为便于程序中调用,以及对自增生成失败的统一处理,项目中规范通过com.example.zookeeper.sequence.Sequences类统一暴露生成自增主键的功能,相关代码如下:

    @Component
    public class Sequences {
    
        @Autowired
        private ZookeeperClient client;
    
        public Long sequenceApLikes() {
            return this.client.sequence(ZkSequenceEnum.AP_LIKES);
        }
    
        public Long sequenceApReadBehavior() {
            return this.client.sequence(ZkSequenceEnum.AP_READ_BEHAVIOR);
        }
    
        public Long sequenceApCollection() {
            return this.client.sequence(ZkSequenceEnum.AP_COLLECTION);
        }
    
        public Long sequenceApUserFollow() {
            return this.client.sequence(ZkSequenceEnum.AP_USER_FOLLOW);
        }
    
        public Long sequenceApUserFan() {
            return this.client.sequence(ZkSequenceEnum.AP_USER_FAN);
        }
    
    }
    

    8. 测试

    @SpringBootTest
    class ZookeeperApplicationTests {
    
        // 第一步,注入Sequences
        @Autowired
        private Sequences sequences;
    
        @Test
        void contextLoads() {
            for (int i = 0; i < 10; i++) {
                System.out.println("sequenceApCollection生成的自增id为:" + sequences.sequenceApCollection());
            }
        }
    
    }
    

    9. 扩展

    如后期需要新增ZkSequence自增表,可参考以下操作步骤,快速实现:

    • 在`ZkSequenceEnum中定义对应的枚举项,规范要求枚举项与物理表名一致且大写
    • Sequences中定义对应的调用方法,规范要求方法由sequence前缀+驼峰表名组成

    10. 代码

    微云下载

  • 相关阅读:
    用 Python 带你看各国 GDP 变迁
    Fluent Interface(流式接口)
    probing privatePath如何作用于ASP.NET MVC View
    Word插入htm文件导致文本域动态增加的一个问题
    Visual Studio 2013附加进程调试IE加载的ActiveX Control无效解决方法
    Ubuntu下Chrome运行Silverlight程序
    Windows Phone Bing lock screen doesn't change解决方法
    SPClaimsUtility.AuthenticateFormsUser的证书验证问题
    Web Service Client使用Microsoft WSE 2.0
    Visual Studio 2013安装Update 3启动crash的解决方法
  • 原文地址:https://www.cnblogs.com/ifme/p/12785497.html
Copyright © 2011-2022 走看看