zoukankan      html  css  js  c++  java
  • zookeeper删除kafka元数据

    问题:卸载kafka前未删除kafka topic,重新安装kafka后,生成跟之前topic名字相同的topic时报错,显示topic已存在

    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
    WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
    Error while executing topic command : Topic "hive-test-table" already exists.
    [2017-01-16 10:48:30,766] ERROR kafka.common.TopicExistsException: Topic "hive-test-table" already exists.
    	at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:261)
    	at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:245)
    	at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:107)
    	at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
    	at kafka.admin.TopicCommand.main(TopicCommand.scala)
     (kafka.admin.TopicCommand$)
    
    

    原因:kafka topic的元数据存储在zookeeper里,卸载kafka前未删除topic,之前topic的元数据仍存在zk里,需要手动删除zk里的元数据

    解决:

    zookeeper shell命令

    [zk: d65:2181/kafka_hive(CONNECTED) 7] ll
    ZooKeeper -server host:port cmd args
    	connect host:port
    	get path [watch]
    	ls path [watch]
    	set path data [version]
    	rmr path
    	delquota [-n|-b] path
    	quit 
    	printwatches on|off
    	create [-s] [-e] path data acl
    	stat path [watch]
    	close 
    	ls2 path [watch]
    	history 
    	listquota path
    	setAcl path acl
    	getAcl path
    	sync path
    	redo cmdno
    	addauth scheme auth
    	delete path [version]
    	setquota -n|-b val path
    

    进入zookeeper目录,删除kafka topic 元数据

    [root@d100 bin]# sh zkCli.sh -server  d65:2181/kafka_hive
    
    [zk: d65:2181/kafka_hive(CONNECTED) 8] ls /
    [consumers, config, controller, isr_change_notification, brokers, admin, controller_epoch]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 9] ls /brokers
    [seqid, topics, ids]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 10] ls /brokers/topics
    [略...]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 11] rmr /brokers/topics/hive-test-table
    
    [zk: d65:2181/kafka_hive(CONNECTED) 13] ls /brokers/topics
    [略...]
    

    重新创建topic成功

    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
    Created topic "hive-test-table".
    
    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --desc  --topic hive-test-table
    Topic:hive-test-table	PartitionCount:5	ReplicationFactor:2	Configs:
    	Topic: hive-test-table	Partition: 0	Leader: 1285	Replicas: 1285,1281	Isr: 1285,1281
    	Topic: hive-test-table	Partition: 1	Leader: 1281	Replicas: 1281,1282	Isr: 1281,1282
    	Topic: hive-test-table	Partition: 2	Leader: 1282	Replicas: 1282,1283	Isr: 1282,1283
    	Topic: hive-test-table	Partition: 3	Leader: 1283	Replicas: 1283,1284	Isr: 1283,1284
    	Topic: hive-test-table	Partition: 4	Leader: 1284	Replicas: 1284,1285	Isr: 1284,1285
    
  • 相关阅读:
    Docker 简单部署 ElasticSearch
    Anaconda更新失败简单解决[CondaHTTPError: HTTP 000 CONNECTION FAILED for url]
    TermKit的新一代Mac终端,在Ubuntu 11.04 轻松安装TermKit
    Linux下如何测试网速
    centos7安装yum
    Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
    yum 安装指定版本Docker
    如何在 Apache Hive 中解析 Json 数组
    unzip解压失败( cannot find zipfile directory)
    Linux 定时任务crontab_014
  • 原文地址:https://www.cnblogs.com/xiaodf/p/6289383.html
Copyright © 2011-2022 走看看