zoukankan      html  css  js  c++  java
  • Step by Step Recipe for Securing Kafka with Kerberos

    Short Description:

    Step by Step Recipe for Securing Kafka with Kerberos.

    Article

    I found it is a little tricky to get started with a Kerberos enabled Kafka cluster. I created this step by step recipe for securing Kafka with Kerberos, sending and receiving data on console. This is tested on HDP2.5.0 and Ambari 2.4.1.

    • Enabled Kerberos using the Ambari Kerberos setup wizard under Admin -- Kerberos menu.
    • On Ambari Kafka Config UI, change "listeners" property to "PLAINTEXTSASL://localhost:6667".
    • Restart Kafka as requested by Ambari.
    • Create a test topic in Kafka. Must use the kafka service user to do this.

      

    $ cd /usr/hdp/current/kafka-broker/bin
    $ sudo su kafka
    $ kinit -k -t /etc/security/keytabs/kafka.service.keytab kafka/ip-10-0-1-130.ap-northeast-1.compute.internal
     
    $ ./kafka-topics.sh --zookeeper ip-10-0-1-130.ap-northeast-1.compute.internal:2181 --create --topic foo --partitions 1 --replication-factor 1
     
    Created topic "bar".
    • Grant permission to user. This can be done using Kafka native ACL mechanism or Apache Ranger. In the example, we use Kafka ACL. User bob needs to be existing in KDC.
      # Grant user bob as producer on topic foo
      ./kafka-acls.sh --authorizer-properties zookeeper.connect=ip-10-0-1-130.ap-northeast-1.compute.internal:2181 
        --add --allow-principal User:bob 
        --producer --topic foo
       
      Adding ACLs for resource `Topic:foo`:
        User:bob has Allow permission for operations: Describe from hosts: *
        User:bob has Allow permission for operations: Write from hosts: *
       
      Adding ACLs for resource `Cluster:kafka-cluster`:
        User:bob has Allow permission for operations: Create from hosts: *
       
      Current ACLs for resource `Topic:foo`:
        User:bob has Allow permission for operations: Describe from hosts: *
        User:bob has Allow permission for operations: Write from hosts: *
       
      # Grant user bob as consumer
      ./kafka-acls.sh --authorizer-properties zookeeper.connect=ip-10-0-1-130.ap-northeast-1.compute.internal:2181 
        --add --allow-principal User:bob 
        --consumer --topic foo --group *

      #--group后等跟等号,如--group=*。按照上面写法,只是赋给名称为connect-distributed.sh的groupid。因为kafka/bin下面第一个脚本是connect-distributed.sh

      Adding ACLs for resource `Topic:foo`: User:bob has Allow permission for operations: Read from hosts: * User:bob has Allow permission for operations: Describe from hosts: * Adding ACLs for resource `Group:connect-distributed.sh`: User:bob has Allow permission for operations: Read from hosts: * Current ACLs for resource `Topic:foo`: User:bob has Allow permission for operations: Read from hosts: * User:bob has Allow permission for operations: Describe from hosts: * User:bob has Allow permission for operations: Write from hosts: * Current ACLs for resource `Group:connect-distributed.sh`: User:bob has Allow permission for operations: Read from hosts: *
    • Confirm the above works using the kafka console producer and consumer scripts.
      # Switch to bob user and log in to KDC.
      $ kinit bob
       
      # Start console producer 
      $ ./kafka-console-producer.sh --broker-list ip-10-0-1-130.ap-northeast-1.compute.internal:6667 --topic foo --security-protocol PLAINTEXTSASL
       
      # On another terminal, start console consumer
      ./kafka-console-consumer.sh --zookeeper ip-10-0-1-130.ap-northeast-1.compute.internal:2181 --topic foo --security-protocol PLAINTEXTSASL 
       
      {metadata.broker.list=ip-10-0-1-130.ap-northeast-1.compute.internal:6667, request.timeout.ms=30000, client.id=console-consumer-57797, security.protocol=PLAINTEXTSASL}
       
      # Type something on the producer terminal, it should appears on the console terminal immediately.
  • 相关阅读:
    VC++ 利用PDB和dump文件定位问题并进行调试
    MFC限制edit控件的字符输入长度
    VC++ 使用CreateProcess创建新进程
    正则表达式验证HTTP地址是否合法
    C++ _access和_waccess的使用方法
    最后一次谈 VirtualBox的安装方法
    解决/var/log下没有messages文件的问题?
    待续未完- 自己写后台内容管理程序的辅助内容
    php中的正则函数:正则匹配,正则替换,正则分割 所有的操作都不会影响原来的字符串.
    未完待续
  • 原文地址:https://www.cnblogs.com/felixzh/p/10489303.html
Copyright © 2011-2022 走看看