Kafka operators with Kerberos authentication

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Kafka operators with Kerberos authentication

Thomas Weise-2
Hi,

Has anyone run the Apex Kafka consumer or producer with security enabled?

I got authentication working in embedded mode and looking to deploy to the cluster. It will require 

* keytab
* JAAS config with the KafkaClient settings.
* JVM option  -Djava.security.auth.login.config=./kafka_client_jaas.conf
* config properties:

  <property>
    <name>apex.operator.kafkaOutput.prop.properties(security.protocol)</name>
    <value>SASL_SSL</value>
  </property>

  <property>
    <name>apex.operator.kafkaOutput.prop.properties(sasl.kerberos.service.name)</name>
    <value>kafka</value>
  </property>

I guess the JAAS conf and keytab can be pushed with the FILES argument. Any other ideas how to set this up?

Thanks,
Thomas

Reply | Threaded
Open this post in threaded view
|

Re: Kafka operators with Kerberos authentication

Pramod Immaneni
Wouldn't the kafka jaas.conf and keytab be already present on the nodes if managing the kafka deployment through the distro?

Thanks

On Thu, Jul 6, 2017 at 6:05 PM, Thomas Weise <[hidden email]> wrote:
Hi,

Has anyone run the Apex Kafka consumer or producer with security enabled?

I got authentication working in embedded mode and looking to deploy to the cluster. It will require 

* keytab
* JAAS config with the KafkaClient settings.
* JVM option  -Djava.security.auth.login.config=./kafka_client_jaas.conf
* config properties:

  <property>
    <name>apex.operator.kafkaOutput.prop.properties(security.protocol)</name>
    <value>SASL_SSL</value>
  </property>

  <property>
    <name>apex.operator.kafkaOutput.prop.properties(sasl.kerberos.service.name)</name>
    <value>kafka</value>
  </property>

I guess the JAAS conf and keytab can be pushed with the FILES argument. Any other ideas how to set this up?

Thanks,
Thomas


Reply | Threaded
Open this post in threaded view
|

Re: Kafka operators with Kerberos authentication

Thomas Weise-2
The user's keytab needs to be deployed by the application or another user owned process, just like what would need to occur for YARN.

Thomas

On Fri, Jul 7, 2017 at 3:31 PM, Pramod Immaneni <[hidden email]> wrote:
Wouldn't the kafka jaas.conf and keytab be already present on the nodes if managing the kafka deployment through the distro?

Thanks

On Thu, Jul 6, 2017 at 6:05 PM, Thomas Weise <[hidden email]> wrote:
Hi,

Has anyone run the Apex Kafka consumer or producer with security enabled?

I got authentication working in embedded mode and looking to deploy to the cluster. It will require 

* keytab
* JAAS config with the KafkaClient settings.
* JVM option  -Djava.security.auth.login.config=./kafka_client_jaas.conf
* config properties:

  <property>
    <name>apex.operator.kafkaOutput.prop.properties(security.protocol)</name>
    <value>SASL_SSL</value>
  </property>

  <property>
    <name>apex.operator.kafkaOutput.prop.properties(sasl.kerberos.service.name)</name>
    <value>kafka</value>
  </property>

I guess the JAAS conf and keytab can be pushed with the FILES argument. Any other ideas how to set this up?

Thanks,
Thomas