Kafka connector security configuration

You can configure security for reactive messaging channels at both the channel level and the connector level. Connector-wide properties, like bootstrap.servers apply globally, whereas channel-specific properties, such as topic or group.id, customize the individual channel behavior.

Open Liberty enhances secure communication by supporting various protocols, each designed to safeguard data exchanges.

Secure Sockets Layer (SSL): For encrypted data transmission.
Simple Authentication and Security Layer (SASL)/PLAIN: For authenticated and encrypted communication.
Mutual TLS (mTLS): For mutually authenticated and encrypted communication.

These protocols help maintain the interactions with services and data to be confidential and secure. To secure communication with Kafka brokers, set the necessary security properties in the microprofile-config.properties file to support various authentication protocols such as SSL/TLS, SASL/PLAIN or mTLS.

Secure Sockets Layer (SSL)

The following example demonstrates how to configure a Kafka client for secure SSL communication with Kafka brokers in the microprofile-config.properties file. The following configuration enables SSL-based authentication so that the client can securely verify the identity of the Kafka server it connects to.

mp.messaging.connector.liberty-kafka.bootstrap.servers=SSL\://kafka-server\:34691
mp.messaging.connector.liberty-kafka.security.protocol=SSL
mp.messaging.connector.liberty-kafka.ssl.truststore.password=kafka-teststore
mp.messaging.connector.liberty-kafka.ssl.truststore.location=kafka-truststore.jks

In the example, specifying the truststore location and password is crucial for securing communications with Kafka brokers. The client can authenticate the server identity through trusted certificates, hence establishing a secure SSL/TLS connection. The truststore location (ssl.truststore.location) identifies the file that contains these certificates. Meanwhile, the truststore password (ssl.truststore.password) protects this file, safeguarding the integrity and confidentiality of data transmitted between the client and server. This streamlined setup is essential for preventing unauthorized access and facilitating encrypted, authenticated communication in line with best security practices.s

Simple Authentication and Security Layer (SASL)/PLAIN

The following example demonstrates the setup of the SASL/PLAIN authentication with either the Open Liberty Kafka Login Module or the Kafka Plain Login Module. This configuration enables encrypted communication and authentication with Kafka brokers. It uses properties set in the microprofile-config.properties file to support different authentication protocols, including password encryption with Open Liberty securityUtility encode. Applications can maintain the confidentiality and integrity of messages, ensuring secure data flow across distributed systems.

Authenticating with Open Liberty’s Kafka Login Module that can use passwords encoded by Open Liberty securityUtility encode on a per channel basis.

mp.messaging.incoming.aes-test-in.connector=liberty-kafka
mp.messaging.incoming.aes-test-in.bootstrap.servers=SASL_SSL\://kafka-boostrap-server\:39643
mp.messaging.incoming.aes-test-in.security.protocol=SASL_SSL
mp.messaging.incoming.aes-test-in.sasl.mechanism=PLAIN
mp.messaging.incoming.aes-test-in.ssl.truststore.password=kafka-teststore
mp.messaging.incoming.aes-test-in.sasl.jaas.config=com.ibm.ws.kafka.security.LibertyLoginModule required username\="test" password\="{aes}<encoded password>";
mp.messaging.incoming.aes-test-in.ssl.truststore.location=kafka-truststore.jks
mp.messaging.incoming.aes-test-in.group.id=group-id-1
mp.messaging.incoming.aes-test-in.auto.offset.reset=earliest

mp.messaging.outgoing.aes-test-out.connector=liberty-kafka
mp.messaging.outgoing.aes-test-out.bootstrap.servers=SASL_SSL\://kafka-boostrap-server\:39643
mp.messaging.outgoing.aes-test-out.security.protocol=SASL_SSL
mp.messaging.outgoing.aes-test-out.sasl.mechanism=PLAIN
mp.messaging.outgoing.aes-test-out.sasl.jaas.config=com.ibm.ws.kafka.security.LibertyLoginModule required username\="test" password\="{aes}<encoded password>";
mp.messaging.outgoing.aes-test-out.ssl.truststore.location=kafka-truststore.jks
mp.messaging.outgoing.aes-test-out.ssl.truststore.password=kafka-teststore

Authenticating with Kafka’s Plain Login Module.

mp.messaging.connector.liberty-kafka.security.protocol=SASL_SSL
mp.messaging.connector.liberty-kafka.bootstrap.servers=SASL_SSL\://kafka-boostrap-server\:34696
mp.messaging.connector.liberty-kafka.ssl.truststore.location=kafka-truststore.jks
mp.messaging.connector.liberty-kafka.sasl.mechanism=PLAIN
mp.messaging.connector.liberty-kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username\="test" password\="test-QmCFfb";
mp.messaging.connector.liberty-kafka.ssl.truststore.password=kafka-teststore

Mutual TLS (mTLS)

Mutual TLS is an enhanced security protocol that requires both the client and server to authenticate each other, providing a two-way SSL authentication. Each channel uses a separate keystore to authenticate itself with the Kafka Bootstrap server.

The following example configures each channel with its own keystore to authenticate itself with the Kafka bootstrap server, as detailed in the configuration settings. With the mp.messaging.connector.liberty-kafka and specific channel configurations, the example demonstrates how to establish a secure, encrypted channel by using SSL. Mutual TLS not only secures the data in transit but also makes sure that each communication partner is authenticated, thus adding another layer of security.

mp.messaging.connector.liberty-kafka.bootstrap.servers=SSL\://kafka-boostrap-server\:39647
mp.messaging.connector.liberty-kafka.security.protocol=SSL
mp.messaging.connector.liberty-kafka.ssl.truststore.location=kafka-truststore.jks
mp.messaging.connector.liberty-kafka.ssl.truststore.password=kafka-teststore

mp.messaging.incoming.test-in.connector=liberty-kafka
mp.messaging.incoming.test-in.ssl.keystore.location=kafka-keystore.jks
mp.messaging.incoming.test-in.ssl.keystore.password=kafka-teststore
mp.messaging.incoming.test-in.group.id=group-id-1
mp.messaging.incoming.test-in.topic=incoming-topic
mp.messaging.incoming.test-in.auto.offset.reset=earliest

mp.messaging.outgoing.test-out.connector=liberty-kafka
mp.messaging.outgoing.test-out.topic=outgoing-topic
mp.messaging.outgoing.test-out.ssl.keystore.location=kafka-keystore2.jks
mp.messaging.outgoing.test-out.ssl.keystore.password=kafka-teststore