kafka -version 3.0.0

当把kafka部署在云服务器或者虚拟机上,使用本地主机的java客户端连接kafka进行消息的收发操作,可能在程序一开始就会产生java.net.UnknownHostException错误,这是由于没有正确配置kafkaadvertised.listeners地址产生的

这是我遇到的报错信息 lavm-rl6vynrf1w: nodename nor servname provided, or not known

java.net.UnknownHostException: lavm-rl6vynrf1w: nodename nor servname provided, or not known
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1343)
	at java.net.InetAddress.getAllByName0(InetAddress.java:1295)
	at java.net.InetAddress.getAllByName(InetAddress.java:1205)
	at java.net.InetAddress.getAllByName(InetAddress.java:1127)
	at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
	at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:109)
	at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:508)
	at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:465)
	at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:170)
	at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:975)
	at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
	at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:354)
	at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:327)
	at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:256)
	at java.lang.Thread.run(Thread.java:750)
17:17:51.661 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:17:51.712 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:17:51.764 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:17:51.815 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:17:51.866 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:17:51.917 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available
17:17:51.968 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up sending metadata request since no node is available

kafka/config/server.properties的配置文件中有一个advertised.listeners的配置(在配置文件中用esc + / + advertised.listeners 查找) 以下为默认值

# Hostname and port the broker will advertise to producers and consumers. If not set,
# it uses the value for "listeners" if configured.  Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
#advertised.listeners=PLAINTEXT://your.host.name:9092

这个参数的作用用于指定 Kafka Broker 向客户端(如生产者和消费者)公开的地址。它直接影响到客户端如何连接到 Kafka Broker。也就是说当客户端连接kafka进行操作,kafka会将这个地址发送给客户端,用于后续通信

那么如果客户端和kafka在同一主机上,客户端直接使用主机名也可以进行通信

如果在虚拟机或者云服务器,要么客户端在DNS中注册主机名和ip要么修改这个配置信息,暴露ip和端口

然后重启kafka

$KAFKA_HOME/bin/kafka-server-stop.sh
$KAFKA_HOME/bin/kafka-server-start.sh -daemon $KAFKA_HOME/config/server.properties

补充

客户端连接到 Broker时,会先连接到 bootstrap.servers 中指定的地址来获取元数据信息。

通过这个可以访问到kafka集群(如果存在),而集群中的其他成员,通过这个广告地址,使得通信得以建立

这一配置确保了客户端连接时的地址实际数据传输的地址可以不同,适用于跨网络或需要特定访问控制的环境

应用场景

  1. 可以将 Kafka 广告的地址设置为内网 IP,以限制外网访问
  2. Kafka 部署在内网中,而客户端在外网访问 Kafka,advertised.listeners 设置成一个外网ip
  3. Kafka 在 Docker 中运行,通过容器的 IP 连接无法访问 Kafka 实例,但可以通过宿主机的 IP 连接

Logo

技术共进,成长同行——讯飞AI开发者社区

更多推荐