KAFKA_CFG_ADVERTISED_LISTENERS: BROKER://kafka-broker-0.kafka-headless.observe.svc.cluster.local:9092,EXTERNAL://20.46.101.13:31052 openjdk version "17.0.9" 2023-10-17 OpenJDK Runtime Environment Temurin-17.0.9+9 (build 17.0.9+9) OpenJDK 64-Bit Server VM Temurin-17.0.9+9 (build 17.0.9+9, mixed mode, sharing) Picked up JAVA_TOOL_OPTIONS: -Xms7000m -Xmx7000m [2024-10-28 03:44:38,322] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) [2024-10-28 03:44:38,608] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util) [2024-10-28 03:44:38,759] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler) [2024-10-28 03:44:38,763] INFO [BrokerServer id=1000] Transition from SHUTDOWN to STARTING (kafka.server.BrokerServer) [2024-10-28 03:44:38,764] INFO [SharedServer id=1000] Starting SharedServer (kafka.server.SharedServer) [2024-10-28 03:44:38,830] INFO [LogLoader partition=__cluster_metadata-0, dir=/opt/kafka/data] Recovering unflushed segment 0. 0/1 recovered for __cluster_metadata-0. (kafka.log.LogLoader) [2024-10-28 03:44:38,831] INFO [LogLoader partition=__cluster_metadata-0, dir=/opt/kafka/data] Loading producer state till offset 0 with message format version 2 (kafka.log.UnifiedLog$) [2024-10-28 03:44:38,832] INFO [LogLoader partition=__cluster_metadata-0, dir=/opt/kafka/data] Reloading from producer snapshot and rebuilding producer state from offset 0 (kafka.log.UnifiedLog$) [2024-10-28 03:44:38,832] INFO [LogLoader partition=__cluster_metadata-0, dir=/opt/kafka/data] Producer state recovery took 0ms for snapshot load and 0ms for segment recovery from offset 0 (kafka.log.UnifiedLog$) [2024-10-28 03:44:38,845] INFO [LogLoader partition=__cluster_metadata-0, dir=/opt/kafka/data] Loading producer state till offset 0 with message format version 2 (kafka.log.UnifiedLog$) [2024-10-28 03:44:38,845] INFO [LogLoader partition=__cluster_metadata-0, dir=/opt/kafka/data] Reloading from producer snapshot and rebuilding producer state from offset 0 (kafka.log.UnifiedLog$) [2024-10-28 03:44:38,845] INFO [LogLoader partition=__cluster_metadata-0, dir=/opt/kafka/data] Producer state recovery took 0ms for snapshot load and 0ms for segment recovery from offset 0 (kafka.log.UnifiedLog$) [2024-10-28 03:44:38,883] INFO Initialized snapshots with IDs SortedSet() from /opt/kafka/data/__cluster_metadata-0 (kafka.raft.KafkaMetadataLog$) [2024-10-28 03:44:38,911] INFO [raft-expiration-reaper]: Starting (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper) [2024-10-28 03:44:39,054] ERROR [SharedServer id=1000] Got exception while starting SharedServer (kafka.server.SharedServer) java.lang.IllegalStateException: Configured voter set: [0, 1, 2] is different from the voter set read from the state file: [0]. Check if the quorum configuration is up to date, or wipe out the local state file if necessary at org.apache.kafka.raft.QuorumState.initialize(QuorumState.java:132) at org.apache.kafka.raft.KafkaRaftClient.initialize(KafkaRaftClient.java:375) at kafka.raft.KafkaRaftManager.buildRaftClient(RaftManager.scala:248) at kafka.raft.KafkaRaftManager.(RaftManager.scala:174) at kafka.server.SharedServer.start(SharedServer.scala:260) at kafka.server.SharedServer.startForBroker(SharedServer.scala:122) at kafka.server.BrokerServer.startup(BrokerServer.scala:175) at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:96) at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:96) at scala.Option.foreach(Option.scala:437) at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:96) at kafka.Kafka$.main(Kafka.scala:113) at kafka.Kafka.main(Kafka.scala) [2024-10-28 03:44:39,056] INFO [BrokerServer id=1000] Starting broker (kafka.server.BrokerServer) [2024-10-28 03:44:39,077] INFO [broker-1000-ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:39,077] INFO [broker-1000-ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:39,078] INFO [broker-1000-ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:39,079] INFO [broker-1000-ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:39,119] INFO [BrokerServer id=1000] Waiting for controller quorum voters future (kafka.server.BrokerServer) [2024-10-28 03:44:39,119] INFO [BrokerServer id=1000] Finished waiting for controller quorum voters future (kafka.server.BrokerServer) [2024-10-28 03:44:39,123] INFO [BrokerServer id=1000] Transition from STARTING to STARTED (kafka.server.BrokerServer) [2024-10-28 03:44:39,125] ERROR [BrokerServer id=1000] Fatal error during broker startup. Prepare to shutdown (kafka.server.BrokerServer) java.lang.NullPointerException: Cannot invoke "kafka.raft.RaftManager.leaderAndEpoch()" because the return value of "kafka.server.RaftControllerNodeProvider.raftManager()" is null at kafka.server.RaftControllerNodeProvider.getControllerInfo(BrokerToControllerChannelManager.scala:129) at kafka.server.BrokerToControllerChannelManagerImpl.newRequestThread(BrokerToControllerChannelManager.scala:242) at kafka.server.BrokerToControllerChannelManagerImpl.(BrokerToControllerChannelManager.scala:184) at kafka.server.BrokerServer.startup(BrokerServer.scala:225) at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:96) at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:96) at scala.Option.foreach(Option.scala:437) at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:96) at kafka.Kafka$.main(Kafka.scala:113) at kafka.Kafka.main(Kafka.scala) [2024-10-28 03:44:39,126] INFO [BrokerServer id=1000] Transition from STARTED to SHUTTING_DOWN (kafka.server.BrokerServer) [2024-10-28 03:44:39,126] INFO [BrokerServer id=1000] shutting down (kafka.server.BrokerServer) [2024-10-28 03:44:39,130] INFO [BrokerLifecycleManager id=1000] Skipping controlled shutdown because we are in state NOT_RUNNING. (kafka.server.BrokerLifecycleManager) [2024-10-28 03:44:39,131] INFO [BrokerLifecycleManager id=1000] beginShutdown: shutting down event queue. (org.apache.kafka.queue.KafkaEventQueue) [2024-10-28 03:44:39,132] INFO [BrokerLifecycleManager id=1000] Transitioning from NOT_RUNNING to SHUTTING_DOWN. (kafka.server.BrokerLifecycleManager) [2024-10-28 03:44:39,136] INFO Shutting down. (kafka.log.LogManager) [2024-10-28 03:44:42,003] INFO Shutdown complete. (kafka.log.LogManager) [2024-10-28 03:44:42,007] INFO [broker-1000-ThrottledChannelReaper-Fetch]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,009] INFO [broker-1000-ThrottledChannelReaper-Fetch]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,009] INFO [broker-1000-ThrottledChannelReaper-Fetch]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,009] INFO [broker-1000-ThrottledChannelReaper-Produce]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,009] INFO [broker-1000-ThrottledChannelReaper-Produce]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,009] INFO [broker-1000-ThrottledChannelReaper-Produce]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,010] INFO [broker-1000-ThrottledChannelReaper-Request]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,010] INFO [broker-1000-ThrottledChannelReaper-Request]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,010] INFO [broker-1000-ThrottledChannelReaper-Request]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,010] INFO [broker-1000-ThrottledChannelReaper-ControllerMutation]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,011] INFO [broker-1000-ThrottledChannelReaper-ControllerMutation]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,011] INFO [broker-1000-ThrottledChannelReaper-ControllerMutation]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-10-28 03:44:42,017] INFO Broker and topic stats closed (kafka.server.BrokerTopicStats) [2024-10-28 03:44:42,017] INFO [BrokerLifecycleManager id=1000] closed event queue. (org.apache.kafka.queue.KafkaEventQueue) [2024-10-28 03:44:42,018] INFO [BrokerServer id=1000] shut down completed (kafka.server.BrokerServer) [2024-10-28 03:44:42,018] INFO [BrokerServer id=1000] Transition from SHUTTING_DOWN to SHUTDOWN (kafka.server.BrokerServer) [2024-10-28 03:44:42,018] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$) java.lang.NullPointerException: Cannot invoke "kafka.raft.RaftManager.leaderAndEpoch()" because the return value of "kafka.server.RaftControllerNodeProvider.raftManager()" is null at kafka.server.RaftControllerNodeProvider.getControllerInfo(BrokerToControllerChannelManager.scala:129) at kafka.server.BrokerToControllerChannelManagerImpl.newRequestThread(BrokerToControllerChannelManager.scala:242) at kafka.server.BrokerToControllerChannelManagerImpl.(BrokerToControllerChannelManager.scala:184) at kafka.server.BrokerServer.startup(BrokerServer.scala:225) at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:96) at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:96) at scala.Option.foreach(Option.scala:437) at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:96) at kafka.Kafka$.main(Kafka.scala:113) at kafka.Kafka.main(Kafka.scala) [2024-10-28 03:44:42,023] INFO App info kafka.server for 1000 unregistered (org.apache.kafka.common.utils.AppInfoParser)