Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23131

Apache Kafka Server Configuration (Consumers and Producers are in different machines)

$
0
0

I'm testing Apache Kafka between 2 vps. (First vps IP: x.x.x.x and Second vps IP: y.y.y.y) Os course I've installed zookeeper and kafka server on both of them and are same producer and consumer file Here are producer.py and consumer.py codebase:(First VPS)Producer.py:

from kafka import KafkaProducerimport json# Initialize a Kafka Producerproducer = KafkaProducer(bootstrap_servers=['x.x.x.x:9092'],                         api_version=(0,10),                         value_serializer=lambda x: json.dumps(x).encode('utf-8'))# Send a messagemessage = {"data": "from first message"}producer.send('first', value=message)# Ensure all messages are sentproducer.flush()

Consumer.py:

from kafka import KafkaConsumerimport json# Initialize a Kafka Consumerconsumer = KafkaConsumer('second',                         bootstrap_servers=['y.y.y.y:9092'],                         auto_offset_reset='earliest',                         api_version=(0, 10),                         value_deserializer=lambda x: json.loads(x.decode('utf-8')))# Print received messagesfor message in consumer:    print(message.value)

(Second VPS)Producer.py:

from kafka import KafkaProducerimport json# Initialize a Kafka Producerproducer = KafkaProducer(bootstrap_servers=['y.y.y.y:9092'],                         api_version=(0,10),                         value_serializer=lambda x: json.dumps(x).encode('utf-8'))# Send a messagemessage = {"data": "from second message"}producer.send('second', value=message)# Ensure all messages are sentproducer.flush()

Consumer.py:

from kafka import KafkaConsumerimport json# Initialize a Kafka Consumerconsumer = KafkaConsumer('first',                         bootstrap_servers=['x.x.x.x:9092'],                         auto_offset_reset='earliest',                         api_version=(0, 10),                         value_deserializer=lambda x: json.loads(x.decode('utf-8')))# Print received messagesfor message in consumer:    print(message.value)

Current status:

  • When ran producer file in First VPS, I can receive message on consumer in Second VPS
  • But when ran producer file in Second VPS, I can't receive any message on consumer in First VPS.I think it's Apache Kafka Server Configuratioin why Second VPS doesn't work, since codebase are same.

Thanks

I'd like to receive message in Second VPS from first VPS.


Viewing all articles
Browse latest Browse all 23131

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>