Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23305

Python kafka getting consumer messages in an async manner

$
0
0

I'm using kafka-python. I'm trying to define for message in kafka_consumer to run async in the background. Currently, it seems to me that it runs in sync thus blocking other methods. Does kafka-python support this? If not, what are my alternatives? I definitely need this to run in an async manner.

My code:

import asynciofrom kafka import KafkaConsumerasync def read_messages(kafka_consumer: KafkaConsumer) -> None:    while True:        for message in kafka_consumer:            print (f"message is {message}")async def main() -> None:    consumer = KafkaConsumer('testing', bootstrap_servers='localhost:9092', api_version=(0, 11, 5))    asyncio.ensure_future(read_messages(consumer))if __name__ == "__main__":    loop = asyncio.new_event_loop()    asyncio.set_event_loop(loop)    try:        loop.run_until_complete(main())        loop.run_forever()        loop.close()

Thanks!


Viewing all articles
Browse latest Browse all 23305

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>