Empresas
Empleos
  • Sobre nosotros
  • Soluciones
    • Publicación de vacantes
      Publica tu vacante y recibe candidatos calificados en 48h.
    • Evaluación de candidatos
      500+ pruebas técnicas y psicológicas, más anti-fraude.
    • Headhunting
      Búsqueda ejecutiva a la medida de principio a fin.
    • Nómina + EOR
      Dispersión de nómina y EOR en más de 15 países de LATAM.
  • Precios
  • Empleos

0

210
Vistas
Consume Kafka messages from a different container in python

Below is my docker-compose file:

   version: "3"
    services:
      app:
        image: app
        restart: always
        ports:
          - "8001:8081"
        depends_on:
          - kafka
          - zookeeper
          - consumer
        environment:
          - KAFKA_HOST = kafka

  zookeeper:
    image: "wurstmeister/zookeeper:latest"
    ports:
        - "2181:2181"
    hostname: zookeeper   

  kafka:
    image: "wurstmeister/kafka:2.12-2.2.0"
    ports:
      - "9092:9092"
    hostname: kafka
    links: 
     - zookeeper:zookeeper
    environment:
      KAFKA_CREATE_TOPICS: "Topic01:2:2" #TOPIC:PARTITON:REPLICATION
      KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: "60000"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_LISTENERS: 'PLAINTEXT://:9092'
      KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://kafka:9092'

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  consumer:
    image: consumer:latest
    build:
      context: ./consumer
    ports:
      - "8283:8283"

Text stream is generated using container for app image and is able to produce the messages.

It is verified by navigating inside the kafka container:

docker exec -it <CONTAINER ID of Kafka Image> /bin/bash

when I manually run the below script inside the container

kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic Topic01 --from-beginning

I can see the chunk of text that is being produced.

But when I try to read the same data from the container of the image consumer, It gives me blank,

Method tried:

docker exec inside the consumer container, and ran the below python code:

from kafka import KafkaConsumer
consumer = KafkaConsumer('Topic01', bootstrap_servers='kafka:9092')
for messages in consumer:
        print(messages)

It prints nothing.

Is it because my docker-compose file is wrong or the mistake in the python code?

over 4 years ago · Santiago Trujillo
1 Respuestas
Responde la pregunta

0

According to Kafka documentation for listeners:

Listener List - Comma-separated list of URIs we will listen on and the listener names. If the listener name is not a security protocol, listener.security.protocol.map must also be set. Specify hostname as 0.0.0.0 to bind to all interfaces. Leave hostname empty to bind to default interface. Examples of legal listener lists: PLAINTEXT://myhost:9092,SSL://:9091 CLIENT://0.0.0.0:9092,REPLICATION://localhost:9093

You are setting KAFKA_LISTENERS to PLAINTEXT://:9092, so it binds to default interface, which might not be accessible from the outside of your Kafka container (while at the same time it works fine with console consumer using localhost). Give it a try and specify KAFKA_LISTENERS as PLAINTEXT://0.0.0.0:9092, to check if you consumer starts to consume messages.

over 4 years ago · Santiago Trujillo Denunciar
Responde la pregunta
Encuentra empleos remotos

¡Descubre la nueva forma de encontrar empleo!

Top de empleos
Top categorías de empleo
Empresas
Publicar vacante Precios Comercial
Legal
Términos y condiciones Política de privacidad
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomiéndame algunas ofertas
Necesito ayuda