I have a simple app in nodejs where I'm trying to create a transaction in KafkaJS. I've followed the docs and have my client connected and a transactional producer created with a transaction id.

The problem is that when I try to run the transaction example in the docs, I get this error:

    KafkaJSError: The producer is disconnected  
    at validateConnectionStatus (/usr/local/apps/app/node_modules/kafkajs/src/producer/messageProducer.js:31:15)

I've looked online but couldn't find any related issues. I'm confused why this is occuring because I create the transaction producer right before using it. I'm also pretty sure this is not a problem with connecting to the kafka broker in any way as when I create a normal producer, it sends the message with no problem.

Here is the code:

       const client = new Kafka({
            clientId: 'app-transactional-clienet',
            brokers: ['host.docker.internal:9092'],
        })
        const producer = client.producer({
            transactionalId: 'transaction.connectedservice.producer',
            maxInFlightRequests: 1,
            idempotent: true
        })

        const transaction = await producer.transaction();

        try {
            const connectedAccountMsg: ConnectedAcccountMsg = {
                userId: req.session.userId,
                accountId: profile.id,
                accountUsername: profile.username,
                accountType: 'twitter',
                authorized: true,
            };
            await transaction.send({ 
                topic: 'user.connectedaccount.status',
                messages: [{ 
                    value: JSON.stringify(connectedAccountMsg) 
                }],
            })

            await transaction.commit()
            return done(null, profile);
        } catch (error) {
            console.log(error)
            await transaction.abort()
        }