when i run airflow UI and refresh it for refreshing the DAG, the error comes which says no module pymongo. So i have tried to put it in requirements.txt and use that by volumes. Also i have installed it in cli. But the error is not going.
Is there a proper way we can use the etl technique using python on the DAG.
pip install -r requirements.txt
pip install pymongo
import pymongo
from pymongo import MongoClient
client = pymongo.MongoClient(---)
Kindly need suggestions on it.
You need to build your own Airlfow image with extra dependencies if the ones installed by default are not enough for you.
This is described in detail in the docuemntation - with plenty of examples and explanation on when and how you should build the image