when i run airflow UI and refresh it for refreshing the DAG, the error comes which says no module pymongo. So i have tried to put it in requirements.txt and use that by volumes. Also i have installed it in cli. But the error is not going.
Is there a proper way we can use the etl technique using python on the DAG.
pip install -r requirements.txt pip install pymongo import pymongo from pymongo import MongoClient client = pymongo.MongoClient(---)
Kindly need suggestions on it.
You need to build your own Airlfow image with extra dependencies if the ones installed by default are not enough for you.
This is described in detail in the docuemntation - with plenty of examples and explanation on when and how you should build the image