I moved to this company after the previous company left Russia. Working with Yandex Cloud, writing dags in AirFlow, building storage architecture, optimizing existing flows is the main tasks of this job. I work with a large amount of data, various data sources: Mongo DB, MS SQL, PostgreSQL, ClickHouse.

It was the most interesting work experience. I didn't want to quit, but SoftwareONE left Russia. I work in an international team of IT specialists on the Big Data stack: Databticks and Azure Cloud. Communication with foreign customers and colleagues was in English. I wrote notebooks like Jupyter in Python (PySpark) and SQL for extracting, loading, processing data from various sources: SQL / NoSQL databases, REST API, XML. Our storage consisted of raw/bronze/silver data layers.

I built dashboards in Redash. My main project is getting data from ServiceNow: diving into the subject area, learning the REST API, developing and writing a loader. Orchestrator - Azure Data Factory (ADF). Git is built into the Azure cloud.