Juan Camilo Calero Espinosa
Senior big data cloud engineer
Professional profile
In my seven-year career, I’ve grown to a senior position where I’ve significantly contributed to the modernization of widely-used analytical systems. My expertise encompasses a broad range of programming and data processing methodologies complemented by a master’s degree in natural language processing. My professional path has been diverse, with impactful experiences in sectors such as national defense, healthcare, finance, and research. As I see
Work experience
Processing of millions of members claims from thousands of health providers for the purpose of assisting in making decisions about the care of individual patients and improving health providers performance. The analytics was implemented in Spark (AWS Glue) in conjunction with several cloud services like AWS step functions, S3, lambda and EC2.
Major achievementThanks to the Spark migration and all the optimizations made from the legacy version in SQL server, the execution time went from weeks on expensive SQL instances to just a couple of hours on AWS Glue. Top performer and Hall of Fame for refactoring and optimizing several Spark modules
Conversion of code from SAS to PySpark, automation through Airflow, and migration to a Lakehouse architecture.
Major achievementTask automation through Airflow and migration to a Lakehouse architecture using Delta Lake
Extraction and indexing of unstructured documents, creation of a knowledge base, development of a document search engine, and deployment of BERT neural networks.
Major achievementDevelopment of a document search engine and training and deployment of BERT neural networks
Education experience
Universidad nacional de colombia | Maestría
