Beschreibung

I am a Data Engineer with 3.5+ years of experience in designing, building, and optimizing data pipelines and cloud-based data solutions. I specialize in:

ETL & Data Integration: Python, PySpark, SQL for high-volume, scalable data processing.

Cloud Data Platforms: Databricks, AWS (S3, Lambda, Step Functions, Glue), Azure(ADLS, Data Factory, Key vault, Databricks).

Data Modeling & Warehousing: Snowflake. Creating efficient, structured data warehouses for analytics and reporting.

Branchenexpertise

Sprachen

Englisch
Muttersprachlich oder zweisprachig
Deutsch
Grundkenntnisse

Arbeitsortpräferenzen

Vor Ort möglich

Frankfurt am Main (bis zu 50 km), Düsseldorf (bis zu 50 km), Stuttgart (bis zu 40 km)

Deloitte,
Data Engineer
Oktober 2024 - Mai 2025 (7 Monate)
Bengaluru, Karnataka, India
• - Built and maintained ETL pipelines using AWS Glue writing Pyspark code, handling flat file and API ingestion.
• - Integrated datasets from multiple sources to ensure data consistency and reduce complexity.
• - Enhanced Amazon Aurora PostgreSQL performance by implementing pg_partman based table partitioning.
• - Implemented AWS Lambda functions to perform serverless transformations and trigger workflows, improving scalability and reducing infrastructure overhead.
• - Used AWS Step Functions for orchestrating modular, maintainable data workflows.
• - Resolved production issues to ensure minimal downtime in pipelines.
AWS Glue Pyspark ETL
PwC Acceleration Center,
Data Engineer
August 2021 - Oktober 2024 (3 Jahre und 2 Monate)
Bengaluru, Karnataka, India
Azure Databricks & ADF: - Developed scalable ETL solutions using Azure Databricks and Azure Data Factory. - Designed optimized Apache Spark workflows via Databricks notebooks to reduce runtime. - Implemented Delta Lake for transactional reliability and version control in data lakes. - Automated data workflows, reducing manual intervention by 40%. - Mentored junior engineers on Spark and Databricks best practices.

EDW Data Modernization Project: - Developed cloud data pipelines with Informatica IICS and PushDown Optimization. - Migrated data from Teradata, FTP, and Oracle to Snowflake Data Warehouse. Implemented
SCD Type 1 & 2 logic. - Refactored Teradata views (SQL) and ported logic to Snowflake. - Handled deployments across Dev → SIT → Test → Prod using CI/CD tools. - Resolved defects across IICS taskflows and Snowflake views to ensure data accuracy.
Databricks Azure Data Factory SQL Snowflake