Data Engineer

Data Engineer

- Design and develop scalable ETL/ELT data pipelines using Python, PySpark, and Databricks.
- Work with AWS, Azure, Snowflake, and data warehousing solutions.
- Build reliable, analytics-ready datasets and ensure data quality and performance.

Requirements

Key Roles & Responsibilities

- Develop and optimize ETL pipelines and data ingestion workflows.
- Implement CDC, SCD Type 1/2, and Medallion Architecture .
- Perform data validation, transformation, and quality checks.
- Develop data models and support BI reporting using Power BI/QuickSight .
- Work with AWS Glue, S3, Azure Data Factory, Airflow, and Snowflake .
- Collaborate with stakeholders to understand requirements and deliver data solutions.

Primary Skills:

- Python, SQL, PySpark
- Databricks & Apache Spark
- ETL/ELT & Data Pipelines
- AWS (S3, Glue)
- Snowflake

Secondary Skills:

- Azure Data Factory
- Apache Airflow, dbt, Fivetran
- Data Modeling & Data Warehousing
- CDC & SCD Type 1/2
- Power BI & Amazon QuickSight
- Delta Lake, Terraform & CI/CD