Data Engineer

Key Responsibilities

- Design, develop, and support batch and incremental data ingestion pipelines into the data lake.

- Develop and optimize complex SQL queries, data transformations, and ETL workflows.

- Build, maintain, and monitor Airflow DAGs for orchestration and workflow automation.

- Analyze and troubleshoot data quality, data reconciliation, and pipeline execution issues.

- Perform end-to-end data lineage analysis and trace data from source systems to downstream consumers.

- Debug data discrepancies by investigating source systems, transformation logic, and target datasets.

- Support data migration and modernization initiatives from legacy platforms to cloud-based data platforms.

- Work closely with business stakeholders, architects, and upstream/downstream teams to understand data requirements.

- Validate data completeness, accuracy, and consistency across multiple systems.

- Participate in production support activities and resolve data-related incidents within SLA timelines.

- Develop reusable data engineering components and recommend process improvements.

Preferred Candidate Profile

- Strong analytical and problem-solving skills.
- Ability to independently investigate and resolve data issues.
- Good communication and stakeholder management skills.
- Experience in cloud data lake or warehouse environments.
- Exposure to customer, transactional, or enterprise-scale datasets is preferred.

Screening Focus Areas

- Advanced SQL/PLSQL
- ETL & Data Pipeline Design
- BigQuery Concepts
- Data Debugging & Root Cause Analysis
- Airflow DAG Development
- Python Basics
- Data Migration & Data Validation
- Source-to-Target Traceability and Data Lineage

Must Have: SQL/PLSQL, BigQuery/Cloud Data Warehouse, ETL, Data Analysis & Debugging
Good to Have: Airflow DAG Development, Python, GCP Services

Kafka, Pub/Sub optional