Senior Data Engineer

# Job Role: Senior Data Engineer
Location: Hyderabad, India Experience: 5+ Years
##

## POSITION OVERVIEW
Anblicks is seeking an experienced Senior Data Engineer to design, develop, and optimize scalable data infrastructure that integrates large volumes of data from diverse sources into reliable, secure, and analytics-ready platforms. The ideal candidate will have strong expertise in distributed data processing, data pipeline architecture, identity resolution, data modeling, and modern cloud data technologies. In this role, you will take ownership of end-to-end data engineering solutions, architect high-performance ETL/ELT pipelines, and develop scalable data integration frameworks. You will collaborate with Data Scientists, platform engineers, and cross-functional stakeholders to deliver governed, privacy-conscious data solutions that support analytics, reporting, and machine learning use cases. You will also contribute to architectural decisions, establish engineering best practices, and mentor junior engineers.
## DUTIES AND RESPONSIBILITIES

- Own the technical design, development, and maintenance of scalable data pipelines supporting complex data engineering and analytics use cases.
- Architect and implement robust ETL/ELT pipelines to ingest, transform, standardize, and process large volumes of structured and semi-structured data.
- Design and maintain scalable data architectures for entity resolution, identity matching, record linkage, and integration of data from multiple sources.
- Develop match/merge logic and identity resolution workflows to connect disparate records while maintaining data accuracy, consistency, and traceability.
- Work with graph databases or graph processing frameworks to support high-cardinality data relationships and large-scale identity graph processing, where applicable.
- Build and optimize data solutions using modern cloud and data platforms such as Snowflake, Databricks, and AWS.
- Integrate data from external systems, partner platforms, APIs, and third-party sources using reliable and reusable data integration patterns.
- Implement data governance, privacy, security, consent management, and access control practices to ensure compliant handling of sensitive data.
- Collaborate with Data Scientists and analytics teams to design feature pipelines and deliver well-modeled, high-quality datasets for analytical and machine learning applications.
- Establish data quality, lineage, observability, and monitoring frameworks to identify data inconsistencies and ensure pipeline reliability.
- Optimize distributed data processing workloads for performance, scalability, resource utilization, and cost efficiency.
- Lead technical design discussions and architecture reviews, define coding standards, and promote engineering best practices.
- Mentor junior Data Engineers, conduct code reviews, and support technical problem-solving across the team.
- Develop and maintain technical documentation, architecture diagrams, data models, and operational runbooks.
- Evaluate and adopt emerging technologies, including AI-assisted development tools, to improve engineering productivity while maintaining code quality and security.

QUALIFICATIONS

Required Qualifications

- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- 5+ years of hands-on experience in Data Engineering, including ownership of complex data pipelines, distributed processing systems, or data platform architecture.
- Strong proficiency in SQL and at least one programming language, preferably Python, Scala, or Java.
- Demonstrated experience designing, developing, and operating large-scale ETL/ELT pipelines and distributed data processing workflows.
- Hands-on experience with cloud and data platforms such as Snowflake, Databricks, and AWS.
- Experience with workflow orchestration tools such as Apache Airflow or equivalent platforms.
- Practical experience with entity resolution, identity matching, large-scale record linkage, or graph-based data processing.
- Familiarity with graph databases or graph processing technologies such as Neo4j, TigerGraph, or equivalent solutions.
- Strong understanding of data modeling, data quality, data governance, and pipeline monitoring.
- Working knowledge of data privacy, security, consent management, and privacy-conscious data processing practices.
- Experience designing scalable, maintainable, and cost-efficient data architectures.
- Strong analytical, problem-solving, communication, and stakeholder-management skills.
- Ability to communicate technical architecture decisions effectively to engineering teams and business stakeholders.

Preferred Qualifications

- Experience transforming custom data engineering solutions into standardized, reusable, and production-ready data products.
- Experience in AdTech, MarTech, digital analytics, audience analytics, or other data-driven product environments.
- Familiarity with integrating data from external platforms, APIs, publishers, demand-side platforms (DSPs), supply-side platforms (SSPs), or retail media networks.
- Exposure to platforms such as Meta, Google, Amazon Marketing Cloud (AMC), or The Trade Desk for data integration and analytics use cases.
- Experience with streaming and event-processing technologies such as Apache Kafka or AWS Kinesis.
- Familiarity with data clean room technologies and privacy-preserving data