AWS & Airflow Developer

Where Ambition Meets Innovation

At LPL’s Global Capability Center , you'll find a collaborative culture where your voice matters, integrity guides every decision, and technology fuels progress. Your skills, talents, and ideas will redefine what's possible. LPL's success reflects its exceptional employees, who together pursue one noble purpose: empowering financial advisors to deliver personalized advice for all who need it. We’re proud to be expanding and reaching new heights in Hyderabad.

Join us as we create something extraordinary together.

Job Overview:

We are seeking a highly motivated AWS & Airflow Developer with 4-6 years of experience in designing, developing and supporting cloud-based data pipelines, workflow orchestration and data integration solutions. The ideal candidate will have hands-on expertise in Python, Apache Airflow, AWS cloud services, SQL and ETL/ELT development , with a strong understanding of data engineering best practices.

This role will be part of the Data Engineering team responsible for building scalable, secure and reliable data pipelines that support enterprise analytics, reporting and downstream applications.

Responsibilities:

- Develop data transformation logic using Python, SQL and PySpark where applicable.
- Design, develop and maintain scalable and reliable data pipelines using Apache Airflow and AWS services.
- Build and optimize ETL/ELT workflows to ingest, transform and process data from various source systems.
- Develop and manage Airflow DAGs, task dependencies, scheduling, monitoring and error handling.
- Implement data ingestion solutions using AWS services such as S3, Glue, Lambda, Step Functions, EventBridge, Athena and RDS/Aurora .
- Monitor data pipeline performance and troubleshoot workflow failures, data quality issues and operational incidents.
- Collaborate with architects, business analysts and application teams to understand data requirements and deliver scalable solutions.
- Support production deployments, incident management, root cause analysis and performance tuning activities.
- Implement data validation, reconciliation and quality checks to ensure accurate and reliable data delivery.
- Participate in cloud modernization and migration initiatives to AWS-based platforms.
- Contribute to CI/CD implementation, automation, documentation and engineering best practices.
- Develop and maintain operational dashboards, monitoring and alerting solutions for pipeline observability.
- Participate in Agile ceremonies, sprint planning and technical design discussions.

What are we looking for?

We are looking for strong AWS & Airflow Developer who can deliver reliable, scalable, and high-quality data solutions. The ideal candidate thrives in a fast-paced environment, is passionate about data engineering, and is comfortable working across teams to solve complex data problems.

Requirements:

- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- 4-6 years of experience in Data Engineering, ETL Development, or Data Integration.
- Strong proficiency in Python programming.
- Strong hands-on experience with Apache Airflow for workflow orchestration and scheduling.
- Experience with AWS cloud services, including: Amazon S3, Glue, Lambda, Step Functions, Athena, EventBridge, RDS/Aurora PostgreSQL, CloudWatch

- Strong SQL skills with experience in relational databases such as PostgreSQL, Oracle, or SQL Server.
- Experience developing and supporting batch and event-driven data pipelines.
- Understanding of data modeling, data warehousing and ETL/ELT concepts.
- Experience with source control systems such as Git.
- Knowledge of monitoring, logging and troubleshooting distributed data workflows.
- Strong analytical, problem-solving and debugging skills.
- Excellent verbal and written communication skills.

Preferences:

- Experience with PySpark and distributed data processing frameworks.
- Knowledge of AWS EMR, Redshift, or Snowflake.
- Experience working with Docker and containerized applications.
- Exposure to Kubernetes or Amazon EKS.
- Familiarity with REST API integration and event-driven architectures.
- Experience implementing CI/CD pipelines using GitHub Actions, GitLab CI, or AWS CodePipeline.
- Understanding of data governance, metadata management and data lineage concepts.
- Experience working in Agile/Scrum environments.
- Exposure to infrastructure-as-code tools such as Terraform or AWS CloudFormation.

Core Competencies:

- Strong ownership and accountability.
- Excellent analytical and troubleshooting skills.
- Ability to work independently and within cross-functional teams.
- Strong attention to detail and commitment to quality.
- Effective communication and stakeholder management skills.
- Continuous learning mindset and adaptability to emerging technologies.
- Ability to manage multiple priorities in a fast-paced environment.

LPL Global Business Services, LLP - PRIVACY POLICY