Airflow Engineer

Job Requirements

At Quest Global, it’s not just what we do but how and why we do it that makes us different. With over 25 years as an engineering services provider, we believe in the power of doing things differently to make the impossible possible. Our people are driven by the desire to make the world a better place—to make a positive difference that contributes to a brighter future. We bring together technologies and industries, alongside the contributions of diverse individuals who are empowered by an intentional workplace culture, to solve problems better and faster.

Key Responsibilities

Work Mode: [On-site / Hybrid]
Shift: Rotational (24/7 support coverage)

Job Summary

We are seeking a proactive and technically skilled Airflow Engineer with 3–4 years of experience to manage, deploy, and support the Apache Airflow application across multiple AWS EKS clusters. The ideal candidate will be responsible for maintaining high availability of the Airflow platform, assisting consumers with orchestration needs, and ensuring smooth operational support around the clock.

Key Responsibilities

- Deploy and maintain Apache Airflow applications in AWS EKS clusters.
- Provide 24/7 support for Airflow consumers, ensuring SLAs are met.
- Monitor system performance and troubleshoot issues related to DAG execution, scheduler latency, or pod lifecycle.
- Implement and manage CI/CD pipelines for Airflow deployment.
- Collaborate with data engineering teams to onboard and support DAGs and orchestration logic.
- Maintain and manage integrations with AWS services like S3, RDS, Load Balancer, IAM, and CloudWatch.
- Work with Kubernetes manifests, Helm charts, and ServiceAccount configurations to ensure secure and scalable deployments.
- Create and maintain dashboards, alerts, and operational documentation.

Required Skills

- 3–4 years of hands-on experience in deploying and supporting Apache Airflow (v2.9.x preferred).
- Strong knowledge of Python or Java , along with Shell scripting .
- Hands-on experience with AWS services: EKS, S3, RDS, CloudWatch, Load Balancers, IAM.
- Experience with Kubernetes and Docker for container orchestration and deployment.
- Familiarity with CI/CD tools and automation pipelines (GitHub Actions, Jenkins, ArgoCD, etc.).
- Comfortable writing and managing configuration in YAML .

Good to Have

- Working knowledge of Terraform for infrastructure as code.
- Experience with observability tools like Prometheus, Grafana, or Datadog.
- Exposure to managing multi-cluster environments and handling Airflow migrations or upgrades.

Soft Skills

- Excellent communication and problem-solving skills.
- Willingness to work in rotational shifts and respond to incidents during off-hours.
- Strong sense of ownership and accountability in production support environments.
- Ability to collaborate with cross-functional teams in a fast-paced environment.

About the Team

You’ll be part of a high-impact team that owns the deployment and lifecycle management of Airflow across cloud infrastructure, ensuring reliability, performance, and security for thousands of DAGs powering key data pipelines.

We are known for our extraordinary people who make the impossible possible every day. Questians are driven by hunger, humility, and aspiration. We believe that our company culture is the key to our ability to make a true difference in every industry we reach. Our teams regularly invest time and dedicated effort into internal culture work, ensuring that all voices are heard.

We wholeheartedly believe in the diversity of thought that comes with fostering a culture rooted in respect, where everyone belongs, is valued, and feels inspired to share their ideas. We know embracing our unique differences makes us better, and that solving the worlds hardest engineering problems requires diverse ideas, perspectives, and backgrounds. We shine the brightest when we tap into the many dimensions that thrive across over 21,000 difference-makers in our workplace.