About this role
Job description _
About the Role _
Join our client and help bring AI-driven quality control to the data pipelines that power their business. You’ll analyze how ETL processes are validated today and design an AI- powered validation platform on AWS that detects anomalies, reconciles source and target data, and explains issues before they reach downstream consumers. This role blends data engineering , machine learning / anomaly detection , and generative AI (Amazon Bedrock) , giving you the opportunity to shape both the data quality and governance strategy and the roadmap for AI-driven validation across additional workloads.
A day in this role _
- Analyze existing ETL validation processes and identify automation opportunities .
- Develop AI-driven validation frameworks for data pipelines.
- Implement anomaly detection for continuous data quality monitoring .
- Compare source data against ETL outputs to identify and track discrepancies.
- Automate record count, completeness, and consistency checks across pipelines.
- Generate AI-assisted validation reports that explain detected issues in clear, actionable terms.
- Integrate AI validation capabilities into AWS data platforms and workflows (Glue, Lambda, Redshift, Athena, S3).
- Evaluate how AI can be used to deliver in-depth QA analysis of deliverables .
- Collaborate with data engineering, QA, and business teams to define data quality rules, thresholds, and SLAs .
- AI-powered data validation platform .
- Automated ETL quality control framework .
- Anomaly detection models and monitoring dashboards .
- Validation reports and root-cause analysis capabilities.
- Data quality KPIs and governance recommendations .
- Roadmap for expanding AI-driven validation across additional workloads.
The expertise requested _
- Bachelor’s degree in Computer Science, Data Engineering, Software Engineering, or related field (or equivalent experience).
- 5+ years of experience in data engineering , ETL development, or data quality roles.
- Strong programming skills in Python and advanced SQL .
- Hands-on experience with Amazon Bedrock and building generative AI solutions (prompt design, LLM-based analysis and reporting).
- Hands-on experience with AWS Lambda, AWS Glue, Amazon Redshift, Amazon Athena, and Amazon S3 .
- Practical experience with machine learning and anomaly detection techniques applied to data (statistical, time-series, or ML-based).
- Solid understanding of data quality and data governance principles (completeness, accuracy, consistency, lineage).
- Experience designing source-to-target reconciliation and ETL testing strategies.
- Solid understanding of Agile, SDLC, and modern engineering best practices .
Desired or nice to have expertise _
- Experience with data quality frameworks such as AWS Glue Data Quality, Great Expectations, or Deequ .
- Experience building monitoring dashboards (Amazon QuickSight, CloudWatch, or similar).
- Familiarity with Amazon SageMaker or other ML platforms for model training and deployment.
- Exposure to agentic AI workflows and AI-assisted development tools (e.g., GitHub Copilot, Claude Code, Cursor).
- AWS certifications (e.g., Data Engineer – Associate, Machine Learning – Specialty )
Our Core values are focus to inclusion and diversity, all qualified applicants will be considered for employment and will go thru a fair recruitment process regardless of their race, religion, gender identity, sexual orientation, national origin or disability status.
Your journey with us begins here!!