About this role
Responsibilities
- Refactor and upgrade existing ML models, including NLP, generative AI and transcription models, into standardised, production-ready modular contracts.
- Engineer resilient ML workflows using DAG-based orchestration tools such as Argo Workflows and Airflow.
- Ensure robust retry logic, error handling and repeatable execution across ML workflows.
- Define, automate and maintain strict ML evaluation pipelines using golden datasets.
- Ensure new models meet baseline quality and performance thresholds before release.
- Design and implement tracking mechanisms to capture precise model, prompt and input data provenance for full auditability and reproducibility.
- Build infrastructure for shadow testing and A/B testing on live traffic.
- Implement reliable fallback and kill-switch mechanisms to support safe deployment.
- Engineer structured feedback pipelines that capture human reviews and corrections to continuously enrich and refine training datasets.
- Integrate third-party AI APIs and manage adapter interfaces.
- Implement granular telemetry to track compute costs, token usage and latency.
Requirements
- Proven track record operating at the intersection of Machine Learning, MLOps and Platform/Backend Engineering.
- Deep understanding of evaluation metrics for generative AI, LLMs and/or speech models, including Precision, Recall, F1, WER/CER, groundedness and hallucination rates.
- Strong hands-on experience with containerisation using Docker and Kubernetes, alongside modern orchestration frameworks.
- Experience building observable ML systems with logging, monitoring and strict cost-attribution tracking.
- Familiarity with data provenance, compliance standards and building fail-safe mechanisms for sensitive data and AI outputs.