QA Engineering Lead (AI)

We’re looking for QA Engineering Leads to set technical direction and lead major quality initiatives for our GenAI products at Meta. You enjoy solving difficult, ambiguous quality problems and building evaluation systems that enable multiple teams to ship reliable AI experiences. In this hands-on role, you will shape quality strategy across related products and platforms, lead the design and implementation of shared testing and evaluation capabilities, and help other engineers deliver high-quality work.

Responsibilities

Define and drive the technical quality strategy and roadmap for a major Gen AI initiative spanning multiple engineering teams, balancing product risk, delivery speed, and engineering investment
Lead and perform hands-on testing of consumer-facing and business-facing applications, APIs, and AI models, adapting quality priorities and test coverage in a fast-paced development environment
Architect and implement shared AI evaluation and test automation systems that support multiple models, products, and platforms, while directly solving the most challenging technical problems
Establish consistent evaluation methods, benchmark design, product-aligned weighted scoring frameworks, and quality standards for model behavior and end-to-end product experiences, working with research, engineering, and product partners
Lead the development of reliable evaluation signals, including calibration of human and automated judgments, analysis of measurement uncertainty, and investigation of gaps between offline evaluations and real-world outcomes
Identify systemic quality risks across models, data, applications, and infrastructure, and lead cross-team efforts to address their root causes
Drive quality planning for major launches and phased rollouts, aligning teams on success criteria, dependencies, monitoring, and regression response
Define metrics that connect model performance, product reliability, and user experience, and use them to guide quality priorities and investment decisions
Lead adoption of reusable evaluation frameworks, automation, and development practices that improve quality while reducing testing costs and time to feedback
Partner with engineering, infrastructure, research, product, and data teams to resolve technical trade-offs and influence product and system designs before quality issues emerge
Mentor QA engineers and technical leads, provide guidance, and raise the quality of engineering across the broader team

Qualifications

Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
Currently has, or is in the process of obtaining, a Bachelor’s degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience. A degree must be completed prior to joining Meta
10+ years of quality assurance, test engineering, or software engineering experience, including substantial experience in building quality engineering systems
3+ years of experience in AI model testing or evaluation, including designing evaluation frameworks and establishing model quality standards
Experience successfully leading a major technical quality initiative across multiple teams, from problem definition and architectural design through implementation, rollout, and measurable outcomes
Experience architecting and implementing scalable testing, automation, or evaluation systems in Python, PHP, Java, C/C++, or an equivalent programming language
Experience solving complex quality and reliability problems across client applications, backend services, and distributed systems
Hands-on experience testing consumer-facing and business-facing applications and APIs, and evaluating AI models
Hands-on experience with risk-based test strategy, performance and stress testing, and the design and maintenance of client and server test automation
Experience evaluating the reliability and limitations of quality metrics and using data to guide technical decisions
Experience leading quality engineering in fast-paced development teams, adapting test strategies and priorities as requirements evolve
Experience setting technical direction, resolving ambiguous requirements and cross-team dependencies, and influencing stakeholders without direct authority
Experience communicating complex technical problems, quality risks, and investment trade-offs to technical and non-technical stakeholders
Experience mentoring engineers or technical leads and driving the adoption of quality engineering practices beyond a single project Experience leading the evaluation strategy for generative text, image, voice, or multimodal products at scale
Experience designing shared evaluation infrastructure or testing frameworks and leading their adoption across multiple engineering teams
Experience designing evaluation systems that combine human review, model-based judgments, adversarial testing, and automated regression detection
Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
Demonstrated improvements in product reliability, evaluation coverage, regression prevention, or engineering efficiency across a major product initiative
Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
Experience leading cross-platform quality efforts within large engineering organizations and building consensus on technical standards and priorities
Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
Experience applying frontier AI model evaluation practices, including weighted scoring frameworks for assessing model quality and performance
Experience with statistical evaluation methods, benchmark validity, evaluator calibration, and connecting offline evaluation results to production quality
Experience partnering with AI research teams and translating evolving research capabilities into scalable product quality practices