Senior Technical Product Manager – Agent Evaluation & Observability

Agentic Partner Membership Experiences

The core of our company is not just the products we offer, but the personal connections our customers have with our brand. You can create digital experiences, from payments to rewards to servicing, that keep us connected to our customers and serve them at every touchpoint. From digital servicing and lifestyle features to secure, frictionless payment options, you can make a meaningful difference in our customers’ lives and help set us apart in the industry. Find your place in digital products on #TeamAmex.

The Amex Digital Labs team drives innovation at American Express by originating, incubating, and managing products and platforms that are essential to our customers’ digital lives. We are a fast-paced team that leverages emerging technologies to develop digital payment and financial services products and engaging Membership experiences for Card Members. We often collaborate with technology companies and strategic partners to bring products to the market.

We are looking for a Senior Technical Product Manager to lead the roadmap for tools and workflows that hlep teams test and improve agent-enabled customer experiences. You will help teams measure how well agent's complete customer tasks, understand failures and use the findings to improve their experiences.

You will own the product roadmap, requirements and adoption of shared evaluation tools and workflows. Working with engineering and enterprise platform teams, you will build on existing capabilities to help product teams assess agent performance and decide what to improve. Domain product owners will remain accountable for release decisions.

Minimum Qualifications

- 6-8+ years of experience in product management, product development, software engineering, technical platforms, analytics, quality tooling, or AI products.
- Experience owning evaluation, testing, observability, analytics or related technical products through launch, adoption and iteration, with measurable user or business outcomes.
- Practical understanding of AI or agent-enabled systems, including nondeterministic behavior, tool or API interactions, and the need for both automated tests and human judgment.
- Ability to define metrics, interpret evidence and identify sampling, coverage, or measurement limitations. Familiarity with traces, logs, APIs and technical troubleshooting.
- Strong product discovery, roadmap and prioritization skills, including translating ambiguous quality problems into usable tools and repeatable workflows.
- Cross-functional leadership and clear communication, with the ability to balance customer outcomes, engineering constraints and privacy or risk requirements.

Preferred Qualifications

- Experience with LLM or agent evaluation, scenario design, human review, model-assisted scoring or production failure analysis.
- Familiarity with MCP, function calling, telemetry standards or evaluation frameworks; ability to use code or queries to investigate evidence.
- Experience building shared capabilities adopted by multiple teams or delivering products in financial services or another regulated industry.
- Bachelor’s degree or equivalent experience.

Employment eligibility to work with American Express in the United States is required as the company will not pursue visa sponsorship for these positions.