About this role
You will own the behaviour
of a specific AI system or workflow inside a larger product. You will work with
a team and a defined set of internal stakeholders. Your primary stakeholders
are business teams who will use the workflow and engineering teams who will
build it. You will be responsible for creating a spec that the business teams
can approve and engineering teams can build efficiently. Your success is
measured by whether the AI system behaves as specified in production over time.
You will work with QA and
engineering to build and review evaluations. This is a critical part of the
role. Evaluations are how you verify the AI system keeps behaving as specified
once it's live.
Building a working
understanding of how AI systems work, and what that means for product and
analytics, is critical to this role. You will also act as a proxy for the
business team and run UAT.
Alongside that, the role
carries the usual weight of product ownership: picking up an unfamiliar domain
fast, holding the line on scope, and keeping stakeholders aligned on what is
shipping and when.
What you will own
Backlog
and delivery
- Own and prioritise the backlog for your
system; run refinement, planning, and acceptance with the delivery team.
You could also be running kanban boards for your team for a fast paced
project.
- Turn business requirements into AI
system requirements for the engineering team.
- The requirements should meet the Jeavio
guardrails of AI governance, privacy, security and responsible AI
standards, and highlight it when a requirement falls outside them.
- Support adoption: training, workflow
changes, review Evals, user manuals, release notes etc. that might be
required to get the feature to production.
PO
Competency
- Pick up a new domain quickly and get to
the point where you can ask informed questions about the business.
- Should be able to adopt AI tools like
Claude or similar to work upon eliciting and/or prototyping requirements
(including UX designs) independently or collaboratively with technical
peers.
- Should be able to participate in and
lead (where appropriate) discovery sessions to be able to understand the
lay of the land, vision and translate it back into documentation that can
be used by both the business and engineering teams.
- You should be able to figure out which
are the relevant metrics for users and business.
- AI can be more accurate but slower, more
flexible but less predictable, cheaper but occasionally wrong. You should
be able to weigh tradeoffs from a product point of view - i.e. whether a
certain trade off is okay or breaks user experience.
- The Evals you define must align with the
product requirements.
Stakeholder
Management
- Steer stakeholders towards the better
option or push back on scope during scope based negotiations by working
alongside the engineering team.
- Be able to influence the roadmap by
understanding what the team can deliver vs what the stakeholders want.
- Should also be able to present project
status to stakeholders and give an update on delivery timelines by
aligning with the technical teams.
- Should be able to understand and
communicate the limitations of the system, clearly to stakeholders.
Behavioural
specification
- Write acceptance criteria for
probabilistic outputs.
- Work with Engineering and QA colleagues
to define system guardrails
- Specify the confidence thresholds that
trigger human review, a fallback path, or a graceful exit.
- Read and assess system prompts well
enough to tell whether they match the spec.
- Write the spec for two readers at once:
the AI engineer who implements it and the business user who will live with
it.
Evaluation
and human oversight
- Own the evaluation dataset for your
system across its lifecycle.
- Interpret results and translate them for
non technical stakeholders.
Security
& Compliance:
- Define and prioritize security
requirements in product backlog
- Ensure data protection, privacy, and
compliance with ISO 27001 policies
- Collaborate with engineering and
security teams for secure product delivery
- Manage risks related to features,
integrations, and data handling
- Support audit readiness and continuous
security improvements
What we are looking for
Required
- 6+ years in product ownership, product
management, or business analysis, including at least two years owning a
backlog with a delivery team.
- Direct experience shipping or operating
an AI/ML or LLM-based feature in production — not only prototyping.
- Fluency in AI tools - ChatGPT or Claude
including an understanding of skills, plugins, and context
management.
- Demonstrable understanding of how LLMs
work and how they impact product decisions
- Demonstrated ability to write acceptance
criteria for a non-deterministic output.
- Working fluency in Evals.
- Judgment about where human in the loop
use case fits in.
- Strong written communication.
- Should be able to keep up with evolving
trends and is open to unlearn and learn quickly.
Good
to have
- Experience with agentic systems: tool
use, multi-step orchestration, retrieval, or observability tooling.
- Familiarity with our stack: AWS, Claude,
LangSmith, JIRA.