About this role
Role summary
We are looking for a technical architect to
design and build our internal agentic harness: the shared framework that lets
AI agents plan, call tools, read company data, act in our systems and be
evaluated and governed safely. On top of that framework, the architect will
lead delivery of the first internal agentic applications (for example, contract
analysis, integration support for SAP S/4HANA and Ariba, pre-sales and solution
design, and support triage).
The harness is the product. Individual
agents should be quick to build because the hard parts are solved once: model
access, tool and data connectors, memory, permissions, evaluation,
observability and human approval.
Key responsibilities
• Define the reference
architecture for agents: orchestration loop, tool calling, context management,
memory, sub-agents and human-in-the-loop checkpoints.
• Build the harness as a reusable
internal platform with SDKs, templates and documentation that other engineering
teams can adopt.
• Design the tool and connector
layer, including Model Context Protocol (MCP) servers for internal systems
(CLM, CRM, ticketing, document stores, SAP sandboxes).
• Own the evaluation framework:
test sets, automated graders, regression gates in CI, and online quality
monitoring.
• Set security and governance
controls: identity and permission propagation, sandboxed code execution,
prompt-injection defenses, data classification, audit logging.
• Make model choices and routing
decisions on quality, cost and latency; keep the harness model-agnostic.
• Lead delivery of the first two
or three internal applications end to end, from discovery with business owners
to production.
• Set engineering standards,
review designs and mentor engineers moving into agent development.
• Track cost, usage and value per
agent, and report outcomes to leadership.
Core technical skills
Area
What
the architect must be able to do
Priority
Agent architecture
Design agent loops (plan, act, observe),
multi-agent and sub-agent patterns, workflows vs. autonomous agents, and know
when an agent is the wrong tool
Must have
LLM engineering
Prompt and system-prompt design,
structured outputs, tool/function calling, context-window management,
caching, streaming, model routing across providers
Must have
Tools and integration
Build tool schemas and MCP servers; wrap
REST, OData and SOAP APIs safely; handle auth (OAuth 2.0, service accounts)
and rate limits
Must have
Retrieval and data
RAG design: chunking, embeddings, hybrid
search, reranking, vector stores; document parsing (PDF, Word, scanned
contracts); permission-aware retrieval
Must have
Evaluation
Build eval sets from real tasks,
LLM-as-judge and rule-based graders, regression suites in CI, A/B and online
metrics
Must have
Safety and security
Prompt-injection and data-exfiltration
defenses, least-privilege tools, sandboxed code execution, approval gates for
side-effecting actions, PII handling
Must have
Observability
Tracing of every model and tool call,
cost and latency dashboards, replay of failed runs, feedback capture
Must have
Backend engineering
Production Python and/or TypeScript;
async and event-driven design; queues, workflow engines, idempotency and
retries
Must have
Cloud and platform
Containers, Kubernetes or serverless,
infrastructure as code, secrets management, CI/CD on AWS, Azure or GCP
Must have
Memory and state
Short- and long-term memory design,
durable state for long-running agents, checkpointing and resumption
Strong plus
Agent frameworks
Hands-on with at least one framework
(e.g. LangGraph, Claude Agent SDK, OpenAI Agents SDK, Semantic Kernel) and
the judgment to build thin abstractions instead of locking in
Strong plus
Application layer
Chat, copilot and background-agent UX;
human review queues; embedding agents in Slack, Teams, browser or existing
apps
Strong plus
Cost management
Token budgeting, model tiering, caching,
batch processing, unit economics per agent
Strong plus
Fine-tuning and model ops
When to fine-tune vs. prompt vs.
retrieve; open-weight model hosting
Nice to have
Experience requirements
• 10+ years in software
engineering, with at least 4 years in an architect or principal engineer role
owning platform-level designs.
• 2+ years building LLM-based
systems that reached production use, including at least one agentic system with
tool calling and real side effects (not only chat or RAG demos).
• Has built an internal platform
or framework adopted by other teams, and can show how adoption was earned
(SDKs, docs, templates, support).
• Has designed an evaluation
approach that caught regressions before release, and can explain how its
graders were validated.
• Has run production systems with
on-call responsibility, incident reviews and cost accountability.
• Has worked in an enterprise
environment with SSO, data classification, security reviews and audit
requirements.
• Bachelor's or master's in
computer science or equivalent practical experience.
Nice-to-have and domain experience
• Enterprise integration: SAP
S/4HANA or SAP Ariba APIs, SAP Integration Suite, Jitterbit or another iPaaS.
• Contract lifecycle management,
procurement or legal tech: clause extraction, obligation tracking, document
comparison.
• Document AI: OCR, layout-aware
parsing, table extraction from contracts and PDFs.
• Workflow engines (Temporal,
Step Functions or similar) for long-running agents.
• Experience with regulated
customers (financial services, life sciences) and their AI governance
requirements, e.g. ISO 42001, SOC 2, GDPR.
• Open-source contributions or
published writing on agent design or evaluation.
Leadership and working style
• Builder first. Writes production code, not only diagrams; prototypes quickly, then
hardens what works.
• Evidence over hype. Decides with evals and measurements, and is comfortable saying a
use case does not need an agent.
• Platform mindset. Designs for the tenth agent, not just the first; treats internal
teams as customers.
• Clear communicator. Explains trade-offs to engineers, security, legal and executives in
their own terms.
• Security-minded. Assumes agents will be attacked through their inputs and designs
accordingly.
• Mentor. Raises the team's skill in a field where most engineers are still
learning.