About this role
About the Role
Lead the development of agent intelligence that helps engineers complete complex workflows across desktop engineering software. You will be a hands-on technical lead for a small AI team, partnering with user research and domain experts to deliver reliable, cost-efficient capabilities grounded in real engineering workflows.
What You'll Do
- Own agent architecture for tool calling, state management, error recovery, model routing, and context management.
- Define and improve task success metrics, establishing baselines and tracking workflow completion, cost, and failure modes.
- Build rigorous, reproducible evaluation infrastructure that translates validated user stories into test cases.
- Work with researchers, domain experts, and engineers to map user workflows and prioritize valuable agent capabilities.
- Set per-task token budgets and measure cost per completed workflow.
- Lead a small team through technical direction, design reviews, code reviews, and hands-on implementation.
- Collaborate with product, integrations, and customers to align agent behavior with real-world use.
What We're Looking For
- At least 7 years of software engineering experience, including 2 or more years building LLM agents that take real-world actions.
- Deep knowledge of LLM application architecture, including model selection, context management, retrieval, tool calling, and orchestration.
- Experience creating agent evaluation and benchmarking frameworks for task completion, cost efficiency, and failure analysis.
- Strong Python skills and familiarity with function calling, tool APIs, observability or tracing, and evaluation tools.
- Hands-on technical leadership experience guiding a small engineering team while continuing to write production code.
- Experience shipping AI or LLM tools over proprietary engineering data or desktop engineering software.
- Experience with desktop automation or application APIs, such as COM, and familiarity with enterprise workstation constraints.
- Background in mechanical engineering software, CAD/CAE, PLM, or a related engineering domain.
Compensation & Benefits
Base salary: $160,000 to $250,000 annually. Visa sponsorship is not available.
Location
On-site in San Francisco, California, United States.