Test Harness Engineer

We're looking for a Test Harness Engineer to join a client product team and design and build the testing systems that protect the quality of a growing technology platform. This role goes beyond writing test scripts: you will build AI-powered test harnesses where agents help plan, execute, validate, and analyze testing activities.
You'll combine strong software engineering with automation, exploratory testing, and a product-oriented mindset. You'll collaborate closely with engineering and product teams in a fast-paced environment where quality is a shared responsibility and continuous improvement is highly valued. You will make architecture and tooling decisions and take ownership of quality from design to release.

Responsibilities:

- Design and extend AI-powered test harnesses where agents plan, execute, validate, and analyze test scenarios.

- Integrate LLMs (OpenAI, Claude, Gemini, or similar) with tools, APIs, and automation frameworks, with evaluation and guardrails to prevent false positives.

- Build and maintain automated UI, API, and integration test suites that run reliably in CI/CD pipelines, keeping them fast and maintainable.

- Turn Jira acceptance criteria and Figma designs into test plans, and run exploratory, regression, and functional testing across QA and UAT.

- Identify and communicate defects with clear reproduction steps and evidence, and drive triage with developers and product.

- Use AI tools to accelerate test design and failure analysis, critically validating their output.

- Mentor engineers on harness design and automation practices.

Requirements:

- Proven background in software engineering or test automation, including designing QA automation frameworks that teams rely on in production.

- Hands-on experience building AI agents or agent-based workflows, including tool/function calling, context management, and prompt engineering.

- Experience with at least one agent orchestration framework (LangGraph, LangChain, CrewAI, or similar).

- Ability to implement evaluation, validation, and guardrail mechanisms for AI-driven test execution.

- Strong JavaScript/TypeScript skills, with working knowledge of Python or Java.

- Hands-on experience with UI and API test automation tools (e.g., Cypress, Playwright, REST Assured, Postman/Newman, Karate).

- Experience testing REST APIs and asynchronous, event-driven systems (message queues, pub/sub, or similar).

- Experience working with SQL or NoSQL databases for test data setup and validation.

- Experience integrating automated tests into CI/CD pipelines.

- Fluent English.

Nice to have:

- Experience with MCP (Model Context Protocol) or other tool-integration standards for AI agents.

- Experience with performance testing tools such as k6 or Gatling.

- Experience with accessibility testing.

- Experience testing feature-flagged releases and gradual rollouts.

- Experience working with complex SaaS or consumer-facing products.

Soft Skills:

- Senior-level autonomy: you make sound decisions with little oversight and drive solutions end to end.

- Strong analytical skills with a risk-based approach, and clear communication with developers and product.

- Pragmatic about what to automate versus what to test exploratorily.

- Dedication to continuous learning and development, for yourself and for others.