About this role
**The role**
Own the models behind Coval’s judges. Coval is the evaluation platform for conversational AI: we help enterprise teams simulate, monitor and improve voice and chat agents before those agents ever reach a customer. Because our product is the judge, this role sits at the center of the company.
You’ll own the model side end to end. You’ll design and ship the LLM-as-a-judge evaluations that score voice and chat conversations at scale, then sharpen them with LoRA fine-tunes on open-weight models and non-tuning work like tool use and context engineering. Your improvements ship directly into the product enterprises rely on to launch dependable voice agents.
**What you’ll do**
* Design and ship the LLM judges that score conversations: rubrics, judge prompting, and calibration against human raters
* Build the reliability tracking that keeps judges honest from release to release
* Run post-training where training beats prompting: LoRA adapters on open-weight models, trained on data distilled from our own scored conversations
* Everywhere else, do the non-tuning capability work: tool use, structured outputs, and context engineering
* Keep asking which lever actually moves the metric, and focus there
**What we’re looking for**
* 4+ years as an engineer, with strong systems instincts
* A track record of shipping measurable quality gains by pairing rigorous measurement with targeted training, not vibes and prompt tweaks
* You’ve built eval infrastructure you actually trust
* You’ve run LoRA/SFT experiments end to end, including preparing your own data rather than relying on clean data from other teams, so you know which lever really wins
* High agency: on a small Series A team, nobody hands you tickets. You figure out what matters next and drive it there
* Ambitious and kind. You’ll be part of the group that founds our engineering org, sets its bar and grows the team