Skip to main content
Liquid AI logo

Member of Technical Staff - Evaluations

Liquid AI

San Francisco, CA · HybridFull-timePosted 1w agoStill listed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
No compensation found
Location
San Francisco, CAHybrid
Schedule
Full-time
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

Liquid AI is hiring a Member of Technical Staff for its Post-Training team to build and maintain internal evaluations for AI models, especially small edge models. The role partners with researchers and other teams to design reliable, reproducible benchmarks, investigate model outputs, and turn feedback into evaluation environments. It also includes releasing open-source evaluations, publishing research, and reviewing failure modes in existing evaluations.

Skills & qualifications

RequiredNice to have

Skills

Evaluation Design, requiredBenchmark Development, requiredContamination Handling, requiredReproducibility, requiredEvaluation Management, requiredSoftware Engineering, requiredInference Debugging, requiredInfrastructure Debugging, requiredModel Training, nice to haveDeep Systems Expertise, nice to haveEdge Model Evaluation, nice to have

Qualifications

Recognizable Evaluation Work, requiredEvaluation Built From Scratch, required

Benefits

Paid Time Off
Medical Insurance
Parental Leave

Full job description

Role: Member of Technical Staff - Evaluations Team: Post-Training Location: San Francisco Location Type: Hybrid Employment Type: Full-time

About Liquid AI Spun out of MIT CSAIL, we build general-purpose AI systems that run efficiently across deployment targets, from data center accelerators to on-device hardware, ensuring low latency, minimal memory usage, privacy, and reliability. We partner with enterprises across consumer electronics, automotive, life sciences, and financial services. We are scaling rapidly and need exceptional people to help us get there.

The Opportunity Public benchmarks saturate and don't generalize to real-world tasks, so post-training risks overfitting to external scores. Every post-training decision at Liquid AI needs internal evals we trust: evals that track real-world behavior and are cheap and reproducible to run. This role sits in the Post-Training team and exists to make sure those decisions rest on exactly that kind of evidence. You will work on the internal eval suite and infrastructure end to end working closely with the post-training researchers whose decisions depend on it and with other teams (audio and beyond) who need evals for their own models. Evaluating small edge models is an open problem, since nearly every public eval targets frontier models, and the evals you build here change the quality and capabilities of models we ship within weeks, not half a year.

What We're Looking For We need someone who:

  • Proactive execution: You pick up use cases and come back with working benchmarks. You parallelize work, make progress visible, and finish without being chased.

  • White-box about evals: You volunteer where your own evals or scores were wrong and what you changed, and you present numbers with their caveats attached.

The Work

  • Work on the internal eval suite and infrastructure: keep it reliable, optimized, and reproducible at scale.

  • Design held-out evals for priority capabilities such as agentic workflows, computer use, and cybersecurity, with documented construction and contamination checks.

  • Build evals targeted at small edge models, where most public evaluations built for frontier models don't apply.

  • Read generated outputs to debug unexpected scores before anyone acts on them.

  • Turn customer and open-source feedback into environments usable for data, training, and evaluation, and partner with other teams to design evals for their specific needs.

  • Release open-source evals, write papers about them, and publish failure-mode reviews of popular evaluations.

Experience & Capabilities What you'll need:

  • Eval design is where we need you to be unusually strong: you can take a high-level description of a capability and turn it into an evaluation that is accurate, general-purpose, separates good models from bad ones, and is hard to game.

  • Evals you have built before that we can recognize or check, and that hold up on coverage, reproducibility, contamination handling, and the signal they were meant to give. At least one built from scratch: a benchmark, an eval paper, or real contributions to lm-eval-harness, lighteval, OLMES, HELM, or Inspect.

  • Fluency with the fundamentals: contamination, saturation, judge bias, and variance.

  • Hands-on experience running and managing evals in a team, whether on an eval team, a training team, or in an engineering/DevOps role.

  • The habit of debugging unexpected scores by reading individual generated answers before believing the number.

  • Software engineering strong enough to own eval infrastructure and debug the inference and infra failures that surface when running evals at scale, without hand-holding.

Helpful, not required:

  • Model training experience.

  • Publications or research background (industry backgrounds are equally welcome).

  • Deep systems expertise.

  • Familiarity with what small edge models do and don't do well.

  • An eval-specific title is not needed: training-team or DevOps profiles who ran a lot of evals are a good fit.

What Success Looks Like (Year One)

  • Internal evals cover every main target capability and drive post-training decisions, so that model decisions in post-training cite an internal eval you built or hardened rather than an external benchmark.

  • The eval suite runs reliably at scale and is co-owned, with other teams coming to you for evals for their own models.

  • At least one open-source eval released with a paper, plus a failure-mode report on a popular evaluation, with Liquid's edge-model evals becoming an external reference.

What We Offer

  • Evals are the job: You work on the whole eval function alongside one engineer, not one benchmark inside a large eval org, with real compute to run evals at scale and a hiring manager with a large public following who wants to open-source and publish the work.

  • Compensation & Equity: Competitive base salary plus equity, so you share in what we build. Final offers reflect skills, experience, and location.

  • Time Off: Unlimited PTO, plus company-wide Refill Days, long weekends when the whole company rests together.

  • Health & Wellbeing: Comprehensive health coverage, with details varying by location.

  • Paid Leave: Paid leave for pregnancy, parenting, medical needs, and caring for family.

How this job compares

Set against the 1,899 open Machine Learning jobs on Olive Jobs, counted November 17, 2026.

Pay
  • This postingNot listed
  • California median$192–288K/yr348 salary ranges
  • California median$54–68/hr31 hourly ranges
Openings
Olive Jobs lists 498 open Machine Learning jobs in California and 1,899 across the US. 8% of them nationwide are listed as remote or flexible.
Hiring the most in California
Experience
Machine Learning postings that state a minimum ask for 5 years of experience at the median; the middle half ask for 3 to 8 years (1,371 postings).
Education
637 of the 1,899 open Machine Learning jobs (34%) ask for a bachelor's degree or higher.

Counted across the open, public US postings on Olive Jobs on November 17, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.