Judgment Labs logo

Applied AI Engineer

Judgment Labs

San Francisco, CAJob$140–230K/yrPosted 8mo agoStill listed 3 days ago

Most applications go out cold — see where you stand first. No sign-up to start.

Watch jobs like this.

At a glance

Compensation
$140–230K/yr
Location
San Francisco, CA
Work Authorization
Not specified

Olive lists jobs from US employers, including remote roles you can work from the United States.

Job overview

Applied AI Engineers will build AI systems that analyze large-scale agent interaction data, design post‑training workflows, and create infrastructure for agent platforms. They will own end‑to‑end projects, collaborate with product and external partners, and translate frontier research into production to support high‑stakes finance, legal, and operations workflows.

Skills & qualifications

RequiredNice to have

Skills

ResearchAgent SystemsWhiteboardingInfrastructureLegal OperationsArtificial IntelligenceFinanceAI SystemsAgent Interaction DataReinforcement LearningData QualityEvaluationBenchmarkingProduction Data HandlingOrchestrationRuntimesDeveloper ToolsTraining PipelinesModel ServingResearch TranslationCustomer‑Facing Workflow IntegrationAmbiguous Problem Solving

Qualifications

Intellectually CuriousSelf‑DirectedOwnership Mindset

Full job description

The Role: We are looking for Applied AI Engineers to build AI systems that use agent interaction data to understand how agents behave, evaluate them at scale, and improve them through learning and feedback. Your research will not live on a whiteboard. You’ll work directly with real-world agent data, apply frontier methods in production, and see your work ship into the product. By making agent behavior measurable and debuggable, your systems will support teams deploying agents across finance, legal, operations, and other high-stakes workflows. You will own projects end-to-end, with significant autonomy, and work closely with the team to build self-improving agent systems.

What You'll Do:

  • Build AI systems to aggregate, index, and analyze large-scale long-running agent interaction data in order to extract meaningful signals

  • Design and implement post-training and optimization workflows to improve agents, both internally and for customers

  • Build agent platform infrastructure, including orchestration, runtimes, and developer tools that help teams define, test, deploy, and iterate on complex agent workflows

  • Build internal tools and infrastructure that support rapid experimentation, analysis, and training

  • Work closely with product to integrate agents into customer-facing workflows

  • Collaborate with external companies and research partners on frontier AI research

What We're Looking For Every hire clears three bars, no exceptions:

  • Agency. You are intellectually curious, self-directed, and stay up to date with the latest research, blogs, trends, and ideas.

  • Depth of thought. You can reason clearly about abstract systems, and ideally have experience working on agents, RL, or the infrastructure that supports them.

  • Ownership. You own outcomes, not just tasks. You use freedom to experiment responsibly, make business-driven decisions, and focus first on work that moves the company forward.

More specifically, you should bring strength in at least one of the following areas:

  • Data quality, evaluation, benchmarking, and hands-on work with messy production data

  • Agent systems built or evaluated in real-world or production settings

  • Reinforcement learning, post-training, agents, or machine learning fundamentals

  • Infrastructure and systems work across training, data pipelines, evaluation, or model serving

  • Translating research into product while balancing customer constraints, technical tradeoffs, and business impact

  • Turning ambiguous problems into clear, well-designed plans

Similar jobs, posted recently

Open roles like this one, listed in the last 30 days.

You've read the whole posting — now see how you match it.