Skip to main content
Thinking Machines Lab logo

Site Reliability Engineer

Thinking Machines Lab

San Francisco, CAFull-time$300–400K/yrPosted todayStill listed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
$300–400K/yr
Location
San Francisco, CA
Schedule
Full-time
Work Authorization
Open to visa holders • Sponsors visas

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

Thinking Machines Lab is hiring a Site Reliability Engineer to drive end-to-end reliability for Tinker, its fine-tuning API. The role partners with platform engineers, research teams, and security teams to improve distributed training systems, observability, incident recovery, multi-tenant isolation, and resource scheduling. Tinker supports researchers and developers customizing open-weight models, and the platform is growing alongside its expanding customer and use-case base.

Skills & qualifications

RequiredNice to have

Skills

Site Reliability Engineering, nice to haveDistributed Computing, nice to haveCloud Infrastructure, nice to haveIncident Response, nice to haveContinuous Monitoring, nice to haveReliability, nice to haveKubernetes, nice to haveAutomation, nice to haveDistributed Systems, requiredSoftware Development, requiredTooling and Automation, requiredPostmortems, requiredReliability Improvement, requiredCommunication, requiredCross-Functional Coordination, requiredProduction Cloud Services, nice to haveDistributed Training Frameworks, nice to haveCheckpoint and Recovery Systems, nice to have

Qualifications

Bachelor's in Computer Science, Engineering, or Similar or Equivalent Experience, requiredExperience in Distributed Systems, Cloud Infrastructure, or Site Reliability Engineering, requiredProduction Incident Response Experience, required

Benefits

Medical Insurance
Dental Insurance
Vision Insurance
Paid Time Off
Parental Leave
Relocation Assistance

Full job description

We're looking for a Site Reliability Engineer (SRE) to drive the reliability of Tinker end-to-end. You'll work alongside the engineers building the platform and research teams to make every layer of the system more robust and resilient.

About Tinker Tinker is our fine-tuning API that empowers researchers and developers to customize frontier AI to their needs — opening access to capabilities that have previously been concentrated in a handful of labs. We manage the infrastructure while allowing Tinkerers full flexibility in training open weights models with their own data, algorithms, and for their own needs. Tinker is rapidly adding new customers, features, and novel use-cases. We’re hiring to grow the platform alongside the Tinker community.

What You’ll Do

  • Define and own end-to-end reliability, from CI/CD flows to production observability and incident response.
  • Develop appropriate Service Level Objectives for distributed training systems, balancing job completion reliability and scheduling latency with development velocity.
  • Design and implement monitoring and observability across the full training path.
  • Drive incident response for Tinker platform issues, ensuring rapid recovery, thorough incident reviews, and systematic improvements that prevent recurrence.
  • Harden multi-tenant isolation and resource scheduling so that LoRA-based workload co-scheduling maximizes utilization without compromising reliability or data separation
  • Collaborate with security teams to address production vulnerabilities

Skills and Qualifications Minimum qualifications

  • Bachelor's degree or equivalent experience in computer science, engineering, or similar.
  • Experience in distributed systems, cloud infrastructure, or site reliability engineering.
  • Proficiency writing software to solve reliability problems, including building tooling and automation.
  • Experience with production incident response, postmortems, and systematic reliability improvement.
  • Strong communication skills and track record of coordination across engineering and research teams.

Preferred qualifications

  • Deep experience operating production cloud services at scale (e.g., public cloud platforms, internal cloud services)
  • Background in distributed training frameworks and how infrastructure failures surface in training behavior.
  • Track record building checkpoint and recovery systems for long-running distributed jobs.
  • Expertise in Kubernetes at scale: deploying, operating, debugging, and tuning clusters handling heterogeneous GPU workloads.

Logistics

  • Location: This role is based in San Francisco, California.
  • Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $300,000- $400,000 USD.
  • Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
  • Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.

How this job compares

Set against the 2,231 open Site Reliability Engineer jobs on Olive Jobs, counted November 17, 2026.

Pay
  • This posting$300–400K/yr
  • California median$165–245K/yr315 salary ranges
  • US median$125–219K/yr1,836 salary ranges

This range's midpoint, $350K, is 71% above the California median midpoint ($205K) — in the top quarter of Site Reliability Engineer salary ranges in California.

Openings
Olive Jobs lists 402 open Site Reliability Engineer jobs in California and 2,231 across the US. 19% of them nationwide are listed as remote or flexible.
Hiring the most in California
Experience
Site Reliability Engineer postings that state a minimum ask for 5 years of experience at the median; the middle half ask for 5 to 8 years (1,831 postings).
Education
454 of the 2,231 open Site Reliability Engineer jobs (20%) ask for a bachelor's degree or higher.

Counted across the open, public US postings on Olive Jobs on November 17, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.