Skip to main content

Senior Software Engineer, Reliability

Temporal

Remote · USFull-time$141–231K/yrPosted 1 day agoStill listed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
$141–231K/yr
Location
Remote · US
Schedule
Full-time
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

The Senior Software Engineer, Reliability will help improve the end-to-end reliability of Temporal Cloud. The role includes building an AI-first Reliability Agent to support SRE triage, strengthening incident response, and making reliability practices accessible across engineering. The engineer will partner with engineering, infrastructure, and product teams to improve service health, operational readiness, and reliability outcomes.

Skills & qualifications

RequiredNice to have

Skills

Distributed Systems, requiredConcurrency, requiredPerformance, requiredFault-Tolerant Design, requiredHigh-Load Systems, requiredAI-Powered Tools, requiredProduction Automation, requiredIncident Response, requiredPost-Incident Reviews, requiredGamedays, requiredChaos Testing, requiredLoad Testing, requiredCapacity Planning, requiredReliability Scorecards, requiredObservability Programs, requiredSound Judgment, requiredCommunication, requiredStakeholder Alignment, requiredMentoring, requiredMulti-Tenant Systems, nice to haveInternal Platforms, nice to haveLLM Observability, nice to haveAgent Evaluation, nice to haveRetrieval-Augmented Generation, nice to haveWorkflow Orchestration, nice to haveOpen Source Contributions, nice to have

Qualifications

Computer Science Fundamentals, requiredComplex Systems Experience, requiredReliability Improvement Track Record, requiredOn-Call Rotation Participation, required

Full job description

Role Summary Join the Reliability team as a Senior Software Engineer, reporting to Tushar Roy, and help improve the reliability of operating Temporal Cloud end to end. You’ll build an AI-first reliability experience, including building a Reliability Agent from scratch that helps the SRE team triage common issues, surface useful context, and recommend next steps. You’ll also help make reliability knowledge and practices accessible across engineering so ownership is shared rather than concentrated with a small number of experts.

What You’ll Do

  • Own reliability outcomes for operating Temporal Cloud end to end, partnering across engineering, infrastructure, and product to drive measurable improvements.

  • Build and evolve the Reliability Agent, including workflows that gather context from telemetry, incidents, runbooks, deployments, and service history to support faster, more consistent triage.

  • Design safe, useful agent experiences for handling routine reliability questions and issues, with appropriate human oversight, escalation paths, auditability, and safeguards against incorrect actions.

  • Strengthen incident response for Temporal Cloud, from detection and triage through escalation, mitigation, and communication, and lead post-incident reviews that turn into durable corrective actions.

  • Define and evolve reliability practices, including service-level objectives, alerting thresholds, operational readiness criteria, and escalation paths.

  • Plan and run gamedays and safely expand chaos testing to validate failure response, evaluate agent-assisted operations, and reduce operational risk.

  • Build and maintain reliability scorecards that make service health, operational readiness, agent effectiveness, and improvement priorities visible.

  • Lead load, performance, and capacity testing, and improve observability standards across services.

What You’ll Bring

  • Strong computer science fundamentals, especially in distributed systems, concurrency, performance, and fault-tolerant design.

  • Experience designing, building, or operating complex systems under high load and partial failure.

  • Experience building AI-powered tools, agents, or production automation, with an understanding of tool use, evaluation, state, failure modes, and human-in-the-loop design.

  • A track record of driving reliability improvements across multiple services, systems, or engineering teams.

  • Hands-on experience with incident response and post-incident reviews, including leading incidents and following corrective actions through to completion.

  • Hands-on experience with one or more of gamedays, chaos testing, load testing, capacity planning, reliability scorecards, or observability programs.

  • Sound judgment in ambiguous situations, with the ability to balance reliability, agent autonomy, safety, customer impact, and delivery speed.

  • Clear communication skills and the ability to align stakeholders, share operational knowledge, mentor others, and participate in an on-call rotation. This is a remote United States role with no regular travel expected.

Nice to Haves

  • Experience operating multi-tenant systems and designing protections against noisy-neighbor behavior.

  • Experience building internal platforms or tooling that enables teams to meet reliability standards.

  • Familiarity with LLM observability, agent evaluation, retrieval-augmented generation, or workflow orchestration systems.

  • Open source contributions in infrastructure, distributed systems, AI, or reliability engineering.

Temporal Technologies is an Equal Opportunity Employer. Temporal Technologies does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need. We embrace and celebrate differences and diversity. Temporal is committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, its services, programs, and activities. If you need to request a reasonable accommodation, please let your Recruiter know so we can assist.

How this job compares

Set against the 9,435 open Software Engineering jobs on Olive Jobs, counted November 18, 2026.

Pay
  • This posting$141–231K/yr
  • US median$146–250K/yr7,463 salary ranges

This range's midpoint, $186K, is 8% below the US median midpoint ($202K) — in the middle half of Software Engineering salary ranges across the US.

Openings
Olive Jobs lists 9,435 open Software Engineering jobs across the US. 11% of them nationwide are listed as remote or flexible.
Hiring the most across the US
Experience
Software Engineering postings that state a minimum ask for 7 years of experience at the median; the middle half ask for 5 to 10 years (7,168 postings).
Education
2,030 of the 9,435 open Software Engineering jobs (22%) ask for a bachelor's degree or higher.

Counted across the open, public US postings on Olive Jobs on November 18, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.