Cloud Inference Engineer
San Francisco, CAFull-time$150–250K/yrPosted 2 days agoStill listed today
Most applications go out cold — see where you stand first. No sign-up to start.
You apply on the employer's site, with no Olive account needed.
Watch jobs like this. New roles like this one near San Francisco, CA, by email.
Don't just apply. Show up ready.
Olive works from this exact posting.
At a glance
Olive lists jobs from US employers, including remote roles you can work from the United States.
Job overview
Luminal builds an AI compiler and serving stack for making models faster and production ready. The founding Cloud Inference Engineer will work on Luminal Cloud, shipping low-latency, high-throughput model serving. The role is on-site in downtown San Francisco and focuses on model deployment and tuning, performance reviews, scheduler and autoscaling improvements, and profiling.
Skills & qualifications
Skills
Full job description
Qualifications
-
CUDA + GPU inference optimization
-
vLLM, SGLang, or TensorRT-LLM experience
-
KV caching, paged attention, batching, token streaming, etc.
-
Distributed compute (with GPUs is a super plus)
-
No degree required
Company Luminal (YC S25) builds an AI compiler and serving stack that makes models 10x faster and production ready with one line. Role Founding, on site in downtown SF. Ship low latency, high throughput model serving on Luminal Cloud. Day to day responsibilities:
-
Deploy and tune models with optimizations like KV caching, paged attention, sequence packing, etc.
-
Conducting model performance reviews
-
Improve scheduler, batcher, autoscaling; profile latency, cost, utilization
-
Sometimes write kernels and, yes, occasional tasteful shitposting
Location and compensation San Francisco, CA, US · Full-time · On-site. Annual base salary: $150,000–$250,000 USD. Equity: 0.15%–0.75%. Visa: US citizen/visa only.
How this job compares
Set against the 41,413 open Software Engineer jobs on Olive Jobs, counted November 9, 2026.
- Pay
- This posting$150–250K/yr
- California median$165–250K/yr7,725 salary ranges
- US median$135–219K/yr32,968 salary ranges
This range's midpoint, $200K, is 3% below the California median midpoint ($205K) — in the middle half of Software Engineer salary ranges in California.
- Openings
- Olive Jobs lists 9,630 open Software Engineer jobs in California and 41,413 across the US. 8% of them nationwide are listed as remote or flexible.
- Experience
- Software Engineer postings that state a minimum ask for 5 years of experience at the median; the middle half ask for 3 to 7 years (33,918 postings).
- Education
- 12,939 of the 41,413 open Software Engineer jobs (31%) ask for a bachelor's degree or higher.
Counted across the open, public US postings on Olive Jobs on November 9, 2026. Pay medians are the middle values of the low ends and of the high ends of postings that list both ends of a range in the same cadence (a yearly salary or an hourly rate); a range's midpoint is halfway between its ends. Pay and experience figures resting on fewer than 20 postings are left out.
Prepare for this role
You've read the whole posting — now see how you match it.
