Inference Engineer
San Francisco, CA · HybridFull-timePosted 1w agoStill listed · checked in the last 2 days
See where you stand before you apply. No sign-up to start.
You apply on the posting's application page, with no Olive account needed.
Watch jobs like this. New roles like this one near San Francisco, CA, by email.
Don't just apply. Show up ready.
Olive works from this exact posting.
At a glance
Olive focuses on jobs in the United States, including remote roles you can work from there.
Job overview
The Inference Engineer will own the cost and performance of Adaption’s inference stack, improving how efficiently models are served as workloads, traffic, and hardware change. The role focuses on throughput, latency, and reliability through inference optimization, production workload tuning, provider routing, serving engines, and profiling systems. The engineer will work closely with the team operating the serving fleet.
Skills & qualifications
Skills
Qualifications
Benefits
Full job description
About Us Most AI is frozen in place - it doesn’t adapt to the world. We think that’s backwards. Our mandate is to build efficient intelligence that evolves in real-time. We bring together exceptional people with the drive and creativity to push the boundaries of continual adaptation. We’re looking for builders and creative thinkers ready to shape the next era of intelligence. Our team is small, talent-dense, and moving quickly, with a culture rooted in accountability, initiative and shared success. We look for people who care deeply, ship fast, and are eager to make a dent in the world. We operate with low egos, high velocity, and extreme ownership to build technology that expands access and benefits the many, not the few.
How We Work
-
Impact over activity: We are building a generational company. We value people who execute on ideas rather than comment from the sidelines. We believe our competitive edge lies in caring about excellence across the entire lifecycle - from research to product. No task is beneath anyone: no one is too senior to test the build, clean the data, or unblock a teammate.
-
Vulnerable, low-ego collaboration: We believe exceptional work comes from people who challenge, support and push one another to think bigger. Rapid iteration requires stepping into discomfort. We encourage bold, half-baked, and unorthodox ideas. When we disagree, we share feedback directly, never through side channels.
-
Celebrate decisively: Extraordinary effort should be deeply memorable. We celebrate milestones generously and publicly - via spot bonuses, team dinners, spontaneous gifts, and company-wide recognition - to fuel each other for the next sprint.
The Role Our vision is AI systems that are flexible, adaptable, and accessible to everyone. We believe efficiency is what makes this possible. It’s how we expand access and ensure innovation benefits the many, not the few. You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change. You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization. Success means improving throughput and latency without compromising reliability or model quality.
What You'll Do
-
Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
-
Optimize long-context prefill and decode workloads based on real production traffic.
-
Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
-
Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
-
Build profiling and measurement systems that show where time, memory, and compute are being spent.
Skills You'll Bring We look beyond traditional markers of experience. What matters most to us is what you’ve built and the impact you’ve had. We value demonstrable output, curiosity and alignment with our core values:
-
5+ years in ML systems, inference infrastructure, or performance engineering, with measurable improvements in cost or latency.
-
Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
-
Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
-
Strong Python skills and proficiency in C++, Rust, or another systems language.
-
Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.
Above all, we're looking for great teammates who make work feel lighter and aren't afraid to go out on a limb with bold ideas. You don't need to be perfect, but you do need to be adaptable. We encourage you to apply, even if you don't check every box.
What We Offer
-
Innovative culture: You’ll be part of a generational opportunity to define the trajectory of AI, surrounded by a team pushing the boundaries of what’s possible.
-
Growth paths: A dynamic environment with opportunities to take on new challenges, expand your responsibilities, and make an impact beyond your immediate role.
-
Adaption Passport: An annual travel allowance to explore somewhere new, experience different cultures and keep expanding your horizons. It can be used to travel to any country you haven’t previously been to.
-
Lunch on us: A weekly allowance for takeout or grocery delivery, wherever you are in the world. Whether it's stocking your fridge or having dinner delivered, it’s one less thing to think about.
-
Comprehensive benefits: High-quality medical coverage and benefits designed to support your health and well-being.
-
Team offsites and visits to SF HQ: We're a global team, but believe there's real value in spending time together in person. We regularly bring teammates together at our SF HQ and for team offsites in locations around the world, including London, Mexico City, and New York.
How this job compares
Set against the 6,238 open Machine Learning Engineer jobs on Olive Jobs, counted November 20, 2026.
- Pay
- This postingNot listed
- California median$175–262K/yr1,391 salary ranges
- California median$50–70/hr27 hourly ranges
- Openings
- Olive Jobs lists 1,819 open Machine Learning Engineer jobs in California and 6,238 across the US. 9% of them nationwide are listed as remote or flexible.
- Experience
- This posting asks for at least 5 years of experience; Machine Learning Engineer postings that state a minimum ask for 5 years at the median (5,105 postings).
- Education
- 2,122 of the 6,238 open Machine Learning Engineer jobs (34%) ask for a bachelor's degree or higher.
Counted across the open, public US postings on Olive Jobs on November 20, 2026. Figures based on too few postings are left out.
Prepare for this role
You've read the whole posting — now see how you match it.

