Member of Technical Staff, Inference

Inferact

San Francisco, CAHybridFull-time$200–400K/yrPosted 2mo agoVerified open 6 days ago

Evaluate my fit

Most applications go out cold — see where you stand first. No sign-up to start.

Don't just apply. Show up ready.

Olive works from this exact posting — no sign-up to start.

Evaluate your fitSee where you stand — strengths, gaps, and what to fix first.Get started →Tailor your résuméRewritten against this posting, with a diff of every change.Get started →Practice the interviewA voice mock interview built from this job description.Get started →

At a glance

Compensation

$200–400K/yr

Location

San Francisco, CAHybrid

Schedule

Full-time

Work Authorization

Visa required • Visa sponsorship

Requirements

Credentials this posting asks for.

Bachelor's degree

Job overview

Inferact is hiring a Member of Technical Staff, Inference. Inferact seeks an inference runtime engineer to advance LLM and diffusion model serving, optimizing model execution across diverse hardware and architectures, directly impacting global AI inference performance.

Key focus areas include Push the boundaries of LLM and diffusion model serving, Optimize model execution across diverse hardware and architectures, and Impact how the world runs AI inference.

Successful candidates bring Bachelor's Degree In Computer Science, Bachelor's Degree In Engineering, and Bachelor's Degree In Similar Field. Important skills include Transformer Architectures, Python, PyTorch Internals, vLLM, TensorRT-LLM, and SGLang. Preferred (not required): KV-Cache Memory Management, Prefix Caching, Hybrid Model Serving, and RL Frameworks.

Skills & qualifications

RequiredNice to have

Skills

Transformer ArchitecturesPythonPyTorch InternalsvLLMTensorRT-LLMSGLangTGIRead and Implement Model ArchitecturesRead and Implement Inference Techniques From Research PapersContribute Performant CodeContribute Maintainable CodeDebug in Complex ML CodebasesKV-Cache Memory ManagementPrefix CachingHybrid Model ServingRL FrameworksRL Algorithms for LLMsMultimodal InferenceOpen-Source ML ContributionsSystem Infrastructure Projects ContributionsImplemented Core Features in Other Inference Engine ProjectsVerlOpenRLHFUnslothLlamaFactoryTechnical Blogs on LLM InferenceSide Projects on LLM Inference

Qualifications

Bachelor's Degree in Computer ScienceBachelor's Degree in EngineeringBachelor's Degree in Similar FieldEquivalent Experience

Benefits

Medical Insurance

Dental Insurance

Vision Insurance

401(k) Match

Full job description

Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware—a position that took years to build.

About the Role We're looking for an inference runtime engineer to push the boundaries of what's possible in LLM and diffusion model serving. Models grow larger. Architectures shift: mixture-of-experts, multimodal, agentic. Every breakthrough demands innovations on the inference engine itself. You'll work at the core of vLLM, optimizing how models execute across diverse hardware and architectures. Your work will directly impact how the world runs AI inference.

Skills and Qualifications Minimum qualifications:

Bachelor's degree or equivalent experience in computer science, engineering, or similar.
Deep understanding of transformer architectures and their variants.
Strong programming skills in Python with experience in PyTorch internals.
Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI).
Ability to read and implement model architectures and inference techniques from research papers.
Demonstrate the ability to contribute performant and maintainable code and debug in complex ML codebases.

Preferred qualifications:

Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving.
Familiarity with RL frameworks and algorithms for LLMs.
Experience with multimodal inference (audio/image/video/text).
Contributions to open-source ML or system infrastructure projects.

Bonus points if you have:

Implemented core features in vLLM or other inference engine projects.
Contributed to vLLM integrations (verl, OpenRLHF, Unsloth, LlamaFactory, etc).
Written widely-shared technical blogs or side projects on vLLM or LLM inference.

Logistics

Location: This role is based in San Francisco, California. Will consider remote in the US for exceptional candidates.
Compensation: Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity.
Visa sponsorship: We sponsor visas on a case-by-case basis.
Benefits : Inferact offers generous health, dental, and vision benefits as well as 401(k) company match.

You've read the whole posting — now see how you match it.

Evaluate my fit