Sesame logo

ML Model Serving Engineer

Sesame

San Francisco, CAFull-time$175–280K/yrPosted 1y agoStill listed 2 days ago

Evaluate my fit

Most applications go out cold — see where you stand first. No sign-up to start.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
$175–280K/yr
Location
San Francisco, CA
Schedule
Full-time
Work Authorization
Not specified

Olive lists jobs from US employers, including remote roles you can work from the United States.

Job overview

Sesame is hiring a ML Model Serving Engineer. Sesame is seeking an ML Model Serving Engineer to enhance its serving layer for LLM, speech, and vision models. The role involves collaborating with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer for a new consumer product. The engineer will modify and extend LLM serving frameworks and optimize models for high throughput and low latency.

Key focus areas include Turbocharge serving layer for LLM, speech, and vision models, Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer, and Modify and extend LLM serving frameworks like VLLM and SGLang.

Successful candidates bring Significant systems programming experience, Significant performance engineering experience, and Up to date on model serving techniques. Important skills include PyTorch, Optimizing Machine Learning Models, High Throughput, Low Latency, Systems Programming, and High-Performance Server Systems. Preferred (not required): High-Performance LLM Serving, VLLM Deployment, SGLang Deployment, and GCP.

Skills & qualifications

RequiredNice to have

Skills

PyTorchOptimizing Machine Learning ModelsHigh ThroughputLow LatencySystems ProgrammingHigh-Performance Server SystemsVLLM InternalsPerformance EngineeringBottleneck AnalysisProfiling Low-Level Systems CodeModel Serving OptimizationHigh-Performance LLM ServingVLLM DeploymentSGLang DeploymentGCPAWSAzureKubernetesRayShippingLeading Complex Multi-Month ProjectsLearning New ThingsWorking in a Multitude of Roles

Qualifications

Expert in Differentiable Array Computing Framework

Benefits

401(k) Match
Medical Insurance
Vision Insurance
Dental Insurance
Paid Time Off

Full job description

About Sesame

Sesame believes in a future where computers are lifelike - with the ability to see, hear, and collaborate with us in ways that feel natural and human. With this vision, we're designing a new kind of computer, focused on making voice agents part of our daily lives. Our team brings together founders from Oculus and Ubiquity6, alongside proven leaders from Meta, Google, and Apple, with deep expertise spanning hardware and software. Join us in shaping a future where computers truly come alive.

RESPONSIBILITIES:

  • Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.

  • Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer to power a new consumer product category.

  • Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest techniques in high-performance model serving.

  • Work with the training team to identify opportunities to produce faster models without sacrificing quality.

  • Use techniques like in-flight batching, caching, and custom kernels to speed up inference.

  • Find ways to reduce model initialization times without sacrificing quality.

Required Qualifications:

  • Expert in some differentiable array computing framework, preferably PyTorch.

  • Expert in optimizing machine learning models for serving reliably at high throughput, with low latency.

  • Significant systems programming experience; ex. Experience working on high-performance server systems—you’d be just as comfortable with the internals of VLLM as you would with a complex PyTorch codebase.

  • Significant performance engineering experience; ex. Bottleneck analysis in high-scale server systems or profiling low-level systems code.

  • Always up to date on the latest techniques for model serving optimization.

Preferred Qualifications:

  • Familiarity with high-performance LLM serving; ex. experience with VLLM, SGlang deployment, and internals.

  • Experience with a public cloud platform such as GCP, AWS, or Azure.

  • Experience deploying and scaling inference workloads in the cloud using Kubernetes, Ray, etc.

  • You like to ship and have a track record of leading complex multi-month projects without assistance.

  • You’re excited to learn new things and work in a multitude of roles.

Sesame is committed to a workplace where everyone feels valued, respected, and empowered. We welcome all qualified applicants, embracing diversity in race, gender, identity, orientation, ability, and more. We provide reasonable accommodations for applicants with disabilities. Contact [email protected] for assistance.

Full-time Employee Benefits: 

  • 401 (k) max employer match: 3.5% of compensation

  • 100% employer-paid health, vision, and dental benefits for you and your dependents

  • Unlimited PTO and sick time

  • Flexible spending account with employer matching up to $1,650/year (medical FSA)

  • Guardian Employee Assistance Program (EAP)

  • Opportunity to share in the company's success with competitive stock options

Benefits do not apply to contingent/contract workers.

Similar jobs, posted recently

Open roles like this one, listed in the last 30 days.

You've read the whole posting — now see how you match it.

Evaluate my fit