Skip to main content
Keebler Health logo

Senior/Staff Platform Engineer, Data & AI Infrastructure

Keebler Health

Durham, NCFull-time$190–260K/yrPosted 1 day agoStill listed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
$190–260K/yr
Location
Durham, NC
Schedule
Full-time
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

Keebler Health seeks a hands-on senior engineer to scale its data and AI infrastructure as clinical data volume grows. The role owns storage, retrieval, model throughput, and underlying compute and networking in a HIPAA-regulated AWS environment. The engineer will work with a small, senior team on retrieval, model gateways, data architecture, network and cluster foundations, and self-hosted inference.

Skills & qualifications

RequiredNice to have

Skills

Amazon Web Services, nice to haveKubernetes, nice to havePostgreSQL, nice to haveData Architecture, nice to haveCapacity Planning, nice to haveReliability, nice to havePython (Programming Language), nice to haveTypeScript, nice to haveSearch Systems, nice to haveVector Databases, nice to haveANN Indexing, nice to haveToken Buckets, nice to havePriority Queues, nice to haveIdempotent Retries, nice to haveGraceful Degradation, nice to haveQuery Planning, nice to havePartitioning, nice to haveBulk Ingestion, nice to haveConnection Management, nice to haveReplica Strategy, nice to haveCDC, nice to haveAWS Networking, nice to haveVPC Design, nice to haveSubnet Planning, nice to haveKubernetes Networking, nice to haveCost Modeling, nice to haveObservability, nice to haveMetrics, nice to haveTracing, nice to haveProfiling, nice to haveProduction Model Inference, nice to haveGPU Model Serving, nice to haveContinuous Batching, nice to haveKV Cache Management, nice to haveQuantization, nice to haveTensor Parallelism, nice to haveGPU Instance Selection, nice to haveGPU Workloads on Kubernetes, nice to haveInfrastructure as Code, nice to haveRegulated Environments, nice to haveLLM Infrastructure, nice to haveEmbedding Pipelines, nice to haveMulti-Provider Routing, nice to haveBatch Inference APIs, nice to have

Benefits

Tuition Assistance

Full job description

About The Role

Keebler Health builds AI-driven risk adjustment and hospitalization-prevention products for healthcare organizations. Over the next few quarters we’re tripling the volume of clinical data we manage, and the systems that got us here won’t get us there. We’re looking for a hands-on senior engineer to own the scaling of our data and AI infrastructure: storage, retrieval, model throughput, and the compute and network underneath it.

This isn’t a greenfield role. You’ll work on real systems under real load in a HIPAA-regulated AWS environment, alongside a small, senior team.

This is a US-based remote position. We have a strong preference for candidates based in North Carolina or New York, where our team hubs are.

What You’ll Build

We’re tripling the clinical data we manage over the next few quarters. You’ll work on the data and AI platform that makes that growth routine rather than heroic, working in a HIPAA-regulated AWS environment alongside a small, senior team.

  • A retrieval layer that scales with the data. Our embedding store is 8TB today and heading toward ~25TB. You’ll decide how semantic search is built for that scale: index strategy, quantization, partitioning, tiering, or a different storage engine entirely. You’ll set the latency and recall targets and the cost model behind them.
  • A model gateway for every LLM workload. Today, high-volume workloads can run into provider rate limits. You’ll design the shared layer that all model traffic goes through: queueing and backpressure, batching, caching, prioritization, and routing across providers and models. The goal is that product teams never think about quotas.
  • Network and cluster foundations that scale with us. You’ll evolve the AWS networking and compute foundations behind our data and AI workloads, partnering with our DevOps engineer to anticipate capacity limits and address bottlenecks before they affect delivery. That could mean improving network addressing, cluster capacity, or resource utilization.
  • A data architecture that separates workloads by design. Bulk ingestion, transactional traffic, and analytics currently share Aurora. You’ll define how each should be served, whether through staging pipelines, replicas, CDC into a warehouse or lakehouse, or something else, so that ingestion volume can grow without touching application performance.
  • A self-hosted inference platform. We already serve OCR models on vLLM and are expanding into open-weight LLMs. You’ll build the platform around them: benchmarking and model selection, GPU capacity planning, autoscaling, throughput tuning, and a clear framework for which workloads belong on self-hosted models versus commercial APIs.

Experience That’s Relevant

You don’t need to have solved these exact problems before. We care about adjacent experience and sound judgment about tradeoffs. Strong candidates will have several of the following:

  • Large-scale search or retrieval systems. This could be vector databases (pgvector, OpenSearch k-NN, Pinecone, Turbopuffer, Milvus), but classic search infrastructure (Elasticsearch/Lucene), recommender candidate retrieval, or any work tuning latency on large indexes is directly applicable. You should understand the tradeoffs between ANN index types, memory footprint, recall, and cost.
  • High-throughput systems built on quota-limited dependencies. Work in payments, ad-tech, data integrations, or anywhere you’ve had to push large volumes through third-party APIs with hard limits. You know how to use token buckets, priority queues, idempotent retries, and graceful degradation.
  • Postgres at scale. Deep experience with query planning, partitioning, bulk ingestion (COPY, staging tables, batching), connection management, and replica strategy. Aurora-specific experience is a plus. Experience separating OLTP from analytical workloads (CDC into a warehouse or lakehouse, for example) is especially valuable.
  • AWS networking and Kubernetes. You’re comfortable with VPC design, subnet planning, and EKS networking (VPC CNI behavior, prefix delegation, secondary CIDRs, IPv6), or you have the equivalent depth on another major cloud.
  • Capacity planning and cost modeling. You can project how a system behaves at 3x, identify which components scale nonlinearly, and make a cost-aware case for when to optimize and when to re-architect.
  • Observability. You instrument before you optimize, and you’re comfortable with metrics, tracing, and profiling across application and database layers.
  • Production model inference. Experience serving models on GPUs in production, whether that’s LLMs on vLLM, SGLang, TGI, or TensorRT-LLM, or earlier-generation serving on Triton or TorchServe. You should understand the levers that matter for throughput and cost, including continuous batching, KV cache memory, quantization (AWQ, GPTQ, FP8), tensor parallelism, and GPU instance selection. Experience running GPU workloads on Kubernetes (node pools, the NVIDIA device plugin, Karpenter or a similar autoscaler) is especially relevant.

Nice to have

  • Experience in a regulated environment (HIPAA, SOC 2, PCI, or similar), including working under BAAs and data-residency constraints
  • Familiarity with LLM application infrastructure: embedding pipelines, multi-provider routing, Bedrock or OpenAI, batch inference APIs
  • Startup experience, where you were the person who both made the architectural decision and shipped it
  • Python and/or TypeScript fluency, plus infrastructure-as-code (Terraform, CDK, Pulumi)

What Success Looks Like In The First Six Months

Semantic search latency is well within product requirements at current volume, with a credible path to 3x. Model throughput is no longer a recurring incident category. Nightly loads complete without database alarms. The team has a capacity plan it trusts.

What we offer

  • Competitive salary, equity options, and benefits package.
  • A fast-paced, innovative environment.
  • Professional growth and development opportunities.
  • A collaborative and supportive team culture.
  • The chance to make a meaningful impact on the healthcare industry.

Compensation Range: $190K - $260K

How this job compares

Set against the 3,445 open Platform Engineer jobs on Olive Jobs, counted November 22, 2026.

Pay
  • This posting$190–260K/yr
  • North Carolina median$105–196K/yr64 salary ranges
  • US median$123–215K/yr2,860 salary ranges

This range's midpoint, $225K, is 50% above the North Carolina median midpoint ($150K) — in the top quarter of Platform Engineer salary ranges in North Carolina.

Openings
Olive Jobs lists 102 open Platform Engineer jobs in North Carolina and 3,445 across the US. 12% of them nationwide are listed as remote or flexible.
Hiring the most in North Carolina
Experience
Platform Engineer postings that state a minimum ask for 5 years of experience at the median; the middle half ask for 4 to 8 years (2,773 postings).
Education
621 of the 3,445 open Platform Engineer jobs (18%) ask for a bachelor's degree or higher.

Counted across the open, public US postings on Olive Jobs on November 22, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.