Rhoda AI logo

Research Member of Technical Staff - Training Platform

Rhoda AI

Mountain View, CAFull-timeNo compensation foundPosted 3mo agoVerified open 4 days ago

Most applications go out cold — see where you stand first. No sign-up to start.

At a glance

Compensation
No compensation found
Location
Mountain View, CA
Schedule
Full-time
Work Authorization
Not specified

Job overview

Rhoda AI is hiring a Research Member of Technical Staff - Training Platform. Rhoda AI is seeking a Research Engineer to develop and maintain a training platform for generalist intelligent robots. This role involves building orchestration systems for large-scale distributed model training, developing experiment management tools, and optimizing the research iteration loop. The engineer will also manage job scheduling and cluster utilization, and collaborate with various teams to support platform needs.

Key focus areas include Build and maintain training orchestration systems for large-scale distributed model training across GPU clusters, Develop experiment management tooling: job configuration, tracking, reproducibility, and artifact management, and Build observability infrastructure for training runs: loss curves, compute utilization, gradient statistics, and anomaly detection.

Important skills include Software Engineering, MLOps, ML Platform Engineering, PyTorch DDP, FSDP, and DeepSpeed. Preferred (not required): Slurm, Kubernetes, Ray, and Weights & Biases.

Skills & qualifications

RequiredNice to have

Skills

Software EngineeringMLOpsML Platform EngineeringPyTorch DDPFSDPDeepSpeedMegatronExperiment Tracking SystemsReproducibility SystemsArtifact Management SystemsSlurmKubernetesReliability EngineeringMonitoringAlertingFailure RecoveryRayWeights & BiasesMLflowLarge Model Training PipelinesLLMsVLMsVideo ModelsParallelism StrategiesAWSGCPAzure

Full job description

At Rhoda AI, we’re building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots. Our robots are designed to be generalists capable of operating in complex, real-world environments and handling long-tail edge cases, made possible by our cutting edge research and end-to-end system design. We've raised over $450M and are investing aggressively in model research, infrastructure, hardware development, and manufacturing scale-up to make generalist robotics a reality.

We're looking for a Research Engineer to build and maintain the training platform that powers our model development — experiment orchestration, job management, observability, and the tooling that lets researchers move from idea to result as fast as possible.

What You'll Do

  • Build and maintain training orchestration systems for large-scale distributed model training across GPU clusters

  • Develop experiment management tooling: job configuration, tracking, reproducibility, and artifact management

  • Build observability infrastructure for training runs: loss curves, compute utilization, gradient statistics, and anomaly detection

  • Optimize and automate the research iteration loop from experiment launch to results analysis

  • Manage job scheduling and cluster utilization for efficient use of GPU compute

  • Build internal tooling and interfaces that help researchers move faster

  • Collaborate with training systems, data infrastructure, and research teams to support their platform needs

What We're Looking For

  • Strong software engineering skills with experience in MLOps or ML platform engineering

  • Familiarity with distributed training frameworks (PyTorch DDP, FSDP, DeepSpeed, Megatron, or similar)

  • Experience building experiment tracking, reproducibility, and artifact management systems

  • Comfortable managing and operating GPU cluster environments (Slurm, Kubernetes, or similar)

  • Strong reliability engineering instincts: monitoring, alerting, and failure recovery

Nice to Have (But Not Required)

  • Experience with training orchestration tools (Slurm, Ray, Kubernetes, or similar schedulers)

  • Familiarity with experiment tracking tools (Weights & Biases, MLflow, or custom solutions)

  • Experience supporting large model training pipelines (LLMs, VLMs, or video models)

  • Understanding of parallelism strategies and how they affect training efficiency and debugging

  • Experience with cloud-based training infrastructure (AWS, GCP, or Azure)

Why This Role

  • Your platform is the daily tool every researcher and engineer uses to train models

  • Improvements to training velocity and reliability compound across every experiment the team runs

  • High visibility with direct feedback from researchers and ML engineers

  • Build systems that scale from today's models to future frontier training runs

You've read the whole posting — now see how you match it.