Principal AI Engineer
El Segundo, CAFull-time$240–310K/yrPosted todayStill listed · checked in the last 2 days
See where you stand before you apply. No sign-up to start.
You apply on the posting's application page, with no Olive account needed.
Watch jobs like this. New roles like this one near El Segundo, CA, by email.
Don't just apply. Show up ready.
Olive works from this exact posting.
At a glance
Olive focuses on jobs in the United States, including remote roles you can work from there.
Requirements
Credentials this posting asks for.
Job overview
Gallatin AI is hiring a Principal AI Engineer to anchor its engineering team building AI-driven logistics planning and decision support for military operations. The role owns architecture, scalability, and engineering quality for AI services, while partnering with platform and backend teams. It also involves production coding, technical leadership, mentoring, and decisions about models, infrastructure, and deployment constraints. Deep AI experience is a plus; deep engineering judgment is the requirement.
Skills & qualifications
Skills
Qualifications
Full job description
About the role Gallatin AI builds AI-driven logistics planning and decision support for military operations. Our AI team is growing quickly, and we are hiring a Principal AI Engineer to be a technical anchor for it: someone who sets the engineering bar, owns architecture and scalability, and helps the team design, build, review, and ship its own work with high quality, in close partnership with our platform and backend teams. We are looking for an exceptional engineer with broad range and sound judgment. You have designed systems that other engineers build on, you make the people around you better, and you care as much about how a team builds as what it builds. Deep AI experience is a plus. Deep engineering judgment is the requirement.
What you will do
-
Serve as a quality gate for our AI services: hold the bar on design, testing, and scalability, and review the changes that matter most before they ship.
-
Be accountable for how our AI services integrate with our event-driven backend (NATS-based messaging), data models, and APIs, working with platform and backend engineers to keep those integrations safe, scalable, and consistent.
-
Be a trusted code reviewer for the AI team, including changes into shared backend and data systems
-
Build a review culture that scales: clear architecture guidelines, common patterns to follow and avoid, design review templates, and risk-based review tiers, including AI-assisted review, so that quality holds as PR volume grows.
-
Prevent architectural drift. Make sure engineers working in parallel, with or without AI coding assistants, are building toward the same design instead of undoing each other’s work.
-
Own scalability: set latency, throughput, concurrency, and cost targets, find bottlenecks, and fix them. Ask “what happens at 10x?” before things ship.
-
Write production code on difficult problems, including AI-specific backend features, agent and retrieval services, and data pipelines.
-
Lead design reviews and RFCs and drive them to decisions. Know when infrastructure or data model changes are safe and how to roll them out without breaking existing systems.
-
Make build-versus-buy and model-selection calls, including self-hosted versus cloud tradeoffs under data-handling and deployment constraints.
-
Unblock the AI team’s access to data sources, infrastructure, and tooling so they can move on their own.
-
Mentor engineers, grow future technical leads, and help set the hiring bar.
What you bring
-
10+ years of software engineering, with a record of designing and scaling complex production systems through multiple orders of magnitude of growth.
-
Principal-level backend and distributed systems depth: service design, event-driven and message-based architectures, APIs, data modeling, concurrency, caching, and failure handling.
-
Demonstrated trust as a code and architecture gatekeeper for a team or platform, and a track record of spreading that capability across other engineers.
-
Strong computer science fundamentals, applied with judgment.
-
Deep Python fluency, plus strong TypeScript or comparable full-stack experience.
-
Production experience with, or the demonstrated ability to quickly ramp on, LLM systems such as RAG, agents, tool use, and evaluation.
-
Strong testing and observability instincts, including reliable tests around nondeterministic components.
-
Working fluency with cloud infrastructure, containers, and CI/CD. You can ship what you build.
-
Clear written and verbal communication with engineers, product leaders, and customers.
-
Eligibility for a US security clearance.
Bonus Points
-
Prior principal or staff-level experience on a platform other teams build on.
-
Experience scaling an engineering organization’s practices, such as review, design standards, and onboarding, as headcount tripled.
-
Experience with data platforms, feature stores, or ML pipelines.
-
Experience with optimization, simulation, or logistics systems.
-
Experience in constrained, air-gapped, or regulated deployments.
-
Background in military logistics, planning doctrine, or defense programs.
How this job compares
Set against the 7,568 open AI Engineer jobs on Olive Jobs, counted November 21, 2026.
- Pay
- This posting$240–310K/yr
- California median$151–253K/yr1,171 salary ranges
- US median$126–232K/yr6,651 salary ranges
This range's midpoint, $275K, is 34% above the California median midpoint ($205K) — in the top quarter of AI Engineer salary ranges in California.
- Openings
- Olive Jobs lists 1,395 open AI Engineer jobs in California and 7,568 across the US. 9% of them nationwide are listed as remote or flexible.
- Experience
- This posting asks for at least 10 years of experience — more than three in four AI Engineer postings that state a minimum ask for (median 5 years, across 6,611 postings).
- Education
- 2,867 of the 7,568 open AI Engineer jobs (38%) ask for a bachelor's degree or higher.
Pay, requirements and demand for Software Engineer jobs
Counted across the open, public US postings on Olive Jobs on November 21, 2026. Figures based on too few postings are left out.
Prepare for this role
You've read the whole posting — now see how you match it.

