Senior Backend Engineer
Most applications go out cold — see where you stand first. No sign-up to start.
Don't just apply. Show up ready.
Olive works from this exact posting — no sign-up to start.
At a glance
Requirements
Credentials this posting asks for.
Job overview
LiteLLM is hiring a Senior Backend Engineer. LiteLLM, a leading AI Gateway, seeks a Senior Backend Engineer to enhance its platform's guardrails and observability. This role involves owning the logging and error handling for guardrail calls, ensuring system transparency, and scaling observability integrations for high-volume traffic. The engineer will contribute to the security and compliance features trusted by top companies.
Key focus areas include Build and scale product, ensuring performance, reliability, and continuous improvement, Ensure guardrail and policy enforcement calls are properly logged and traceable, and Build and design CPU-level guardrails to cover common attacks on LLM APIs.
Successful candidates bring Bachelor's Or Master's In Computer Science Or Related Field and 1+ Years Of Experience With Python And Backend Frameworks. Important skills include Python, FastAPI, Flask, Logging Best Practices, Error Handling, and Secure Backend Development. Preferred (not required): Performance, Reliability, and Security.
Skills & qualifications
Skills
Qualifications
Benefits
Full job description
Senior Backend Engineer
LiteLLM is the world's most popular AI Gateway, trusted by top companies like Adobe, Netflix, and NASA. Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services, and we're looking for a Senior Backend Engineer to help us build rock-solid guardrails and observability tooling at scale.
About The Role You’ll focus on owning our guardrails and logging world-class. You will be in charge of the backend code that ensures all guardrail calls are consistently logged, errors are surfaced to users (not silently swallowed), and our observability instrumentation works for real-world, high-volume traffic. Your attention to detail in areas like latency metrics, logging traceability, and backend guardrail registration will directly impact user trust in our security and compliance features.
Responsibilities
-
Build and scale our product, ensuring performance, reliability, and continuous improvement.
-
Ensure all guardrail and policy enforcement calls (e.g., applyguardrail) are properly logged and traceable through our SpendLogs and relevant database tables
-
Build and design CPU-level guardrails to cover common attacks on LLM API's / MCP servers / Agents
-
Identify and fix areas where silent failures occur in guardrail creation, registration, and policy application—ensuring robust error handling and transparency to end users
-
Work with observability integrations, including Datadog, Splunk, Prometheus, and OpenTelemetry, to maintain accurate, configurable, and usable monitoring and logging for backend systems
-
Enhance observability integrations to work for 1B+ requests/mo., with minimal latency overhead and no memory leaks (e.g. due to cardinality of Prometheus metrics)
-
Collaborate cross-functionally on backend engineering priorities (performance, reliability, security)
What We’re Looking For
-
Bachelor’s or Master’s in Computer Science or related field
-
1+ years of experience with Python and backend frameworks (e.g. FastAPI, Flask)
-
Understanding of logging best practices, error handling, and secure backend development
-
Exposure to monitoring, logging, or metrics platforms (Datadog, Splunk, Prometheus, OpenTelemetry)
-
Familiarity with database integration and troubleshooting (PostgreSQL, Redis, etc.)
-
Driven to deliver high-quality backend code with strong guardrails, auditing, and debugging capabilities
-
Eagerness to tackle hard bugs and ensure system transparency for end users
Why Join LiteLLM?
-
High-impact, mission-critical work on the core of compliance and reliability
-
Contribute directly to features used by enterprise customers at global scale
-
Fast-paced growth environment with room for technical ownership
-
Competitive salary, health, dental, and vision benefits
About LiteLLM LiteLLM (https://github.com/BerriAI/litellm) is a Python SDK and Proxy Server enabling seamless calls to 100+ LLM APIs in the OpenAI format, trusted by industry leaders worldwide.
Ready to shape the future of secure, observable AI infrastructure? Apply now!
You've read the whole posting — now see how you match it.