Skip to main content
Microsoft Corporation logo

Principal Software Engineering, AI Infrastructure

Microsoft Corporation

United StatesJob$143–275K/yrPosted 4w agoStill listed 3 days ago

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
$143–275K/yr
Location
United States
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Requirements

Credentials this posting asks for.

Microsoft Cloud Background Check, required

Job overview

The FIT Infrastructure team builds accelerated compute platforms for large-scale AI training and inference. This principal engineering role focuses on cloud infrastructure, distributed systems, and AI model serving, delivering GPU platform capabilities with Azure and other Microsoft teams. The engineer will shape platform strategy and build infrastructure for internal and external customers.

Skills & qualifications

RequiredNice to have

Skills

C (Programming Language), nice to haveC++ (Programming Language), nice to haveC# (Programming Language), nice to haveJava (Programming Language), nice to haveJavaScript (Programming Language), nice to haveLinux, nice to haveGo (Programming Language), nice to havePython (Programming Language), nice to haveKubernetes, nice to haveDevOps, nice to haveLarge-Scale Infrastructure Design, nice to haveInfrastructure Operations, nice to haveProblem Solving, nice to haveDebugging, nice to haveTechnical Leadership, nice to haveMentoring, nice to haveResource Scheduling, nice to havePreemption, nice to havePriority Handling, nice to haveCommunication, nice to haveOnline Inference, nice to haveMultitenant AI Platforms, nice to haveInference Stacks, nice to haveDistributed Training, nice to haveNCCL Collectives, nice to haveModel Parallelism, nice to haveData Parallelism, nice to haveLarge Language Models, nice to haveGPU Computing, nice to haveCUDA, nice to haveGPU Drivers, nice to haveDevice Plugins, nice to haveRuntime Integration, nice to haveMicroservices Architecture, nice to have

Qualifications

Bachelor's in Computer Science or Related Technical Field or Equivalent Experience, required6+ Years Technical Engineering Experience, requiredMicrosoft Cloud Background Check, requiredMaster's in Computer Science or Related Technical Field, nice to have8+ Years Technical Engineering Experience, nice to haveBachelor's in Computer Science or Related Technical Field, nice to have12+ Years Technical Engineering Experience, nice to have

Full job description

Overview The FIT Infrastructure team builds the foundational accelerated compute platforms that power large scale AI training and inference for Microsoft. Our mission is to deliver secure, reliable, and highly efficient GPU infrastructure that enables multi-tenant AI systems at global scale while maximizing utilization, availability, performance, and reliability.

This role sits at the intersection of cloud infrastructure, distributed systems software, and frontier AI model serving, working closely with Azure Infrastructure, Microsoft Foundry, Microsoft AI and Microsoft 365 teams to deliver end-to-end platform capabilities.

As a a Principal Software Engineering, AI Infrastructure, you will work on mission critical infrastructure that directly powers largescale AI systems. Influence the future of cloud GPU platforms used by internal and external customers and ollaborate with experts across hardware, networking, and AI deployment and serving software. This a an opportunity to grow as a technical leader, shaping long term platform strategy.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more, and we’re dedicated to this mission across every aspect of our company. Our culture is centered on embracing a growth mindset and encouraging teams and leaders to bring their best each day. Join us and help shape the future of the world.

Responsibilities

  • Develop systems for AI Workload management, scheduling, preemption, and failover.
  • Develop systems for AI Workload serving.
  • Design and build GPU accelerated infrastructure for training and inference workloads, spanning bare metal, virtual machines, and containerized environments.
  • Build and operate advanced orchestration and resource governance scenarios using platforms such as AKS and related Kubernetes ecosystem capabilities to enable fair sharing, isolation, and efficient utilization of accelerated resources.
  • Design, Develop, test, and maintain services written in C#, Golang hosted on Kubernetes clusters.
  • Optimize performance, reliability, and utilization across large GPU fleets, including scaleup and scale out configurations.
  • Drive end-to-end platform features from design through production, including observability, diagnostics, and operational excellence.
  • Influence platform architecture and technical direction across teams through design reviews and technical leadership.
  • Embody our Culture and Values

Qualifications Required Qualifications

  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Linux, Go or Python• OR equivalent experience.

Other Requirements

  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.

Preferred Qualifications

  • Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Linux, Go or Python • OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, linux, Go or Python

  • OR equivalent experience.

  • Proven ability to design and operate large scale, production infrastructure with high reliability and performance requirements.

  • Demonstrated problem-solving skills and the ability to debug complex, cross layer systems issues.

  • Demonstrated technical leadership, including mentoring engineers and driving cross team architectural alignment.

  • Hands-on experience with resource scheduling, preemption, priority handling on large clusters.

  • Advanced collaboration and communication skills, with the ability to work across organizational boundaries.

  • Experience with large-scale online Inferencing services.

  • Experience in building or operating multitenant AI platforms in cloud environments.

  • Familiarity with inference and/or distributed training stacks (e.g., NCCL style collectives, model/data parallelism, Large-Language Models).

  • Familiarity with GPU accelerated computing (e.g., CUDA, GPU drivers, device plugins, or runtime integration).

  • Dev-ops experience with microservices architecture in a complex infrastructure and operational environment.

#AIINFRA

Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

How this job compares

Set against the 9,067 open Software Engineering jobs on Olive Jobs, counted November 15, 2026.

Pay
  • This posting$143–275K/yr
  • US median$143–250K/yr7,144 salary ranges

This range's midpoint, $209K, is 3% above the US median midpoint ($202K) — in the middle half of Software Engineering salary ranges across the US.

Openings
Olive Jobs lists 9,067 open Software Engineering jobs across the US. 9% of them nationwide are listed as remote or flexible.
Hiring the most across the US
Experience
This posting asks for at least 6 years of experience; Software Engineering postings that state a minimum ask for 7 years at the median (6,824 postings).
Education
2,061 of the 9,067 open Software Engineering jobs (23%) ask for a bachelor's degree or higher.

Counted across the open, public US postings on Olive Jobs on November 15, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.