Skip to main content
Apple logo

AI Measurement Scientist, AI Evaluation Platform

Apple

Seattle, WAFull-timeSeen todayIn the employer's feed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
No compensation found
Location
Seattle, WA
Schedule
Full-time
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

Apple Services Engineering is building the scientific foundation for evaluating AI systems across Apple. The Measurement Scientist will establish scientifically valid measurement standards and develop methods to validate AI evaluation tools and workflows. The role bridges measurement science and AI evaluation, collaborating with ML researchers, platform engineers, and evaluation practitioners. Work may be published at measurement and ML venues and productionized into Apple’s evaluation SDK.

Skills & qualifications

RequiredNice to have

Skills

Measurement Modeling, requiredConstruct Validation, requiredBenchmark Design, requiredExperimental Design, requiredCausal Inference, requiredPower Analysis, requiredSampling Theory, requiredUncertainty Quantification, requiredMeasurement Instrument Design, requiredPython, nice to haveR, nice to haveStatistical Analysis, requiredPsychometric Modeling, requiredGenerative AI, requiredCommunication, requiredAI/ML Evaluation, nice to haveAutomated Scoring, nice to haveComputational Assessment, nice to haveLLM-as-Judge, nice to haveAutomated Metrics, nice to haveAgentic Systems, nice to haveComputational Social Science, nice to haveComputational Behavioral Science, nice to haveProduction Tool Development, nice to haveOnline Experimentation, nice to haveRare Event Estimation, nice to have

Qualifications

PhD in Listed Fields or Equivalent Experience, requiredTrack Record in Measurement Validation, requiredPublications or Applied Work, requiredPublications at Measurement or ML Venues, nice to have

Full job description

Weekly Hours: 40

Role Number: 200695284-3337

Summary

Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or experience we deliver is the result of us making each other's ideas stronger.

Description

Our team, part of Apple Services Engineering, is building the scientific foundation for how AI systems are evaluated across Apple. Our goal is to create the evidence that teams can act on: what to measure, whether a change helped, why a system failed, and whether it is ready to ship. We are seeking a Measurement Scientist to ensure that our evaluation methods are not just sophisticated, but scientifically valid and trustworthy. In this role, you will apply apply state-of-the-art measurement methods, validity frameworks, experimental design, and statistical rigor to establish measurement standards for AI evaluation, ensuring that when we claim an evaluator measures "helpfulness" or "safety," it actually does. We are looking for individuals across a range of experience levels.

This role uniquely bridges measurement science and cutting-edge AI evaluation. This role uniquely bridges measurement science and cutting-edge AI evaluation. Our team is building AI tools and capabilities to help developers do evaluation better and faster. You will develop methods for validating these tools: making sure that they generate evaluation artifacts and workflows that give developers the right signals to make sound engineering decisions and build great products for Apple customers.

You will work on an interdisciplinary team with with other measurement scientists and ML researchers to solve new problems in AI evaluation. Your work will be both published at top measurement and ML venues and productionized into the evaluation SDK used across Apple. The successful candidate will have deep expertise in psychometrics or other measurement disciplines, or in experimental design and causal inference, with the ability to apply these principles to novel AI evaluation challenges. Our team deliberately combines ML science and measurement science. We especially welcome people who bridge those areas, including computer scientists who specialize in measurement, and measurement scientists who have worked on ML problems. You will also work collaboratively with platform engineers and evaluation practitioners to translate measurement science into practical tools that scale across the organization.

Minimum Qualifications

  • PhD in Computer Science or Machine Learning with a measurement or evaluation focus, Psychometrics, Educational Measurement, Quantitative Psychology, Statistics, Biostatistics, Econometrics, or equivalent research/work experience.

  • Deep expertise in modeling test or benchmark data (IRT, latent variable, or other response models) and construct validation, in benchmark design and validity for AI systems, or in experimental design and causal inference (surrogate outcomes, observational causal methods).

  • Strong statistical foundation including experimental design, power analysis, sampling theory, and uncertainty quantification.

  • Track record of designing and validating measurement instruments or experiments as demonstrated through publications or applied work.

  • Proficiency in Python (preferred) or R for statistical analysis, psychometric modeling, and method implementation.

  • Strong working knowledge of generative AI technology.

  • Excellent communication skills with the ability to explain complex measurement concepts to engineers, ML researchers, and non-technical stakeholders.

Preferred Qualifications

  • Research or experience applying measurement science to AI/ML evaluation, benchmark validity, automated scoring systems, or computational assessment.

  • Knowledge of modern ML evaluation challenges including LLM-as-judge, automated metrics, benchmark design, and agentic systems.

  • Publications at measurement venues or top ML conferences (NeurIPS, ICML, ICLR).

  • Expertise in computational social or behavioral science using generative AI.

  • Experience collaborating with engineers to turn research methods into production tools and scalable infrastructure.

  • Experience with online experimentation or with validating offline metrics against real-world outcomes (surrogate or proxy metrics).

  • Experience estimating rare events or making decisions when one kind of error costs far more than another.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics.

How this job compares

Set against the 2,048 open Research Scientist jobs on Olive Jobs, counted November 18, 2026.

Pay
  • This postingNot listed
  • Washington median$154–226K/yr137 salary ranges
  • US median$33–49/hr50 hourly ranges
Openings
Olive Jobs lists 209 open Research Scientist jobs in Washington and 2,048 across the US. 3% of them nationwide are listed as remote or flexible.
Hiring the most in Washington
Experience
Research Scientist postings that state a minimum ask for 3 years of experience at the median; the middle half ask for 2 to 5 years (1,119 postings).
Education
1,001 of the 2,048 open Research Scientist jobs (49%) ask for a bachelor's degree or higher.

Counted across the open, public US postings on Olive Jobs on November 18, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.