
Machine Learning Engineer - LLM Evaluator - Hybrid Opportunity
Cupertino, CA · HybridContract$55–65/hrSeen todayIn the employer's feed · checked in the last 2 days
See where you stand before you apply. No sign-up to start.
You apply on the posting's application page, with no Olive account needed.
Watch jobs like this. New roles like this one near Cupertino, CA, by email.
Don't just apply. Show up ready.
Olive works from this exact posting.
At a glance
Olive focuses on jobs in the United States, including remote roles you can work from there.
Job overview
The Machine Learning Engineer will define, measure, and improve the quality of AI systems powered by large language models. The role focuses on evaluation rather than traditional model development, including scalable evaluation frameworks, automated quality assessment, and metrics for model improvement. The engineer will collaborate with Product, Engineering, Research, and Data teams. The position requires onsite attendance in Cupertino three days per week.
Skills & qualifications
Skills
Qualifications
Benefits
Full job description
Description
-
Hybrid role requiring onsite attendance in Cupertino, CA three (3) days per week.
-
Contract opportunity.
-
This role focuses on AI/LLM quality evaluation and measurement, not traditional model development.
-
Candidates should have experience owning or leading evaluation strategies for production AI/LLM systems.
We are seeking an experienced Machine Learning Engineer to help define, measure, and improve the quality of AI systems powered by Large Language Models (LLMs). In this role, you will design and scale evaluation frameworks that ensure AI-generated outputs are accurate, reliable, and consistent across a variety of use cases.
You will partner closely with Product, Engineering, Research, and Data teams to establish evaluation methodologies, automate quality assessment workflows, and develop the metrics that drive model improvements. This is an ideal opportunity for someone who enjoys solving complex problems at the intersection of AI quality, evaluation strategy, automation, and data-driven decision making.
Top Skills
-
AI/LLM Evaluation & Quality Measurement
-
Problem Solving & System Design
-
Communication & Ownership
-
Python, SQL & Automation
Top Responsibilities
-
Design and build LLM evaluation frameworks, including automated scoring systems, rubric-based grading, and LLM-as-a-Judge solutions.
-
Define and track evaluation metrics, KPIs, and dashboards that measure model quality and performance at scale.
-
Perform error analysis, root-cause investigations, and quality deep dives to identify opportunities for model improvement.
-
Partner with cross-functional teams to integrate evaluation standards, workflows, and quality requirements into production environments.
What We're Looking For
-
Experience designing and maintaining LLM evaluation frameworks and measurement methodologies.
-
Hands-on experience with autograders, LLM-as-a-Judge systems, and automated quality validation.
-
Strong prompt engineering skills, including the development of evaluation prompts and grading rubrics.
-
Proficiency with Python, SQL, and large-scale data analysis.
-
Experience creating KPIs, dashboards, and executive-ready reporting on AI quality.
-
Ability to evaluate model performance through error analysis, human-vs-model calibration, and quality benchmarking.
-
Experience working with production AI/LLM systems and large evaluation datasets.
What Will Make You Stand Out
-
You've personally owned or led an AI/LLM evaluation strategy.
-
You've built scalable systems to measure AI quality automatically.
-
You can clearly explain how AI quality is defined, measured, and improved.
-
You have experience influencing product and engineering decisions through evaluation insights and data.
Experience Level: Expert
Job Type & Location
This is a Contract position based out of Cupertino, CA.
Pay and Benefits
The pay range for this position is $55.00 - $65.00/hr.
Individual compensation offered for this position within this range will depend on many factors, including qualifications, skills, relevant experience, job knowledge, geographic location, internal equity, and other pertinent job-related factors.
Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following: Medical, dental & vision Critical Illness, Accident, and Hospital 401(k) Retirement Plan Pre-tax and Roth post-tax contributions available Life Insurance (Voluntary Life & AD&D for the employee and dependents) Short and long-term disability Health Spending Account (HSA) Transportation benefits Employee Assistance Program Time Off/Leave (PTO, Vacation or Sick Leave)
Job Type & Location
This is a Contract position based out of Cupertino, CA.
Application Deadline
This position is anticipated to close on Nov 20, 2026.
Workplace Type
This is a fully remote position.
About TEKsystems
We're partners in transformation. We help clients activate ideas and solutions to take advantage of a new world of opportunity. We are a team of 80,000 strong, working with over 6,000 clients, including 80% of the Fortune 500, across North America, Europe and Asia. As an industry leader in Full-Stack Technology Services, Talent Services, and real-world application, we work with progressive leaders to drive change. That's the power of true partnership. TEKsystems is an Allegis Group company.
The company is an equal opportunity employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
About TEKsystems and TEKsystems Global Services
Were a leading provider of business and technology services. We accelerate business transformation for our customers. Our expertise in strategy, design, execution and operations unlocks business value through a range of solutions. Were a team of 80,000 strong, working with over 6,000 customers, including 80% of the Fortune 500 across North America, Europe and Asia, who partner with us for our scale, full-stack capabilities and speed. Were strategic thinkers, hands-on collaborators, helping customers capitalize on change and master the momentum of technology. Were building tomorrow by delivering business outcomes and making positive impacts in our global communities. TEKsystems and TEKsystems Global Services are Allegis Group companies. Learn more at TEKsystems.com.
The company is an equal opportunity employer and will consider all applications without regard to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
San Francisco Fair Chance Ordinance: Pursuant to the San Francisco Fair Chance Ordinance, for all positions located in the city and county of San Francisco, we will consider for employment qualified applicants with arrest and conviction records.
Massachusetts Lie Detector: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
Use of Artificial Intelligence (AI): We may use Artificial Intelligence (AI) to support parts of our hiring process, including sourcing, screening, and evaluating candidates. AI helps assess applications and qualifications, but final decisions are made by our hiring team. By applying, you acknowledge and agree that your application may be reviewed using AI tools.
How this job compares
Set against the 6,225 open Machine Learning Engineer jobs on Olive Jobs, counted November 18, 2026.
- Pay
- This posting$55–65/hr
- California median$50–70/hr25 hourly ranges
- US median$30–67/hr97 hourly ranges
This range's midpoint, $60/hr, is close to the California median midpoint ($59/hr) — in the middle half of Machine Learning Engineer hourly ranges in California.
- Openings
- Olive Jobs lists 1,850 open Machine Learning Engineer jobs in California and 6,225 across the US. 9% of them nationwide are listed as remote or flexible.
- Experience
- Machine Learning Engineer postings that state a minimum ask for 5 years of experience at the median; the middle half ask for 3 to 7 years (5,192 postings).
- Education
- 2,133 of the 6,225 open Machine Learning Engineer jobs (34%) ask for a bachelor's degree or higher.
Counted across the open, public US postings on Olive Jobs on November 18, 2026. Figures based on too few postings are left out.
Prepare for this role
You've read the whole posting — now see how you match it.
