Skip to main content
Innodata Inc. logo

AI Voice Evaluation Specialist

Innodata Inc.

Ridgefield Park, NJFull-time$20–27/hrSeen 1mo agoIn the employer's feed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
$20–27/hr
Location
Ridgefield Park, NJ
Schedule
Full-time
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Requirements

Credentials this posting asks for.

Bachelor's degree, required

Job overview

Innodata is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations. The specialist conducts comparable roleplays with two models, assesses their responses across defined dimensions, identifies error clusters, and selects the stronger performer. The role also involves recording conversations and writing a detailed, evidence-based rationale while applying evaluation guidelines consistently.

Skills & qualifications

RequiredNice to have

Skills

Attention to Detail, nice to haveListening Comprehension, nice to haveWritten Communication, nice to haveFollowing Evaluation Guidelines, nice to haveNatural Speaking, nice to haveRoleplaying, nice to haveFair Comparison, nice to haveCritical Thinking, nice to haveAnalytical Skills, nice to haveReliability, nice to haveConsistency, nice to haveAI Assistants, nice to haveVoice-Based AI, nice to haveConversational Systems, nice to have

Qualifications

Bachelor's Degree, required

Full job description

Experience Required

None

Minimum Education Required

Bachelor Degree

Compensation

$20.00 - $27.00 / Hourly

Hours Per Week

40

Number Of Positions

1

Shift

First Shift (Day)

Job Description

Innodata(Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked.Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale.We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

Scope of the Role:

We are looking for detail-oriented Voice Specialists to evaluate the performance of AI models through real-time, voice-based conversations. In this role, you will interact with two different AI models using the same assigned scenario, carefully compare their responses, and provide structured evaluations based on defined quality criteria.

The goal of the evaluation is to create a fair, consistent comparison between models and identify which model delivers the stronger overall conversational experience.

What You'll Own:

Review an assigned scenario and roleplay a natural conversation with two different AI models.

Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible.

Maintain approximately the same number of conversational turns with each model to support a fair comparison.

Record your voice during each interaction and pay close attention to the quality and behavior of the model's responses.

Evaluate each model across five defined evaluation dimensions.

Identify and categorize relevant error clusters that may occur in the audio or conversation.

Compare the performance of Model A and Model B based on the evaluation criteria and your observations.

Select the model that demonstrated the stronger overall performance according to the defined evaluation framework.

Write a detailed rationale explaining your final preference, referencing specific examples and observations from both conversations.

Apply evaluation guidelines consistently across different scenarios and model interactions

You'll Thrive in This Role If You Have

Bachelors Degree

Strong attention to detail and ability to notice subtle differences in conversational quality.

Excellent listening and comprehension skills.

Strong written communication skills, particularly the ability to explain observations clearly and objectively.

Ability to follow detailed evaluation guidelines consistently.

Comfort speaking naturally and roleplaying different conversational scenarios.

Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation.

Strong critical-thinking and analytical skills.

Reliability and consistency when completing structured evaluation tasks.

Familiarity with AI assistants, voice-based AI, or conversational systems

The expected hourly salary range for this position is $20-$27 p/hour, based on experience, skills, and qualifications.

Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission's guide athttps://consumer.ftc.gov/articles/job-scams.

If you believe you've been targeted by a recruitment scam, please report it to Innodata [email protected] consider reporting it to the FTC atReportFraud.ftc.gov.

Place of Work

On-site

Requisition ID

4412197009

Job Type

Full Time

Prepare for this role

You've read the whole posting — now see how you match it.