ML Data Quality Analyst

Job Description

About the role: We are open to teaching and growing an individual for this role. You’ll own the quality of the training data behind Calder’s ML and robotics work. You’ll audit annotated video, image and multimodal datasets, turn customer requirements into clear QA standards, and build the dashboards and analysis that show where quality breaks down and how to fix it.

Responsibilities

• Review and audit annotated datasets for quality, accuracy and compliance with customer specs
• Turn customer requirements into annotation guidelines, QA rubrics and automated checks
• Investigate data quality issues, find root causes and spot trends across vendors, operators and annotators
• Write SQL and Python to pull, clean and analyze annotation and operational data
• Build and maintain dashboards, trackers and reports on QA performance, throughput and delivery readiness
• Design audit and sampling processes, and give structured feedback to annotation teams
• Train and calibrate annotation teams, including remote and vendor teams, on guidelines and edge cases
• Maintain label taxonomies and metadata standards
• Test new annotation and QA tool features and give clear, prioritized feedback to product and engineering
• Run pre-delivery validation (duplicates, restricted content, metadata completeness, spec caps)
• Track key quality metrics and recommend process improvements

Skills and Requirements

• 1 year in data annotation, data quality, QA or AI data operations
• Understanding of how labeled data is used to train and evaluate ML models
• Working knowledge of SQL and Python for data analysis
• Advanced Excel and Google Sheets skills
• Hands-on experience with annotation, labeling or QA platforms
• Experience writing or applying annotation guidelines and QA rubrics
• Strong attention to detail, analytical problem-solving and clear written communication
• 2-3+ years of relevant experience
• Strong Python and SQL skills (e.g., building pipelines or automated QA checks)
• Experience with BI or dashboard tools (e.g., Superset, Looker, Tableau)
• Experience using LLMs or scripting to automate QA and data workflows
• Experience with video, image or computer vision datasets, ideally egocentric or sensor data
• Exposure to robotics, spatial computing or 3D/SLAM data
• Experience managing or coordinating vendor or offshore annotation teams
• Experience in fast-paced, high-volume data operations

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].