Member of Technical Staff (Search Crawler Analyst)
Belgrade, Serbia · HybridJobPosted 2 days agoStill listed today
Most applications go out cold — see where you stand first. No sign-up to start.
Watch jobs like this. New roles like this one near Belgrade, Serbia, by email.
Don't just apply. Show up ready.
Olive works from this exact posting.
At a glance
Olive lists jobs from US employers, including remote roles you can work from the United States.
Job overview
The Search Crawler Analyst works at the intersection of data analysis and engineering to improve Perplexity’s crawling and storage system. The role focuses on designing metrics, building data pipelines, diagnosing issues across URL discovery, crawling, parsing, and indexing, and improving search and answer quality. It also involves training small models, building model-training datasets, and validating improvements through experiments.
Skills & qualifications
Skills
Qualifications
Full job description
The internet is vast, containing trillions of URLs. Perplexity’s crawling and storage system is complex and has multiple stages (URL discovery, crawling, parsing, indexing). Each stage offers many opportunities for improvement and room for intricate bugs. You’ll work at the intersection of data analysis and engineering — designing metrics, building data pipelines, and improving the quality of our search and answer systems.
Responsibilities
-
Find and diagnose quality issues in our crawling pipeline
-
Train small models that optimize particular aspects of the pipeline (e.g. parsing quality)
-
Build datasets for model training, including LLM-as-a-judge labeling pipelines
-
Improve page selection algorithms for indexing
-
Design and analyze experiments to validate improvements
Qualifications
-
4+ years of experience as a data analyst, ML engineer, or in a related role
-
Strong coding skills: you should be able to write production-grade code at the level of a mid-level backend engineer
-
Experience designing metrics from scratch
-
Experience training ML models that shipped to production with measurable metric improvements
Nice to have
- Direct experience working on web crawling or indexing pipelines
You've read the whole posting — now see how you match it.