Skip to main content
Perplexity logo

Member of Technical Staff (Index, Search Crawler)

Perplexity

Dublin, CAFull-timePosted 3 days agoStill listed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
No compensation found
Location
Dublin, CA
Schedule
Full-time
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

Perplexity is seeking an engineer to build and operate the indexing and data-processing layer of its Search Crawler. The role focuses primarily on offline and nearline pipelines that ingest, process, and organize crawled content into efficient, queryable structures. The engineer will work across indexing, storage, reliability, observability, and controlled experimentation, partnering with retrieval, ranking, and machine-learning teams to keep data fresh, complete, and accessible.

Skills & qualifications

RequiredNice to have

Skills

Data Processing, nice to haveGo (Programming Language), nice to haveC++ (Programming Language), nice to haveDistributed Computing, nice to haveData Infrastructure, nice to haveRust, requiredHigh-Throughput Data Processing, requiredLarge Datasets, requiredBackend Systems, requiredDatabases, requiredStorage Systems, requiredData Planes, requiredIndexing Pipelines, requiredDistributed Systems, requiredFault Tolerance, requiredConsistency, requiredScalability, requiredProduction Operations, requiredIncident Management, requiredSystems Thinking, requiredEnd-to-End Data Flow Reasoning, required

Qualifications

5+ Years Production Systems Experience, required

Full job description

AI answers are only as trustworthy as the information behind them. Perplexity grounds its answers in fresh, real-world sources, and the index is what makes that possible. We're looking for an engineer to work on the indexing and data processing layer of Perplexity's Search Crawler. You'll build systems that ingest, process, and organize large volumes of crawled content into efficient, queryable structures, working primarily on offline and nearline pipelines that keep data fresh, complete, and accessible to downstream retrieval. Throughput, scalability, and correctness are critical across datasets that change continuously.

Responsibilities

  • Design, implement, and operate large-scale indexing systems and data pipelines at the core of Perplexity's search infrastructure.

  • Develop and optimize indexing strategies that balance performance, freshness, and resource efficiency.

  • Work on storage formats, compaction, and update mechanisms that keep data accessible and current.

  • Keep pipelines reliable and predictable under high-throughput conditions.

  • Build well-tested components with clear responsibilities and interface contracts that stay flexible as the system evolves.

  • Define and implement observability across offline and near line pipelines, including structured logs, metrics, and data quality signals.

  • Monitor throughput, resource usage, and cost, and drive optimisations as business needs change.

  • Partner with retrieval, ranking, and ML teams to make sure indexing outputs meet their requirements.

  • Enable safe experimentation on indexing strategies and processing logic through controlled rollouts and clearly defined quality signals.

Qualifications

  • 5+ years of experience building production backend or data infrastructure systems.

  • Strong programming experience in Go, C++, or Rust.

  • Experience with high-throughput data processing systems and very large datasets.

  • Experience building or operating databases, storage systems, data planes, or indexing pipelines.

  • Strong understanding of distributed systems, fault tolerance, consistency, and scalability.

  • Experience running production systems and handling operational incidents.

  • A systems-thinking mindset and the ability to reason about end-to-end data flows.

How this job compares

Set against the 36,943 open Software Engineer jobs on Olive Jobs, counted November 20, 2026.

Pay
  • This postingNot listed
  • California median$161–243K/yr6,680 salary ranges
  • California median$48–67/hr122 hourly ranges
Openings
Olive Jobs lists 8,260 open Software Engineer jobs in California and 36,943 across the US. 8% of them nationwide are listed as remote or flexible.
Hiring the most in California
Experience
This posting asks for at least 5 years of experience; Software Engineer postings that state a minimum ask for 5 years at the median (30,715 postings).
Education
10,965 of the 36,943 open Software Engineer jobs (30%) ask for a bachelor's degree or higher.

Pay, requirements and demand for Software Engineer jobs

Counted across the open, public US postings on Olive Jobs on November 20, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.