Skip to main content
Perplexity logo

Member of Technical Staff (Web Access Engineer, Search Crawler)

Perplexity

Dublin, CAFull-timePosted 1w agoStill listed · checked in the last 2 days

See where you stand before you apply. No sign-up to start.

You apply on the posting's application page, with no Olive account needed.

Watch jobs like this.

Don't just apply. Show up ready.

Olive works from this exact posting.

At a glance

Compensation
No compensation found
Location
Dublin, CA
Schedule
Full-time
Work Authorization
Not specified

Olive focuses on jobs in the United States, including remote roles you can work from there.

Job overview

Perplexity is hiring a Web Access Engineer for its Crawler team to build reliable fetching and rendering systems for AI-powered search. The role focuses on diagnosing network errors, page-loading issues, and incomplete rendering, then turning findings into production fixes. The engineer works with public web content within site permissions and approved access policies, supporting downstream indexing and retrieval systems.

Skills & qualifications

RequiredNice to have

Skills

Information Infrastructure, nice to haveGo (Programming Language), nice to havePython (Programming Language), nice to haveC++ (Programming Language), nice to haveHttp2, nice to haveDomain Name System, nice to haveJavaScript (Programming Language), nice to haveDevtools, nice to haveRust, requiredHTTP/1.1, requiredHTTP/2, requiredHTTP/3, requiredTLS, requiredDNS, requiredCookies, requiredSessions, requiredCDN Caching, requiredHTTP Error Handling, requiredRetry Backoff, requiredRate Limits, requiredPage Lifecycle, requiredNetwork Idle, requiredService Workers, requiredClient-Side Rendering, requiredShadow DOM, requiredJavaScript Debugging, requiredChrome DevTools, requiredCDP, requiredHAR Files, requiredCurl, requiredPacket Captures, requiredIndependent Work, requiredCDN Development, nice to haveCaching Systems, nice to haveWeb Performance Systems, nice to haveCrawler Operations, nice to haveHeadless Browsers, nice to haveDistributed Content Acquisition, nice to haveBrowser Engine Contributions, nice to haveJavaScript Runtime Contributions, nice to have

Qualifications

5+ Years Software Engineering Experience, requiredWeb Infrastructure, Browsers, CDNs, or Crawling, required

Full job description

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You'll work on freely available public web content, within site permissions and approved access policies, and develop the infrastructure that supplies content to downstream indexing and retrieval systems. Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.

Responsibilities

  • Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.

  • Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side rendering, and classify root causes.

  • Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.

  • Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.

  • Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.

  • Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.

Qualifications

  • 5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.

  • Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3, including status semantics, caching, conditional requests, content encoding, and redirects; strong knowledge of TLS, DNS, cookies, and sessions.

  • Understanding of CDN caching and content delivery, HTTP error handling, retry backoff, and server-defined rate limits.

  • Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM; experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.

  • Comfort working independently when the path forward is not fully defined.

Nice to have

  • Experience developing CDN, caching, or web performance systems.

  • Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.

  • Contributions to a browser engine or JavaScript runtime.

How this job compares

Set against the 36,943 open Software Engineer jobs on Olive Jobs, counted November 20, 2026.

Pay
  • This postingNot listed
  • California median$161–243K/yr6,680 salary ranges
  • California median$48–67/hr122 hourly ranges
Openings
Olive Jobs lists 8,260 open Software Engineer jobs in California and 36,943 across the US. 8% of them nationwide are listed as remote or flexible.
Hiring the most in California
Experience
This posting asks for at least 5 years of experience; Software Engineer postings that state a minimum ask for 5 years at the median (30,715 postings).
Education
10,965 of the 36,943 open Software Engineer jobs (30%) ask for a bachelor's degree or higher.

Pay, requirements and demand for Software Engineer jobs

Counted across the open, public US postings on Olive Jobs on November 20, 2026. Figures based on too few postings are left out.

Prepare for this role

You've read the whole posting — now see how you match it.