← All roles

Software Engineer, Web Crawling

Exa· Engineering
Apply on company site
SingaporeSGD 90k - SGD 300k + equityFull-timePosted 10mo ago

About the role

Exa is hiring a Software Engineer, Web Crawling to build and scale Google-scale web crawling infrastructure and related systems for a high-performance web search platform.

Responsibilities

  • Build a distributed crawler that can handle 100M+ pages per day
  • Optimize crawl politeness and rate limiting across thousands of domains
  • Design systems to detect and handle dynamic content, JavaScript rendering, and anti-bot measures
  • Create intelligent crawl scheduling and prioritization algorithms for maximum coverage efficiency

Qualifications

  • Extensive experience building and scaling web crawlers, or readiness to ramp up quickly
  • Experience with a high-performance language (C++, Rust, etc.)
  • Familiarity with TypeScript, Playwright, modern web design, CDP (Chrome DevTools Protocol)
  • Comfort with optimizing a system to an exceptional degree
  • Interest in solving problems related to finding high-quality knowledge and its societal impact

Skills

Required:Web crawling and distributed systemsHigh-performance programming
Nice to have:TypeScript and PlaywrightChrome DevTools Protocol (CDP)

Benefits

  • premium healthcare benefits (medical, dental, vision)
  • fertility benefits
  • 16 weeks of fully paid parental leave for all new parents
  • monthly wellness stipend
Last updated Jul 29, 2026. Interested? Apply on the company site.
Exa logo

About Exa

Web search rebuilt for LLMs

Exa is a search engine built for AIs. We enable developers to filter the internet's content with complex queries in order to bring the world's knowledge into their AI apps.

ai-infrastructure-and-developmentapisartificial-intelligencebrex-fastest-growing-2025searchwing-list-2026wing-list-2026-mid-stage
Company profileWebsite

More open roles at Exa