Backend Software Engineer - Search, Crawler Team (London, Belgrade, Berlin)
Backend Software Engineer - Search, Crawler Team (London, Belgrade, Berlin)
Backend Software Engineer - Search, Crawler Team (London, Belgrade, Berlin)
Perplexity
We are looking for an experienced Backend Software Engineer to join our Crawler team, focusing on the design, development, and operation of systems for ingesting and processing web-scale data to support advanced search technologies. This role requires expertise in both backend and frontend components of data acquisition.
Qualification
- Minimum of 5 years of software development experience.
- Strong knowledge of data structures and algorithms in Python, C++, Rust, or Go.
- Experience with large-scale web crawlers is highly desirable.
- Proven experience with high-load, distributed, and hardware-adjacent services.
- Deep understanding of cloud infrastructure, with hands-on experience in Kubernetes and AWS.
Responsibility
- Lead projects to develop large-scale web crawlers and ingestion pipelines.
- Build and optimize backend and frontend components for crawler services, including storage and UI dashboards.
- Collaborate with Search and Infrastructure teams for reliable data ingestion and processing.
- Architect and implement fullstack features and scalable distributed systems for high-load data operations.
- Continuously enhance system performance, usability, and reliability through rapid iteration and experimentation.
