Erlang Developer - Remote
Zyte, Hungary
Job Description
About Us
At Zyte (formerly Scrapinghub), we eat data for breakfast and you can eat your breakfast anywhere and work for Zyte. Founded in 2010, we are a globally distributed team of over 190 Zytans working from over 28 countries who are on a mission to enable our customers to extract the data they need to continue to innovate and grow their businesses. We believe that all businesses deserve a smooth pathway to data
For more than a decade, Zyte has led the way in building powerful, easy-to-use tools to collect, format, and deliver web data, quickly, dependably, and at scale. And today, the data we extract helps thousands of organizations make smarter business decisions, secure competitive advantage, and drive sustainable growth. Today, over 2,000 companies and 1 million developers rely on our tools and services to get the data they need from the web.
About the Job:
Smart Proxy Manager (SPM) is a smart downloader designed specifically for web crawling and scraping, removing the headaches of proxy management. It is part of the Zyte platform, the world’s most comprehensive web crawling stack which powers crawls of over 8 billion pages per month.
As an Erlang developer you will help to ensure the robustness of our services. You will learn to investigate production issues on a server executing customer requests. You will be able to navigate a large code-base and find the least obstructive place for extensions. Beside the technicalities you will gain a holistic view of the product and ensure a greater usability of the system with every single task you complete. In this role, you will partake in brainstorming and delivering improvements to the core of SPM.
Job Responsibilities:
- Develop, maintain and support a high load distributed system.
- Analyze our current and historical SPM usage to augment and enhance its routing and rotation logic.
- Leverage the Zyte platform to provide extended functionality, both to end users and for internal purposes.
- Identify and resolve performance and scalability issues with distributed crawling at scale.
- Liaison with other platform teams to provide SPM with the best possible integration to the growing Zyte platform.
Requirements
- 2+ years of production experience with Erlang.
- Strong communication in written and spoken English.
- Strong knowledge of Linux/UNIX, HTTP and Networking.
Bonus points for:
- Python or Golang experience.
- Familiarity with techniques and tools for crawling, extracting, and processing data.
- Knowledge of ELK, Graylog, Docker and Mesos.
- Strong record of open source activity
- Experience working with Lean principles and a Scrum SDLC
Benefits
As a new Zytan, you will:
Become part of a self-motivated, progressive, multi-cultural team.
Have the freedom and flexibility to work from where you do your best work.
Attend conferences and meet with team members from across the globe.
Work with cutting-edge open source technologies and tools.
35 days paid holidays