Web Crawler Engineer
Analyze target sites, APIs and anti-bot defenses, design stable and maintainable collection strategies, and help build an enterprise-grade crawling system with scheduling, monitoring and alerting.
- Monthly pay
- ¥7–12K / month
- Experience
- Under 1 year
- Education
- Any degreeNo degree or major requirements
- Location
- Tianhe, Guangzhou
Why join
- Led by a senior crawler engineer, with joint solution design, code review and engineering rollout
- Build an enterprise data platform instead of scattered one-off scripts
- AI-assisted development across the team, with unlimited Claude and Codex usage
- Interns welcome: we value passion over experience
What you will do
- Own business data collection, including API analysis, parsing, pipeline debugging and stability improvements
- Analyze target pages, APIs and anti-bot strategies, and design stable, maintainable collection approaches
- Help build the collection system, including task scheduling, retries, log monitoring and alerting
- Maintain proxies, cookies, tokens, accounts and other collection resources to raise success rates
- Work with backend, AI and infrastructure teams to turn collection needs into stable data services
What we look for
- Familiarity with Python and experience in data collection or automation, using AI to work faster
- Knowledge of common anti-bot measures such as IP limits, rate limiting, CAPTCHAs, fingerprinting and request signing
- Familiarity with common storage such as MySQL, PostgreSQL and Redis
- Strong troubleshooting skills using logs, API responses and browser developer tools
Nice to have
- Familiarity with tools such as curl_cffi, Playwright or Scrapling
Tech stack
- Python
- Playwright
- curl_cffi
- Scrapling
- JS reverse engineering
- PostgreSQL
- MySQL
- Redis
Apply for this role
Email your CV with the role name in the subject line, or message us directly on BOSS Zhipin.