AI Application Engineer
Build and maintain production AI chat and agent systems, from multi-model integration, agent orchestration, context and memory, and streaming responses to evaluation, cost and reliability.
- Monthly pay
- ¥12–20K / month
- Experience
- Any experience level
- Education
- Any degreeNo degree or major requirements; we look at what you can do
- Location
- Tianhe, Guangzhou
Why join
- Unlimited access to top AI coding tools such as Claude Code and Codex
- Work on live systems that power our core business, not throwaway demos
- Senior engineers help you break down problems and design solutions, while you own your own modules
- Year-end bonuses and equity opportunities for sustained, outstanding contributors
What you will do
- Develop, tune and maintain our production AI chat and agent systems so responses stay stable, accurate and fast
- Maintain and extend the multi-model, multi-vendor integration layer, handling differences in capability, parameters, billing, rate limits and failures, with automatic fallback and fault tolerance
- Design and iterate on agent orchestration, including intent routing, tool calls and multi-step task execution, balancing control and flexibility
- Own context engineering, retrieval and long-term memory: decide what goes into a limited context window, how much, and in what order, and use vector search to improve relevance
- Maintain the streaming response pipeline and handle its consistency, persistence and error cases
- Build and use evaluation and observability so every model or prompt change is measurable and regression-tested, and silent failures in production are quick to trace
- Manage cost and reliability through prompt caching, task-based model selection, timeouts and retries, idempotency and rate limiting
- Improve concurrency and maintainability with async Python, message queues and databases, and take part in microservice deployment and operations
What we look for
- Solid Python skills, fluent with asyncio, with an eye for concurrency and performance
- Hands-on experience building LLM applications, with a deep understanding of streaming, structured output and function calling, multi-turn dialogue and context management
- Experience with at least one agent or orchestration framework (LangChain, LangGraph, CrewAI, Agno, etc.); able to explain how nodes, tools and state fit together and debug them yourself
- Understanding of RAG and vector retrieval, including why naive chunk-embed-retrieve pipelines fail and how to improve them
- An engineering mindset for uncertainty: you know LLMs fail and are willing to invest in reliability, retries, fallbacks and evaluation
- Familiarity with PostgreSQL (or a similar relational database) and Redis, able to design data models, transactions and caching on your own
- Experience with message queues such as RabbitMQ or Kafka and async task processing, including at-least-once delivery and idempotency
- Comfortable developing and debugging in Docker-based environments
Nice to have
- You have built an evaluation system, can explain how to tell an improvement from a regression, and know the pitfalls of LLM-as-judge
- Hands-on experience optimizing token cost and latency, and an understanding of how prompt caching works
- Experience with vector databases such as pgvector, or with long-term memory and knowledge-base systems
- Experience with observability tools such as Langfuse, OpenTelemetry or Prometheus, including end-to-end tracing or quality monitoring for AI
- Awareness of AI-specific security risks such as prompt injection, and of defense in depth
- Familiarity with Kubernetes and microservices, with production deployment and operations experience
Tech stack
- Python
- FastAPI
- PostgreSQL
- Redis
- Kafka
- Docker
- RAG
- LangChain
- Prompt Engineering
Apply for this role
Email your CV with the role name in the subject line, or message us directly on BOSS Zhipin.