Backend Software Engineer (Evals) – Support Automation Engineering
Backend Software Engineer (Evals) – Support Automation Engineering
Backend Software Engineer (Evals) – Support Automation Engineering
OpenAI
The Support Automation team at OpenAI is seeking a Backend Software Engineer to design and build an evaluation infrastructure for measuring the quality of support automation. This role involves creating robust backend systems and APIs, collaborating with data science and research teams, and leveraging OpenAI's technology to enhance automation products. The ideal candidate will have extensive backend engineering experience, particularly in ML/LLM domains, and will focus on building scalable and maintainable systems.
Qualification
- 4+ years of backend engineering experience at product-driven companies
- Proficiency in backend technologies including Python, FastAPI, and Postgres
- Experience designing and scaling distributed systems, APIs, or data processing pipelines
- Experience building AI agents or applications, including designing evals
- Familiarity with evaluation methods for LLMs and multi-agent workflows
- Experience creating production evals and measuring performance of ML/LLM models at scale
- A pragmatic mindset for iterative development
Responsibility
- Design eval pipelines that are reliable, reproducible, and extendable
- Build infrastructure for continuous eval monitoring frameworks
- Design, build, and maintain backend services and APIs for automation and knowledge systems
- Integrate and structure data across internal platforms for AI workflows
- Collaborate with data, research, and engineering teams to integrate OpenAI models
- Own the full development lifecycle of new backend systems and internal platform capabilities
- Build with scale and maintainability in mind while iterating on new ideas
