Role Overview
We are looking for an experienced AI Operations SME / Agent Specialist to serve as a senior escalation point for complex AI and agent-related operational issues. The role will focus on diagnosing failures across prompts, context, workflows, tools, policies and agent behavior, and translating recurring issues into actionable improvements for Operations, Product and Engineering teams.
The ideal candidate combines strong hands-on experience with generative AI and AI-enabled workflows with structured troubleshooting, root-cause analysis and technical communication skills.
Responsibilities
- Serve as the escalation point for complex AI-related issues that cannot be resolved through standard operational playbooks.
- Diagnose failures involving prompts, context, workflows, tools, policies, and repeated agent behavior.
- Perform structured root-cause analysis of recurring AI workflow issues using available logs, context, and supporting information.
- Differentiate AI/model-related failures from data gaps, policy gaps, workflow issues, and tool failures.
- Reproduce issues and document findings clearly to support Product and Engineering investigation.
- Develop and maintain advanced troubleshooting guides, knowledge articles, and operational playbooks.
- Coach Operations teams on complex troubleshooting scenarios and appropriate escalation paths.
- Identify recurring failure patterns and translate them into actionable feedback for Product, Engineering, and Operations.
- Support prompt testing, regression testing, edge-case validation, AI evaluation, and defect reproduction as needed.
Required Skills and Experience
- 5 to 8 years of experience in areas such as AI operations, automation, product operations, QA/testing, or technical support.
- Strong hands-on experience using Generative AI, including prompt and context engineering.
- Experience troubleshooting complex AI or automated workflows and performing structured root-cause analysis.
- Understanding of AI agent behavior, including agent actions, tools, guardrails, retries, workflows, and escalation logic.
- Demonstrated experience resolving complex workflow or system failures.
- Experience creating troubleshooting documentation, knowledge articles, or operational playbooks.
- Strong technical communication skills with the ability to work effectively across Operations, Product, and Engineering teams.
- AI testing experience is preferred and may become a larger part of the role as the program matures.
