This role is focused on ensuring the quality, reliability, safety, and performance of AI-powered agents across a range of user interactions and business workflows. You will work closely with engineering, product, and business stakeholders to validate behaviour, test conversational flows, and continuously improve overall system performance.
The Role Testing & Quality Assurance
- Design, execute, and document comprehensive test plans and test cases
- Perform manual and automated testing across conversational journeys, integrations, APIs, and user interfaces
- Validate response accuracy, intent recognition, business rule adherence, and system behaviour
- Conduct regression testing following updates, configuration changes, or retraining cycles
- Test across multiple user channels (web, mobile, internal systems, chat interfaces)
- Log clear, reproducible defects with detailed supporting information
- Collaborate with technical teams to troubleshoot issues and verify fixes
- Identify edge cases, gaps in requirements, and potential failure points
- Analyse interaction logs to detect inconsistencies, misclassifications, and optimisation opportunities
- Monitor key metrics such as response accuracy, latency, escalation rates, and user experience indicators
- Provide feedback to improve training data, prompts, and configuration
- Ensure solutions meet organisational security, data handling, and responsible AI standards
- Validate that responses adhere to safety guardrails and compliance requirements
- Support documentation and audit readiness processes
- Proven experience in software testing, QA, or AI/conversational system testing
- Strong understanding of functional, regression, exploratory, and UAT testing methodologies
- Experience testing APIs, integrations, and JSON-based payloads
- Familiarity with automation tools and testing frameworks
- Understanding of AI/ML concepts, LLM-based systems, or chatbot technologies
- Strong analytical skills and attention to detail
- Experience working within agile delivery environments
- Experience with Salesforce AgentForce
- Experience testing AI agent or conversational platforms
- Exposure to prompt engineering, NLP, or training data optimisation
- Experience working in secure or regulated environments
- Currently SC Cleared, or
- Eligible for SC Clearance (UK national or minimum five years UK residency with no significant gaps)