Improving Compliance Accuracy with Multi-Expert AI Frameworks

At Wix Payments, our compliance analysts are responsible to ensure all sites which process through Wix Payments meet strict compliance policies. Sites which are suspected to violate the Wix Payments policy are handled manually by expert analysts who follow strict guidelines which ensure Wix Payments is not exposed to financial fines. As Wix Payments grows at a fast rate, the team faces an ever-growing workload. We started Compliance AI with a clear goal: Increase the velocity and work quality of our analysts. With the use of AI, we wanted to automate as much of the manual process as possible and ensure the human experts focus on the most complex cases only. However, we didn’t stop at automation, we used AI to ensure the entire process runs faster, is more accurate, less biased to human judgement and more comprehensive. In this session, we'll share how we built Compliance AI, a framework of specialized large language models (LLMs) that acts as a compliance decision system for Wix Payments. The system automatically reviews e-commerce sites against the policies of different payment providers, helping us scale compliance decisions that were traditionally handled only by human analysts. Our solution was inspired by the medical world: a "family doctor" AI first screens every site with high recall, then routes it only to the relevant specialists, LLMs trained to focus on specific compliance policies. To ensure accuracy, every expert is paired with a “Critic” role, ensuring conclusions are grounded in the actual policy and facts from the site. In this talk, we will walk you through the challenges we faced such as how to capture and represent an entire website for analysis, how to handle complex cases like endless dropshipping catalogs, and how to design a system that can adapt to the unique compliance policies of each provider. Attendees will learn practical strategies for building large-scale LLM frameworks, including approaches to reduce hallucinations, improve accuracy, and create trust between human analysts and AI systems.

Some content is hidden, to be able to see it login here Login