Company Chatbot Made a Mistake: 6 Safeguards to Avoid Repeating It

The Chatbot's Wrong Answer: More Than a Bug, a Legal Risk

The implementation of Artificial Intelligence (AI) chatbots on websites and customer service applications has brought numerous advantages to companies, such as process optimization and 24/7 availability. However, a chatbot's wrong answer is not just a one-off technical glitch; in Brazil, this situation can turn into a significant legal liability. Article 30 of the Consumer Defense Code (CDC) binds the supplier to all information or advertising disseminated, even if it originates from automated systems.

Recently, a notorious case in Canada, Moffatt v. Air Canada, highlighted this responsibility. After receiving incorrect information from a chatbot about bereavement fares, the consumer sought redress. The airline attempted to argue that the chatbot was a separate entity and not responsible for its failures, a claim the court found remarkable. The understanding was clear: the chatbot is an integral part of the company's website, and the origin of the information (static page or AI) does not alter the supplier's liability.

This post will cover six essential safeguards to ensure your AI chatbot operates with accuracy, security, and legal compliance, preventing your company from facing legal consequences and reputational damage. If you are a small or medium-sized business owner or a customer service manager who has already integrated or plans to integrate AI into your operations, this comprehensive guide is for you.

Key Points:

• Legal Responsibility: In Brazil, Article 30 of the CDC makes the supplier responsible for information generated by chatbots.
• International Precedent: The Air Canada case shows that claiming AI is a separate entity does not exempt the company from liability.
• Technical Mitigation: Techniques like Retrieval Grounding (RAG) and layered guardrails drastically reduce LLM (Large Language Model) hallucination.
• Preventing Hallucinations: The base hallucination rate of LLMs can vary, requiring multiple layers of protection.
• Integration and Monitoring: Connecting the bot to internal systems and monitoring logs are crucial for accuracy.

Understanding the Legal Risk: The CDC and Information Binding

In Brazil, the Consumer Defense Code (CDC) is the primary safeguard against abusive commercial practices. Article 30 is categorical: "All information or advertising, sufficiently precise, disseminated by any form or means of communication regarding products and services, obligates the supplier, as well as whoever uses it for commercial practices."

This means that regardless of whether the chatbot is a simple conversational flow system or an advanced generative AI model, any information provided to a customer has the power to bind the company. Legal analyses on the subject in the country are unanimous: claiming that a promise or information was automatically generated by the system does not exempt the supplier from its obligation. Failures in automated systems or algorithmic decisions that result in harm to the consumer cannot be passed on to them.

The chatbot is a tool of the supplier, not an independent legal entity. Therefore, responsibility for its failures falls on the company.

The Air Canada Case: A Global Warning

On February 14, 2024, the Civil Resolution Tribunal of British Columbia, Canada, issued a significant decision in the case of Moffatt v. Air Canada. Jake Moffatt, following his grandmother's death, consulted Air Canada's chatbot on the company's website for information on bereavement fares. The chatbot provided incorrect information about the company's policy.

Air Canada, in its defense, argued that it could not be held responsible for the information provided by its chatbot, claiming that the AI was a separate legal entity responsible for its own actions. The tribunal, in a decision that termed this claim remarkable, considered that the chatbot is, in fact, an integral part of the company's website. Therefore, it makes no difference whether the information was presented on a traditional static page or through a conversational AI interface.

This decision reinforces the argument that the technology used does not absolve the supplier of legal responsibility. Companies implementing chatbots must be aware that interactions mediated by these tools create legal obligations.

The Six Essential Safeguards to Avoid Wrong Chatbot Answers

To mitigate the risk of your chatbot providing incorrect information and protect your company from legal liabilities and reputational damage, it is crucial to implement a robust set of safeguards. These safeguards can be divided into technical and procedural measures.

1. Retrieval Grounding (RAG): The Pillar Against Hallucination

One of the biggest challenges with large language models (LLMs) is hallucination: the tendency to generate false information or information without basis in the training data, presenting it as fact. Retrieval Grounding (RAG) is the most effective standalone technique to combat this problem.

RAG works by retrieving relevant information from a reliable database (company documents, FAQs, manuals) and using this data to