RAG Chatbot for Websites: How an AI Chatbot Answers from Your Own Content

4. November 2026 7 min. read Artificial Intelligence Sven Lorenz

What makes a RAG chatbot different

A RAG chatbot is an AI chatbot that answers questions only from defined content, such as a company’s website, PDFs and data sheets, and links the source for every answer. A language model like ChatGPT answers from its training data.

On a company website, the difference shows quickly: visitors ask about the prices, deadlines and products of that specific company. The model only knows what is typical in the industry. If someone asks for the price of a specific plan, it gives a figure that sounds plausible and may be entirely made up. A RAG chatbot looks it up in the company’s content instead and shows the passage the answer comes from.

At a glance

RAG stands for Retrieval Augmented Generation. For every question, the system searches the stored content for matching passages and passes only those to the language model.
A language model without its own data foundation answers from training data. The answers sound plausible but cannot be verified.
Quality depends mainly on retrieval: how the content is split up and how reliably the system finds the right passage.
If a RAG chatbot finds no basis for an answer, it should hand off to a human.
Two places matter for the GDPR: the database with content and chat histories, and the language model. Under the EU AI Act, a chatbot must also disclose that it is AI.

What happens when companies run ChatGPT directly on their website

A language model without its own data foundation answers from what it learned in training. The company’s website is usually not part of that, or only in an outdated version.

A prospect asks the chat about the notice period. The model names a period that is common in the industry. If the terms and conditions state a different period, the chat has given a wrong answer, and nobody at the company notices. If someone asks about a product the company doesn’t offer, the model describes it anyway.

This behavior is called hallucination: the model produces a plausible answer even when it lacks the facts. It provides no source against which the answer could be checked.

Comparison table: rule-based chatbot, ChatGPT without your data and RAG chatbot by answer source, verifiability, open questions and behavior without a knowledge base.

How RAG works: ingest, structure, store, retrieve, answer

With RAG, the company’s content supplies the knowledge, and the language model phrases the answer from it. Before every answer, the system picks out the relevant passages and hands them to the model.

Ingest. The system pulls content from the connected sources, via CMS interfaces or by crawling, plus PDFs, FAQs and data sheets. A regular sync picks up changes.
Structure. The system splits long texts into shorter passages (chunking). An embedding model calculates a sequence of numbers for each passage that represents its meaning.
Store. The embeddings go into a vector database. Common options are specialized systems such as Qdrant or Weaviate and extensions of existing databases such as pgvector for Postgres.
Retrieve. The system also calculates an embedding for every question. It then searches for the passages closest in meaning.
Answer. The language model phrases the answer from these passages and links the source.

Step two explains why an AI chatbot based on RAG finds more than a full-text search. A visitor asks: “What does it cost if I cancel early?” The website says: “Fees for terminating the contract before the end of the term”. The two sentences share no keywords, so full-text search finds nothing. Their embeddings are close together, so the AI chatbot finds the passage.

What happens when the answer is missing

If a well-configured AI chatbot finds no basis in the stored content, it says so and passes the question on to a person. It does not fill the gap with training data.

This doesn’t happen by itself. Two settings control it: a threshold above which a retrieved passage counts as relevant, and an instruction in the system prompt not to answer without a basis. Whether both work only becomes clear in a test with real questions, including ones the content has no answer to.

Errors cannot be ruled out entirely, even with RAG. The model may summarize a retrieved passage incorrectly. The linked source, however, makes such errors verifiable for visitors and the relevant department.

Unanswered questions show what visitors are looking for and cannot find on the website. Traditional web analytics misses them because it measures page views. A chatbot dashboard should show at least these metrics:

Table of four metrics for a RAG chatbot: answered questions, hand-off rate, topic clusters and content gaps.

Where the data lives: storage and language model

A RAG chatbot creates data in two places. Content, embeddings and chat histories are stored in a database. The language model processes the question and the retrieved passages with every answer. Anyone assessing the chatbot for data protection has to evaluate both places separately.

Storage

The database can run entirely in Europe; the major cloud and database providers have data centers in Frankfurt and other EU locations. These are the key questions before deciding:

In which region are the database and backups located?
Is data separated by customer? On platforms that serve many companies, every query should be technically bound to one tenant, for example through separate partitions. A filter in the application alone offers weaker protection.
How long are chat histories stored? For analysis and quality assurance, a few days are usually enough.

Language model

Where the model runs depends on the provider and the configuration. With US providers, requests often end up in the US unless configured otherwise. Several providers now also process requests in Europe, and there are EU regions of major cloud platforms as well as European model providers. The contract should also specify whether the provider stores the transmitted data or uses it to train its models.

For a website chatbot, the data involved is limited because it processes public content. What visitors type into the chat window can still be personal data, such as a name or a contract number. This belongs in the privacy policy and in a data processing agreement with the model provider.

What the EU AI Act requires of website chatbots

The European AI regulation, the AI Act, sets out transparency obligations in Article 50 for AI systems that interact directly with people. A chatbot must make clear, at the latest at the first interaction, that it is AI.

For a website chatbot, a clear notice in the chat window or in the first message is usually sufficient. A mention in the terms and conditions or the privacy policy does not meet the obligation because it is not visible during the interaction. There is no exemption for small companies.

This article is not legal advice. The assessment in individual cases is up to the company’s data protection officer.

Where RAG reaches its limits

RAG is suited to knowledge that exists as text. For other questions, a chatbot needs other tools.

Real-time data. Stock levels, delivery status or available appointments are not in any text. Here the chatbot has to access the business system directly via an interface.
Calculations. An individual price or a financing rate belongs in a calculation engine. Language models are unreliable at math.
Hard-to-read sources. Scanned PDFs or nested tables only produce good answers if they are cleaned up beforehand.
Little content, few questions. A small website with few inquiries often doesn’t need a chatbot. A well-maintained FAQ page does the same job there.

How to spot a good RAG chatbot

These questions help when comparing solutions:

Checklist of 8 questions for spotting a good RAG chatbot, from source citations to swapping the language model.

Frequently asked questions

What is a RAG chatbot?

A chatbot that, before every answer, retrieves matching passages from defined content and lets the language model answer only from them. RAG stands for Retrieval Augmented Generation. It links the source for every answer.

What exactly does RAG mean in AI?

RAG stands for Retrieval Augmented Generation. Before every answer, a language model is given matching text passages from a knowledge base and phrases the answer on that basis. This lets it work with current, company-specific knowledge that was missing from its training.

What content can a RAG chatbot use?

Anything that exists as text or can be converted into text: website pages, PDFs, FAQs, data sheets or manuals. The content enters the system via CMS interfaces or by crawling. A regular sync ensures that updated pages are reflected in the answers.

How does RAG differ from fine-tuning?

With fine-tuning, the model is further trained on your own data, and the knowledge then sits in the model weights. Every content change requires new training, and no source can be cited. With RAG, the knowledge stays in a database. An updated page is available after the next sync.

Can a RAG chatbot hallucinate?

The risk is lower than with a model without a data foundation, but it still exists. A threshold for relevant passages and a clear instruction in the system prompt ensure that it hands off to a person when there is no basis for an answer. The linked source makes errors verifiable.

Is an AI chatbot GDPR-compliant?

That depends on two places: where content and chat histories are stored and where the language model runs. Both can be operated in the EU. If the model runs with an external provider, a data processing agreement and a notice in the privacy policy are required.

Does a chatbot on a website have to be labeled as AI?

Yes. The EU AI Act requires a chatbot to be recognizable as AI at the latest at the first interaction. A notice in the chat window or in the greeting is usually sufficient. A note only in the terms and conditions or the privacy policy does not meet the obligation, because visitors hardly see it while chatting.

How 360VIER builds RAG chatbots: free choice of model, no CMS migration

We built our own AI chatbot according to these principles. Content and the vector database run on European infrastructure, separated by customer in dedicated database partitions and, on request, in a dedicated environment.

We define the language model per customer and can swap it, including for a model operated in the EU. The prepared content remains in place and can be used for other applications, such as an internal search or an assistant for sales.

The AI chatbot runs on WordPress, TYPO3, headless CMS and other CMS, so you can keep your current system. The chatbot reads content via the WordPress REST API, other interfaces or crawling, and syncs it daily. Your editorial team keeps working in the system it knows.