RAG Chatbots: Your Visitor Speaks French, Your Corpus Speaks English
Cross-lingual embeddings, multilingual rerankers, and the language of the answer: the three places where a RAG chatbot wins or loses when the question and the documentation aren’t in the same language.
I run quite a few AI chatbots, on very different sites and with very different tools. One scenario keeps coming up: the knowledge base is in English (product documentation, spec sheets, in-depth articles), and a share of the visitors ask their questions in French. Sometimes it’s the other way around, with a French corpus and English-speaking visitors. The client, for their part, doesn’t see the problem: “AI speaks every language, doesn’t it?”
Yes and no. A RAG chatbot isn’t a model; it’s a pipeline. The question passes through three links before it becomes an answer, and each one reacts differently to a change of language. The first holds up better than you’d expect. The second is the one almost everyone forgets. The third is the one that betrays you in production, right in front of the visitor.
The pipeline, so we’re talking about the same thing
Whatever the software, the mechanics are the same everywhere.

sequenceDiagram
participant V as Visitor<br/>(question in French)
participant C as Chatbot
participant E as Embedding model
participant B as Vector database
participant R as Reranker<br/>(optional)
participant L as LLM
V->>C: Asks a question
C->>E: Sends the question
E-->>C: Question vector
C->>B: Searches for the closest chunks
B-->>C: Top 20 to 50 candidate chunks (in English)
alt With reranker
C->>R: Question + list of chunks
R-->>C: Reranked chunks (top 3 to 8)
else Without reranker
Note over C: Keeps the top chunks<br/>by similarity
end
C->>L: System prompt + chunks + question
L-->>C: Generated answer
C-->>V: Displays the answer
Code language: PHP (php)
Three verbs sum up these steps: retrieve, rank, respond. A change of language puts each of them to the test in turn.
Retrieve: embeddings hold up better than you’d fear
The good news first. With a modern multilingual embedding model, a French question asked against an English corpus generally surfaces the right passages.
The reason lies in what an embedding encodes. It doesn’t compare words; it places meanings in a space. “Délai de livraison” and “shipping time” share no useful letters, but they mean the same thing and land in the same spot. The model still has to have been trained for this. That’s the case with voyage-3-large, which its publisher describes as a general-purpose, multilingual model, evaluated in part on multilingual datasets. The voyage-3 generation was also tested across 26 languages.
I’d already run into this when indexing German technical documentation that I queried in French: I chose to keep the German rather than translate it, precisely because the embedding bridged the two languages.
Three caveats, to be fair:
- French and English are the easy case: two closely related languages, massively represented in the training data. French-Japanese or English-Arabic won’t behave as well.
- Cross-lingual search is generally a little weaker than single-language search. The drop is often modest, but it’s real, and it hits short, ambiguous questions first.
- An embedding model trained only on English doesn’t build that bridge at all. Check your model’s documentation before blaming the rest of the pipeline.
There’s an extra trap for anyone who has set up hybrid search. The lexical side, BM25 and its cousins, compares words: given a French question and an English corpus, it finds nothing, apart from proper nouns, product references, and technical terms that stayed in English. In this scenario, the entire weight of retrieval rests on the vector. All the more reason to get the next step right.
Rank: why the reranker counts twice
This is the link I’d covered least until now, and it’s the one that changes the game the most.
I described reranking in my RAG guide: you retrieve broadly, twenty to fifty passages, then a second model reorders them and keeps only a handful. What I didn’t say is why that second pass is worth even more when the languages differ.
It all comes down to how the two steps work. The embedding encodes the question on one side and each passage on the other, separately. The French question and the English passage never actually meet: you’re comparing two geometric summaries, computed at different times. That’s exactly where the cross-lingual quality loss hides, because the slightest misalignment between languages shifts the ranking.
The reranker works differently. As Voyage’s documentation puts it, rerankers are cross-encoders that process the query-document pair jointly. The model reads the French question and the English passage together, in a single pass, and judges directly whether one answers the other. It no longer depends on the geometry of a vector space: it does what a bilingual reader would do.
In practice, vector search brings back the right passages, but out of order. The best one often sits in eighth or fifteenth place, behind passages that are only loosely related. Without a reranker, you send the top five to the LLM and the right one isn’t among them. With one, it rises to the top.
Three rules for making this work.
The reranker itself must be multilingual. This is the main trap. Plenty of tutorials reflexively plug in a small cross-encoder from the ms-marco family, because it’s free and runs on a CPU. But that model only understands English: faced with a French question, it ranks essentially at random and undoes what the embedding got right. Serious options exist: Voyage’s rerank-2.5, announced with multilingual support, Cohere’s multilingual variants, or bge-reranker-v2-m3 if you self-host.
Cast a wider net. A reranker can only reorder what it’s given. If the right passage isn’t among the candidates, it’s lost. Cross-lingually, where the initial ranking is noisier, I go up to fifty candidates, even a hundred, instead of twenty.
Make sure your tool supports it. Not every off-the-shelf chatbot solution lets you plug in a reranker yet. It’s a criterion I should have added to my decision grid for client-site chatbots: on a multilingual site, a solution without reranking hits a lower ceiling than the others, whatever model sits behind it.
Respond: when the prompt imposes its language
The right passages have been retrieved and ranked. All that’s left is to answer, and this is where the most visible failure shows up: the visitor writes in French, and the chatbot replies in English.
The mechanism is simple. The LLM doesn’t respond only to the question; it responds to everything in front of it. Do the math: a two-thousand-word system prompt in English, five passages of documentation in English, and at the very end, an eight-word question in French. The visitor’s language makes up a tiny fraction of the context. The model follows the slope.
It gets worse in three situations: the fast, cheap models people use precisely for chatbots; short questions (“Price?”, “Warranty”, a bare product name); and questions packed with English technical terms, which the model mistakes for English. It also works in reverse: a long system prompt written in French will make the model answer an English-speaking visitor in French.
The obvious fix, a sentence somewhere in the system prompt saying “answer in the user’s language,” isn’t always enough. Buried in two thousand words of instructions, it carries little weight against everything else. Here’s what works, from most effective to simplest:
- Repeat the instruction as close to the question as possible. In the message template sent to the model, after the passages and right before the question: “Visitor’s question, which you answer in the same language.” The model gives more weight to what is close and recent.
- Detect the language and state it explicitly. A detection library, or the site’s active language, lets you inject “Respond in French.” An absolute instruction beats a relative one.
- Label the passages as source material. “The documents below may be in English. Don’t adopt their language; translate whatever you use from them, keeping product names and technical terms as they are.” The model understands that the language of the sources isn’t the language of the answer.
- Put the language instruction at the end of the system prompt, not the beginning. That’s the bare minimum if you can’t do anything else.
On a bilingual site, where the same chatbot serves two audiences, only the first two really hold up.
The alternatives, and what they cost
Translate the question before retrieval. A small model translates the question into English, and the search runs in a single language. It’s effective and cheap, but it adds a call and a few hundred milliseconds, and a clumsy translation of a technical term can send retrieval off into the weeds. The cautious version is to search with both the original question and its translation, then let the reranker decide between them.
Translate the corpus. This is the heavy option: two databases to maintain, double the indexing costs, and a translation that freezes one interpretation in place. It only makes sense if most of your visitors speak the other language.
Switch embedding models. If the volume of cross-lingual questions becomes significant, more specifically multilingual models such as BGE-M3 are worth a try, with the added benefit that you can host them yourself. But be careful: switching embedding models means reindexing everything. That’s not a decision you make to gain two points.
The pushback I’ll get
“Context windows are huge now; just send fifty passages to the LLM.” You can. Then you pay for fifty passages on every question, you slow down the answer, and you drown the model in text unrelated to the question, which makes the language drift described above even worse. A reranker costs a fraction of that.
“A reranker adds latency.” Yes, on the order of a few hundred milliseconds with an API. On a chatbot whose answer takes two or three seconds to appear, nobody notices. A bad answer, they do.
“My chatbot answers correctly in French; I don’t have this problem.” Maybe. Have you measured it? A fluent answer in French doesn’t prove the right passage was retrieved: the model is very good at embroidering around a mediocre one.
My take
Multilingual support is the most oversold promise in AI chatbots. You tell the client “it works in every language,” you test three questions in English, and you ship. What works in every language is the LLM. The pipeline feeding it was designed with only one in mind.
My checklist fits in three lines. A multilingual embedding model, or nothing comes back. A multilingual reranker as soon as cross-lingual questions are more than anecdotal, because it’s the link with the best return on effort in the entire pipeline. A language instruction repeated as close to the question as possible, because it’s the only failure the visitor sees with their own eyes.
And before any of that, a check almost nobody does: put together twenty real questions in French, run them against your English corpus, and see where the right passage lands, with and without a reranker. I’ve explained how to build an evaluation that doesn’t fool itself. Ten minutes of measurement are worth more than every product sheet out there, mine included.