In a RAG pipeline, when a user asks a question, the system first searches a vector index (and optionally keyword indices) for passages relevant to the question, then passes those passages to the language model alongside the original query. The model generates its response conditioned on those retrieved passages, citing them where appropriate.
RAG is the standard architecture for enterprise AI agents because it solves two problems at once: it keeps responses grounded in the organization's actual content (cutting hallucinations) and it lets the knowledge base be updated without retraining the model.
A production-grade RAG implementation includes chunking strategy, embedding model choice, hybrid retrieval (vector + keyword), re-ranking, and citation handling. The details matter: naive RAG often underperforms well-tuned keyword search.