RAG is a technique for grounding an LLM's answers in your own documents rather than relying purely on what the model memorized during training. It retrieves the most relevant chunks of text from a knowledge base — typically via vector similarity search — and injects them into the prompt alongside the user's question. RAG ranks #6 among trending AI tags at roughly 10.6% usage share, reflecting how many production AI apps need to answer questions about private or up-to-date data that no base model was trained on.
A basic RAG pipeline embeds the user's question, searches a vector store for similar chunks, then builds a prompt that includes those chunks as context before calling the LLM.
# 1. embed the incoming question
query_vector = embed_model.encode("What's our refund policy?")
# 2. find the most similar chunks in the vector database
results = vector_db.similarity_search(query_vector, top_k=4)
context = "\n---\n".join(r.text for r in results)
# 3. build a prompt that grounds the model in retrieved context
prompt = f"""Answer using only the context below.
Context:
{context}
Question: What's our refund policy?"""
# 4. call the LLM with the grounded prompt
answer = llm.generate(prompt)Pick an embeddings model and a vector store, then wire them together with any LLM — no need to build the search index from scratch.
# install a common RAG toolkit
pip install langchain chromadb sentence-transformers
# then: load docs -> split into chunks -> embed -> store in Chroma -> query