Building a RAG Knowledge Base with Free Embedding APIs

Retrieval-Augmented Generation (RAG) is the most practical AI application pattern today — letting LLMs "reference" your private knowledge base rather than hallucinating answers. This tutorial walks you through building a complete RAG system from scratch using free APIs.

RAG Architecture Overview

User Query → Embed Query → Vector DB Search → Retrieve Relevant Chunks → Assemble Prompt → LLM Generates Answer

Core components:

Step 1: Environment Setup

pip install chromadb openai langchain langchain-community

Access multiple embedding and LLM models through APIShare with a single token:

import os
os.environ["OPENAI_API_KEY"] = "your-apishare-token"
os.environ["OPENAI_BASE_URL"] = "https://apishare.cc/v1"

Step 2: Load and Split Documents

from langchain_community.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

loader = TextLoader("./knowledge_base.txt")
documents = loader.load()

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50,
    separators=["\n\n", "\n", ".", "?", "!", " ", ""]
)
chunks = text_splitter.split_documents(documents)
print(f"Split into {len(chunks)} chunks")

Step 3: Embed and Store in Chroma

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

embedding = OpenAIEmbeddings(
    model="text-embedding-3-small",
    dimensions=512
)

vectorstore = Chroma.from_documents(
    documents=chunks,
    embedding=embedding,
    persist_directory="./chroma_db"
)
print(f"Vector DB created with {len(chunks)} records")

For better Chinese performance, switch to BGE-m3:

embedding = OpenAIEmbeddings(
    model="BAAI/bge-m3",
    openai_api_base="https://apishare.cc/v1",
    dimensions=1024
)

Step 4: Build the Retriever

retriever = vectorstore.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 5}
)

query = "What embedding models does APIShare support?"
results = retriever.invoke(query)
for i, doc in enumerate(results):
    print(f"--- Result {i+1} ---")
    print(doc.page_content[:200])

Step 5: Assemble Prompt and Call LLM

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

llm = ChatOpenAI(
    model="gemini-2.0-flash",
    temperature=0.3
)

template = """You are a knowledgeable assistant. Answer based on the reference materials below.
If you cannot find the answer in the references, say so honestly.

Reference Materials:
{context}

User Question: {question}

Answer:"""

prompt = ChatPromptTemplate.from_template(template)

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

answer = rag_chain.invoke("What embedding models does APIShare support?")
print(answer)

Step 6: Advanced Optimization

6.1 Hybrid Search

Combine keyword (BM25) and vector retrieval for precision + semantics:

from langchain.retrievers import BM25Retriever, EnsembleRetriever

bm25_retriever = BM25Retriever.from_documents(chunks, k=3)
vector_retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

ensemble_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, vector_retriever],
    weights=[0.5, 0.5]
)

6.2 Re-ranking

Re-rank retrieved documents with a stronger model for better Top-3 precision:

from langchain.retrievers import ContextualCompressionRetriever
from langchain_cohere import CohereRerank

compressor = CohereRerank(model="rerank-english-v3.0", top_n=3)
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor,
    base_retriever=retriever
)

6.3 Multi-turn Conversation

Add chat history for follow-up question support:

chat_history = []

def rag_with_history(query):
    context_docs = retriever.invoke(query)
    context = format_docs(context_docs)

    messages = [
        {"role": "system", "content": f"Reference: {context}"},
        *chat_history[-6:],
        {"role": "user", "content": query}
    ]

    response = llm.invoke(messages)
    chat_history.append({"role": "user", "content": query})
    chat_history.append({"role": "assistant", "content": response.content})
    return response.content

Project Structure

my-rag/
├── knowledge_base.txt      # Your knowledge base
├── chroma_db/              # Persisted vector DB
├── rag_engine.py           # Core RAG logic
├── ingest.py               # Document loading & embedding
└── query.py                # Interactive query

Cost Analysis

Running a RAG system with 100 queries/day at zero cost is entirely feasible.

Conclusion

This tutorial demonstrated the complete RAG pipeline: document loading, splitting, embedding, Chroma storage, retrieval, and LLM generation. Through APIShare's unified endpoint, you can freely switch between embedding models and LLMs within a single SDK, building production-grade RAG applications at zero cost.

Next steps:

• Try multimodal RAG (images + text)
• Integrate more data sources (PDF, web, databases)
• Use Agent frameworks for autonomous retrieval decisions

🚀 Get Started: One-Click Free API Access

Want to call all the free models above with a single API key, no need to sign up for each provider? Apishare.cc provides a unified API Key — one key, 100+ models, free models at zero cost.

👉 Register on Apishare.cc → Get your unified API Key

📊 Want to see more free model rankings? Check out the Sep 2026 Free LLM API Rankings →

About the Free API Aggregator

The models covered in this guide are all served through the APIShare free API aggregator, which gives you one key for the whole catalog.

• Full model catalog: APIShare free API directory
• Sign up for a free trial key: Register and claim your API key