From Embeddings to FAISS and Nearest-Neighbor Search

In Day 7, our embedding experiments showed that related sentences produce higher similarity scores while unrelated sentences stay far apart in vector space. But there is one problem: a real knowledge base does not contain only three or four vectors—it may contain hundreds of thousands or even millions. Searching every vector individually would be too slow. This is where vector search comes in. Today, we will look at how nearest-neighbor search works, how FAISS organizes and searches embeddings, and how it turns semantic similarity into practical Top-K retrieval for RAG.

1. . We already know how to turn text into vectors and measure similarity. But a real RAG system may contain thousands or millions of vectors, so we need an efficient way to search them.

2. What Is Vector Search?
   Explain that vector search tries to find vectors closest to the query vector.

    User Query
        ↓
    Embedding Model
        ↓
    Query Vector
        ↓
    Vector Search
        ↓
    Top-K Nearest Vectors
        ↓
    Relevant Documents
   

3. Why Do We Need FAISS?
   Manually comparing a query against a few vectors is easy. At larger scale, we need a dedicated similarity-search library: ** FAISS = Facebook AI Similarity Search **

    Document Chunks
         ↓
    Embeddings
         ↓
    FAISS Index
         ↓
    Query Embedding
         ↓
    Search(k=3)
         ↓
    Top-3 Documents
   

Now we step into the world of implemetation:

Then when you ask:
How does vLLM manage GPU memory?

    Query
      ↓
    Embedding Model
      ↓
    Query Vector
      ↓
    FAISS
      ↓
    Compare with stored vectors
      ↓
    Top-K results
    

Now, we step into the world of practical:

In this experiment, FAISS successfully ranked the documents according to their semantic relevance to the query. The most relevant document achieved a similarity score of 0.8093, followed by the PagedAttention document at 0.5871, while the unrelated Taipei night market document scored only 0.0164. This shows that vector search can retrieve information based on meaning rather than exact keyword matching, which is the foundation of document retrieval in a RAG system.

Conclusion:

In Day 8, we learned how vector search turns embeddings into an actual retrieval system. During indexing, document chunks are converted into vectors and stored in FAISS. At query time, the user question is also embedded, and FAISS searches for the nearest vectors to find the most relevant document chunks.

Reference:

1. Reimers, N. & Gurevych, I. (2019). “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.” EMNLP-IJCNLP 2019, pp. 3982–3992.
2. Sentence Transformers — all-MiniLM-L6-v2
3. Sentence Transformers — Semantic Textual Similarity documentation.
4. Malkov, Y. A. & Yashunin, D. A. — HNSW