Vector Databases: The AI Memory Revolution
The Reader's Dilemma
Dear Marilyn,Everyone's talking about vector databases for AI applications. But I don't understand why we can't just use regular databases with a "similarity" column. What makes vector databases special, and when should I actually use one?
Marilyn's Reply
The key insight is that traditional databases are designed for exact matches—"find the user with ID 12345." Vector databases are designed for similarity—"find items most similar to this." This fundamental difference requires entirely different data structures and algorithms.
The Spark: Understanding Vector Databases
What Are Embeddings?
An embedding is a numerical representation of data (text, images, audio) as a list of numbers—a vector. Similar items have similar vectors.
Example: Word Embeddings
"king" → [0.2, 0.8, 0.1, 0.9, ...]
"queen" → [0.3, 0.7, 0.2, 0.9, ...]
"apple" → [0.9, 0.1, 0.8, 0.2, ...]
Notice: "king" and "queen" have similar numbers (both royalty), while "apple" is very different.
Quick Check
What is the primary purpose of converting text to embeddings?
Why Not Just Use SQL?
Traditional databases use B-trees and hash indexes—great for exact matches, terrible for similarity search.
| Aspect | Traditional DB | Vector DB |
|---|---|---|
| Query Type | Exact match (WHERE id = 5) | Similarity (find nearest neighbors) |
| Index Structure | B-tree, Hash | HNSW, IVF, PQ |
| Scaling | Sharding by key | Approximate nearest neighbor |
| Use Case | Transactions, reports | Semantic search, recommendations |
Quick Check
Why are B-tree indexes ineffective for vector similarity search?
Vector Index Algorithms
HNSW (Hierarchical Navigable Small World)
Creates a multi-layer graph where each layer has fewer nodes. Search starts at the top (sparse) layer and navigates down to find nearest neighbors. Very fast queries, but memory-intensive.
IVF (Inverted File Index)
Clusters vectors into groups. At query time, only searches the most relevant clusters. Good balance of speed and memory, but requires training on your data.
PQ (Product Quantization)
Compresses vectors by splitting them into subvectors and quantizing each. Dramatically reduces memory usage at the cost of some accuracy.
Quick Check
Which vector index algorithm is best for minimizing memory usage?
Popular Vector Databases
| Database | Type | Best For |
|---|---|---|
| Pinecone | Managed SaaS | Quick start, no ops overhead |
| Weaviate | Open Source | GraphQL API, built-in ML models |
| Milvus | Open Source | Large scale, GPU acceleration |
| pgvector | PostgreSQL Extension | Existing Postgres users |
| Chroma | Open Source | Local development, prototyping |
Quick Check
If you already use PostgreSQL and want to add vector search without a new database, which option is best?