Transitioning from naïve Top-K dense retrieval to multi-objective submodular optimization in ContextAware RAG architectures.
Why marginal cosine similarity ranking systematically fails to produce reliable generative grounding
Vector-state evaluation dynamics scaling candidate nodes simultaneously across three axes
How semantically similar is this document to the user's query? Computed via cosine similarity between query and document embeddings.
How much new information does this doc bring? Measures diversity — penalizes documents that duplicate already-selected content.
Is this document's claims supported by other selected evidence? Higher support = greater cross-document agreement = more reliable answers.
α·Rel — semantic similarity | β·Cov — marginal coverage gain | γ·Sup — agreement
How we efficiently find the optimal document set — step by step
The actual Python implementation — scoring functions, greedy loop, and retriever integration
📄 app/services/optimizer.py — Scoring Functions
Each function computes one dimension of the multi-objective score. All use cosine similarity as the base metric.
def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
"""Compute cosine similarity between two vectors."""
norm_a = np.linalg.norm(a)
norm_b = np.linalg.norm(b)
if norm_a == 0 or norm_b == 0:
return 0.0
return float(np.dot(a, b) / (norm_a * norm_b))
def compute_relevance(query_emb, doc_emb) -> float:
"""Rel(q, d) = cosine_similarity(q, d)"""
return cosine_similarity(query_emb, doc_emb)
def compute_coverage(doc_emb, selected_embs) -> float:
"""Cov(d | S) = 1 - max(similarity(d, s)) for all s in S."""
if not selected_embs:
return 1.0 # First doc always gets full coverage
max_sim = max(cosine_similarity(doc_emb, s) for s in selected_embs)
return 1.0 - max_sim
def compute_support(doc_emb, selected_embs) -> float:
"""Sup(d, S) = mean(similarity(d, s)) for all s in S."""
if not selected_embs:
return 0.0
total_sim = sum(cosine_similarity(doc_emb, s) for s in selected_embs)
return total_sim / len(selected_embs)Relevance
Range: [-1, 1]
Higher = more relevant
Coverage
Range: [0, 1]
Higher = more unique info
Support
Range: [0, 1]
Higher = more corroborated
📄 app/services/optimizer.py — Greedy Loop
The core algorithm: iteratively picks the document with the highest combined score considering what's already been selected. This is the key difference from top-k.
def optimize_selection(query_embedding, candidates, k, alpha=0.5, beta=0.3, gamma=0.2):
"""
Greedy multi-objective document selection.
Selects k documents maximizing: α·Rel + β·Cov + γ·Sup
"""
query_emb = np.asarray(query_embedding, dtype=np.float32)
candidate_embs = [np.asarray(c["embedding"]) for c in candidates]
relevance_scores = [
compute_relevance(query_emb, emb) for emb in candidate_embs
]
selected = []
selected_embs = []
remaining = list(range(len(candidates)))
while len(selected) < k and remaining:
best_score = -float("inf")
best_idx = -1
for idx in remaining:
# ── THE KEY DIFFERENCE FROM TOP-K ──
# Coverage & support change at every iteration dynamically
rel = relevance_scores[idx]
cov = compute_coverage(candidate_embs[idx], selected_embs)
sup = compute_support(candidate_embs[idx], selected_embs)
score = alpha * rel + beta * cov + gamma * sup
if score > best_score:
best_score = score
best_idx = idx
doc = candidates[best_idx].copy()
doc["opt_score"] = best_score
doc.pop("embedding", None)
selected.append(doc)
selected_embs.append(candidate_embs[best_idx])
remaining.remove(best_idx)
return selected # ← Replaces naive top-k results💡 Why This Works
Iteration 1: Coverage = 1.0 for all, Support = 0.0 → picks the most relevant doc (like top-k).
Iteration 2+: Near-duplicates get Coverage ≈ 0 (penalized), so diverse docs with moderate relevance win.
Result: A balanced set that covers more ground while staying relevant to the query.
📄 app/services/retriever.py — Before vs After
The retriever was modified to use the optimizer when enabled, while keeping the original top-k path intact as a fallback option.
❌ Before — retriever.py
def retrieve(query, top_k=None, filters=None):
top_k = top_k or settings.retrieval_top_k
store = get_vector_store()
# Dense search — fixed small pool
query_emb = embed_query(query)
dense_results = store.search(
query_emb, top_k=top_k
)
# Optional BM25 hybrid
if settings.enable_hybrid_search:
sparse = _bm25_search(query, dense_results)
results = _merge_results(dense_results, sparse)
else:
results = dense_results
# Just slice top-k — no optimization!
return results[:top_k]✅ After — retriever.py
def retrieve(query, top_k=None, filters=None):
top_k = top_k or settings.retrieval_top_k
store = get_vector_store()
query_emb = embed_query(query)
if settings.enable_optimizer:
# Larger candidate pool for optimization
dense_results = store.search_with_embeddings(
query_emb, top_k=settings.optimizer_candidate_n
)
else:
dense_results = store.search(query_emb, top_k=top_k)
# Optional BM25 hybrid
if settings.enable_hybrid_search:
sparse = _bm25_search(query, dense_results)
results = _merge_results(dense_results, sparse)
else:
results = dense_results
# NEW: Optimization-based selection
if settings.enable_optimizer:
from app.services.optimizer import optimize_selection
return optimize_selection(
query_emb, results, k=top_k,
alpha=settings.optimizer_alpha,
beta=settings.optimizer_beta,
gamma=settings.optimizer_gamma,
)
else:
return results[:top_k]Before: 2 steps
Search → Slice top-k
No inter-document awareness
After: 3 steps
Search (N=30) → Optimize → Return k
Each pick considers the whole set
📄 app/core/config.py & .env
All optimizer parameters are configurable natively via environment variables.
config.py Settings Object
class Settings(BaseSettings):
# ── Optimizer (NEW) ────────────────────────────────
enable_optimizer: bool = True # Toggle on/off
optimizer_candidate_n: int = 30 # Pool size (N)
optimizer_alpha: float = 0.5 # α — Relevance weight
optimizer_beta: float = 0.3 # β — Coverage weight
optimizer_gamma: float = 0.2 # γ — Support weight.env Overrides
# Change without reloading
ENABLE_OPTIMIZER=true
OPTIMIZER_CANDIDATE_N=30
OPTIMIZER_ALPHA=0.5
OPTIMIZER_BETA=0.3
OPTIMIZER_GAMMA=0.2🎛️ Tuning Guide
High α (e.g. 0.7): Prioritize relevance — behaves closer to top-k.
High β (e.g. 0.5): Prioritize diversity — prevents redundant chunks.
High γ (e.g. 0.4): Prioritize agreement — good for fact-checking queries.
Constraint: α + β + γ = 1.0 is recommended but not enforced.
How the optimizer selects a diverse, balanced document set compared to naive top-k
Real metrics from evaluation across 8 queries against 283 indexed chunks
🟢 Evidence Diversity — 22.5% (+2.6% vs Top-K)
Measured as (1 − mean pairwise cosine similarity) × 100. The optimizer consistently selects more diverse chunks from the candidate pool.
🟢 Redundancy Reduction — 77.5%
77.5% of near-duplicate pairs (cosine similarity > 0.85) eliminated compared to standard top-k selection.
🟢 Hallucination Resistance — 81.2% (+24.2% vs Top-K)
LLM-as-judge faithfulness scoring. Optimized retrieval grounds 81% of generated claims in source context vs 57% for top-k.
🟢 Answer Reliability — 63.8%
LLM-as-judge completeness scoring. Comparable to top-k baseline (66.2%), with tradeoff between breadth of coverage and depth of focus.
Evaluating the syntactical generation delta across equivalent retrieval pools
Query: "What are the main causes of LLM hallucinations, and how can teams reduce them in production?"
Generated Answer
LLM hallucinations happen when models guess missing facts, especially with ambiguous prompts and weak context. Teams can reduce this by giving better prompts, setting lower temperature, and adding more checks. In general, better data and model tuning help.
Observed Issues
Query: "What are the main causes of LLM hallucinations, and how can teams reduce them in production?"
Generated Answer
Common hallucination causes in LLM systems include weak retrieval quality, missing source citations, stale knowledge, prompt ambiguity, and over-confident decoding. A practical mitigation stack is: (1) retrieval with source attribution, (2) strict grounding prompts, (3) lower-temperature decoding for factual tasks, (4) post-generation verification checks, and (5) human review for high-risk outputs. In production, combine these with evaluation sets and continuous monitoring of factual error rates.
Observed Improvements
How each metric was measured — reproducible with evaluate_quality.py
🔄 Reproduce These Results
All metrics are generated by tests/evaluate_quality.py. Run it against your own data:
python tests/evaluate_quality.py # Full run (all 4 metrics)
python tests/evaluate_quality.py --skip-llm # Embedding metrics only (free)
python tests/evaluate_quality.py --max-queries 3 # Quick test with 3 queriesHow the retrieval pipeline changes with optimization
🎯 Optimization Objective