# Multi-Modal Fusion Search: Picking the Right Retriever For Every Query

Canonical source: https://maguyva.ai/blog/multi-modal-fusion-search

License: text under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/); code samples under [MIT-0](https://opensource.org/license/mit-0); the Maguyva name, logos, brand assets and other code are not covered. Details: [maguyva.ai/license](https://maguyva.ai/license).

Date: 2026-04-15 · Author: Maguyva Team

> A query like 'where is parseConfig defined' wants a different search than 'how does auth work'. Maguyva classifies the intent, weights four retrieval modalities accordingly, and fuses the results with weighted Reciprocal Rank Fusion.

> Historical design note: this April 2026 article describes the routing design at that time, not the current default behavior. The September correction below addresses a measurement claim only. For current routing, presets, and explicit modality controls, see [How It Works](https://maguyva.ai/docs/how-it-works).

A search query is not one thing.

"where is `parseConfig` defined" wants an exact symbol — one precise location, fast. "how does authentication work" wants meaning — the spread of related code that explains a concept. "what breaks if I change this function" wants the dependency graph. "find the string `ECONNREFUSED`" wants a literal match, nothing clever.

Grep is excellent for literal matches and useful for some reference hunts, but it is not a dependency graph and it does not understand meaning. Embeddings cover the semantic side, but they are the wrong tool for exact strings and impact analysis. Most code search tools pick one engine and make every query live with that choice. Maguyva doesn't pick. It works out what kind of question you asked, then blends four retrievers in the proportion that question deserves.

## Four modalities

Under the hood there are four independent ways to find code:

- **semantic** — vector search over Voyage binary embeddings; finds code by meaning.
- **text** — trigram matching; finds literals, error strings, exact identifiers.
- **structural** — AST queries; finds definitions, signatures, and language constructs.
- **graph** — the dependency graph; finds callers, callees, and blast radius.

Each is strong on a different class of question. The trick is deciding how much to trust each one for the query in front of you.

## Intent classification

Before any retrieval runs, a lightweight classifier sorts the query into one of six intents, with a confidence score. It's deliberately cheap — ordered heuristics, first match wins — because it runs on the hot path and adds only a millisecond or two:

- starts with `def `, `class `, `func `, `import `… → **find_definition** (confidence 0.95)
- "who calls", "usages of", "references to" → **find_references** (0.90)
- "impact", "blast radius", "what depends on" → **impact_analysis** (0.90)
- a quoted `"string"` or an error token like `traceback` → **exact_match** (0.85–0.90)
- a `CamelCase` or `snake_case` identifier → **find_definition** (0.60–0.80)
- "how", "why", "explain", "architecture" → **understand_code** (0.75)
- nothing matches → **understand_code**, low confidence (0.40)

Each intent carries a weight profile across the four modalities. These are the real numbers:

| Intent | semantic | text | structural (AST) | graph |
|---|---|---|---|---|
| find_definition | 0.2 | 0.1 | **0.6** | 0.1 |
| find_references | 0.1 | 0.2 | 0.2 | **0.5** |
| understand_code | **0.5** | 0.2 | 0.2 | 0.1 |
| find_similar | **0.4** | 0.3 | 0.2 | 0.1 |
| impact_analysis | 0.1 | 0.1 | 0.1 | **0.7** |
| exact_match | 0.0 | **0.9** | 0.1 | 0.0 |

So "where is `parseConfig` defined" leans hard on AST (0.6). "how does auth work" leans on semantic vectors (0.5). "what depends on this" is almost all graph (0.7). "find `ECONNREFUSED`" is almost all trigram (0.9), with the embedding model switched off entirely — because semantic similarity is exactly the wrong tool for an exact string.

## The fast path, and the fused path

When the classifier is confident — score ≥ 0.85 — and the query is a normal one, Maguyva skips fusion entirely and routes straight to the single dominant modality. "where is X defined" doesn't need four retrievers; it needs the AST index, now. That direct path is reported back as `fusion_strategy: "direct"`.

Everything ambiguous goes through fusion. The four (or three, on the default preset) modalities run in parallel, each returning its own ranked list, and we combine them.

## Weighted Reciprocal Rank Fusion

Fusing heterogeneous retrievers is harder than it sounds: a cosine similarity of 0.82 and a trigram score of 137 and a graph centrality of 0.004 are not on the same scale, so you cannot just add them. Reciprocal Rank Fusion sidesteps the problem by throwing away the raw scores and keeping only the **rank** each engine assigned. A result's contribution from one modality is:

```text
contribution = weight × 1 / (k + rank + 1)
```

where `rank` is its position in that modality's list and `k` is a smoothing constant. Contributions are summed across modalities for any result that more than one engine found — agreement between retrievers naturally floats to the top. We use **k = 40** on the default preset and 60 on `thorough` (`quick` runs semantic-only, so fusion never kicks in there). The original RRF work landed on k = 60 for general-purpose retrieval; we default slightly sharper, which gives top-ranked agreement between modalities a bit more weight — and we don't recommend hand-tuning it.

On top of that, results carry a **graph-importance boost**. A hub — a function the whole codebase leans on — should outrank an obscure leaf even at the same textual relevance, so we multiply each contribution by:

```text
boost = min(1 + 0.3 × ln(1 + centrality), 1.5)
```

Centrality comes from the pipeline's precomputed PageRank/degree metrics, and the boost is clamped at 1.5× so a popular function can't completely bury a more relevant obscure one. Finally we demote results from vendor, build, and archive paths, and dedupe to the best chunk per file.

## What's still imperfect

The intent classifier is a stack of regexes, not a learned model. It handles common query shapes with heuristics, and an ambiguous query falls through to `understand_code` and a semantic-leaning blend. An earlier version of this post described a zero-result reduction from roughly 15% to under 5% as achieved. The December 2025 proposal actually listed an estimated baseline and a target; no supporting query set or measurement receipt is supplied. We withdraw that achieved-improvement claim; see the [source evidence and correction](https://maguyva.ai/evidence/search-quality.md). We haven't replaced the classifier with a trained model because the cheap version is fast and good enough for its intended routing role, and because a wrong-but-confident classifier is worse than an honest fallback. The weights themselves are hand-chosen priors, not learned from click data we don't collect.

## Ask the Question, Not the Tool

An agent shouldn't have to know whether to reach for grep or embeddings or the call graph — it should ask its question in plain terms and get the right answer. Multi-modal fusion is what lets `find_symbol`, semantic search, and dependency analysis sit behind one query surface: the system reads the shape of the question and quietly assembles the right retriever for it. The model that scores the embeddings matters, but so does knowing when *not* to use them. Picking the right tool for each query is its own kind of quality, and it's one we'd rather own than push onto the caller.

`// you bring the question. it brings the tools.`