# How we measure

Canonical source: https://maguyva.ai/methodology

License: text under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/); code samples under [MIT-0](https://opensource.org/license/mit-0); the Maguyva name, logos, brand assets and other code are not covered. Details: [maguyva.ai/license](https://maguyva.ai/license).

This page is the literal place to read how Maguyva quality and cost claims are grounded. It is facts-only: what we measure, how we score it, and what we do not pretend to have audited.

Last updated 2026-10. Current counts regenerate from source data; this page explains the measurement model. The linked evidence record distinguishes implemented fixture rules from unverified historical search targets.

- Fixture gates
- Language-audit dimensions
- Cost transparency

**tl;dr**: Quality reporting uses fixture classifications and multi-dimension language-audit boards. GREEN on known fixtures is not independently verified correct extraction or a guarantee that every enabled language meets those thresholds. Cost claims are workspace pricing math and public operating cost transparency, not a third-party competitive audit.

## 1. Why This Page Exists

Skeptical buyers should not have to reverse-engineer marketing copy. Maguyva indexes repositories and surfaces precision, language coverage, and cost claims across the site. Those claims need a methodology surface that is honest about measurement scope: what is fixture-backed, what is judgment-backed, and what is framing rather than independent certification.

## 2. What We Measure

Code intelligence quality is measured primarily on the language engine — extraction of symbols, relationships, and graphs from source — not on subjective “agent happiness” scores.

- Per-language fixture suites: expected edges and symbols the handler must extract correctly
- Fixture accuracy classification: GREEN requires precision ≥ 0.95, recall ≥ 0.99, F1 ≥ 0.97, and at least 20 expected edges, subject to additional fixture-curation checks. This is a rule for known test cases, not a statistical confidence estimate or a statement that every enabled language is GREEN.
- Language-audit dimensions: accuracy, structural integrity, completeness, quality, and performance on the validation board
- Corpus and spot-check loops: sampled real-repository edges classified by rubric (correct, false positive, type/scope/metadata error) — described in our language-grind blog posts
- Product capability flags: what the server advertises (AST, locals, graph extraction) separate from catalog size

**Important:** Fixtures validate against fixtures we wrote. GREEN means the known cases pass. It does not automatically mean every real-world idiom extracts cleanly. That distinction is deliberate and public.

## 3. Green vs Independently Verified

The language-audit board uses multiple axes so a single green light cannot be read as “proven perfect.” Headline numbers on the board are typically split:

- overall_green — regression-clean against self-snapshot fixtures and related corpus gates (necessary, not sufficient)
- independently_verified — has a strong judgment signal such as a spot-check seed (includes languages that still show judged errors)
- verified_clean / zero judged error on sampled edges — a stricter subset of judged languages
- curation / oracle trust — whether fixtures themselves are treated as trusted oracles
- structural flags — graph extraction and AST capability are not identical to catalog membership

Marketing language count (for example “285+ languages”) is catalog size: configured languages and server entries. Catalog size is not an AST quality SLA. Prefer tiered reporting over a single vanity headcount. For the current product surface, see Compatibility and Language Guides; for narrative detail, see the language recursive self-improvement post on the blog.

## 4. Search and Retrieval Quality

Semantic search quality is multi-modal: text, AST, graph, and embeddings are fused. We document deliberate engineering trades rather than claiming unbeatable retrieval:

- Embeddings use a commercial model family (voyage-4-large as of the June 2026 blog snapshot) chosen against public retrieval leaderboards at the time of decision
- Vectors are binary-quantized for storage and cost; that deliberately trades some retrieval precision for cheaper, faster Hamming search without a separate vector database
- Intent routing and fusion weights are engineered heuristics. A December 2025 proposal estimated a roughly 15% zero-result baseline and set a target below 5%; we have not supplied a query set or measurement receipt that demonstrates that reduction. The evidence record corrects the earlier claim of an achieved improvement.
- Blog posts include “what’s still imperfect” sections — imperfect signals are part of the record, not footnotes to hide

## 5. Cost Claims

Cost language on Maguyva is about pricing structure and operating transparency, not a formal TCO study certified by a third party.

- Workspace pricing: billed by repositories, indexed lines, and rebuild frequency — not per human or agent seat. FAQ and plan copy spell out the dimensions.
- The pricing example compares 25 paid seats at an illustrative $19–$59 per seat with Crew at $49/month. Savings vary with team size and the alternative product; this is not a like-for-like feature comparison or an independent competitive benchmark.
- Operating cost honesty: the /team page publishes a real monthly software burn breakdown (subscriptions, MCP/search tooling, usage-shaped costs). That is customer-zero transparency, not an audited financial statement.
- Embedding and infra costs are accepted product costs (premium embeddings, storage, graph rebuilds). We pay them on purpose and say so in the voyage-4-large post and pricing narrative.

## 6. What We Do Not Claim

This page is also a list of non-claims. If something is not on the measurement board, do not treat marketing tone as proof.

- No blanket absolute-precision guarantee for every language. Fixture thresholds apply per language on known cases; residual real-world error is expected and ground.
- No claim that overall_green equals production-perfect extraction for every repository idiom
- No third-party compliance certification package asserted as a methodology artifact on this surface (see Security for data-handling facts, not compliance badges)
- No independent multi-vendor bake-off with shared corpora published as a standing scorecard
- Search results and analysis remain best-effort under Terms of Service — Maguyva is not a substitute for code review, testing, or security audits

## 7. How to Verify Yourself

The intended buyer motion is still: index a repo you already understand, ask one real question, and inspect citations.

- Start free: small representative repos beat a whole-company index on day one
- Use MCP tools (intelligent_search, find_symbol, dependency_search) and open the cited paths
- Read How It Works for the ingestion and retrieval architecture
- Read Compatibility and Language Guides for capability tiers, not just catalog size
- Read Security and Privacy for data handling; this page does not replace them

- [Search quality evidence and corrections (English)](https://maguyva.ai/evidence/search-quality.md)
- [Search quality evidence (JSON, English)](https://maguyva.ai/evidence/search-quality.json)
- [How It Works](https://maguyva.ai/docs/how-it-works)
- [Compatibility](https://maguyva.ai/docs/compatibility)
- [Pricing](https://maguyva.ai/pricing)
- [Team costs](https://maguyva.ai/team)
- [Security](https://maguyva.ai/security)
- [Privacy Policy](https://maguyva.ai/privacy)
- [Status](https://maguyva.ai/status)
- [Language Recursive Self-Improvement: Grinding Code Intelligence Across ~280 Languages](https://maguyva.ai/blog/autonomous-quality-loops-language-grind)
- [Why We Upgraded Code Search to voyage-4-large](https://maguyva.ai/blog/upgrading-code-search-to-voyage-4-large)

Questions about measurement scope or a claim that looks ungrounded: [support@maguyva.ai](mailto:support@maguyva.ai)