PageIndex and the case against chunking
The most interesting thing about PageIndex is not the 98.7% accuracy claim. It is the architectural premise underneath it: that retrieval should be an act of reasoning, not similarity matching. That a language model navigating a document’s structure (section by section, like a human analyst flipping through a report) will find better answers than an embedding projected into high-dimensional space and measured by cosine distance.
This premise deserves serious examination. PageIndex has not “killed” vector RAG, whatever some corners of the internet have enthusiastically declared. But it exposes a structural weakness in how the field has been doing retrieval, and the evidence for its own alternative is more incomplete than the headline suggests.
What PageIndex actually does
The system, built by VectifyAI, works in two phases. First, it transforms a document into a hierarchical tree index. Think of it as a machine-generated table of contents in which each node carries a title, page range, summary, and child nodes. The structure mirrors the document’s own organisation: sections, subsections, numbered items, appendices.
Second, at query time, an LLM traverses this tree layer by layer. Rather than computing similarity scores against embedded chunks, the model reasons about which branch of the tree is most likely to contain the answer, descends into it, evaluates what it finds, and continues until it reaches the relevant pages. Every step is traceable: which nodes were considered, which were selected, and why.
The claimed advantage is precision over similarity. When a question asks for a certification date, vector search might return a certifications table: topically related, semantically close, but functionally useless. Tree-based reasoning can navigate to the timeline section instead, because it understands the question’s intent in the context of the document’s structure.
Similarity finds content that looks like the answer. Reasoning finds content that is the answer. The distinction matters more than most benchmarks reveal.
The FinanceBench result, examined
VectifyAI’s Mafin 2.5, powered by PageIndex, achieved 98.7% accuracy on FinanceBench, a benchmark of financial questions answered from SEC filings. For context: ChatGPT-4o with search scores roughly 31% and Perplexity about 45%, both measured on only part of the dataset. Standard vector RAG pipelines land somewhere between 20% and 50%, depending on how retrieval is configured. The gap is not marginal.
Two things make this result credible. First, Mafin 2.5 was evaluated on 100% of the dataset, while some competitors test on only 66.7%. Second, the architectural explanation is sound: financial documents are deeply hierarchical, full of cross-references and structured tables that fixed-size chunking destroys. PageIndex preserves exactly the structure that vector pipelines discard.
But one thing makes this result insufficient as evidence for a general paradigm shift: FinanceBench tests single-document question answering. Each question targets one specific report. This is precisely the scenario PageIndex was designed for.
The scalability wall
This is where the honest analysis begins.
Alden Do Rosario, founder of CustomGPT.ai, ran PageIndex against a multi-document benchmark: 100 questions from Google’s SimpleQA-Verified dataset, across 2,795 documents. The tree indexing could not scale. His team had to fall back to FAISS vector search — the very approach PageIndex claims to replace.
VectifyAI has been transparent about this. In January, their team acknowledged on X that PageIndex was designed for single long document question answering, and that for more than five documents, they supported retrieval through other customized techniques. The open-source version used sequential indexing that they described as a proof of concept, not an enterprise-ready system.
Both limits have moved since. In May 2026, VectifyAI launched the PageIndex File System, which claims that a single index can reason over millions of documents. In August, PageIndex Flash replaced the sequential indexer as the default. No accuracy or latency figures at that scale have been published yet.
A proof of concept that achieves 98.7% on its target benchmark is impressive engineering. Presenting it as a general retrieval paradigm is a different claim entirely — and one the evidence does not yet support.
My criticism is aimed at how the result has been received, not at the project. The community’s eagerness to declare vector search dead reveals more about fatigue with the current paradigm than about the maturity of alternatives.
The metrics that are missing
Tao An made an observation that deserves more attention than it has received: after examining PageIndex’s GitHub, documentation, and every public source available, there is zero published data on latency, throughput, or cost per query.
This is not a minor documentation gap. The architecture requires multiple sequential LLM inference calls per retrieval. The model must load the tree structure, reason about which branch to follow, descend, evaluate, and potentially repeat. These calls cannot be parallelised. There is no apparent caching mechanism.
Publishing 98.7% accuracy without corresponding efficiency data is a benchmark result, not an engineering evaluation. The difference matters for anyone considering production deployment.
For a system that explicitly trades efficiency for accuracy (a PageIndex co-author wrote on Hacker News that on large trees it “may be slower than the vector-based method since it prioritizes accuracy”), the absence of efficiency quantification makes informed architectural comparison impossible. At what cost does that accuracy come? Ten times the latency of vector search? A hundred times? We do not know, because nobody has published the numbers.
Cost, at least, has figures now. VectifyAI puts indexing at about $0.001 per page, and its open benchmark reports $0.003 to $0.08 per question over 62 lookup questions, depending on the model. A cost comparison with vector search is still missing.
In research, we have a term for results presented without the methodology to evaluate their tradeoffs. We call them incomplete.
Where this sits in the landscape
PageIndex is not the first system to use tree structures for retrieval. RAPTOR (Sarthi et al., ICLR 2024) builds hierarchical trees through recursive clustering and summarisation, but it builds them bottom-up from embeddings rather than top-down from document structure. Microsoft’s GraphRAG constructs knowledge graphs through entity extraction and community detection. That is powerful for multi-hop reasoning, but the construction costs scale painfully.
What makes PageIndex architecturally distinct is the top-down approach: deriving the tree from the document’s own hierarchy rather than constructing one from embeddings or entity relationships. This is elegant for well-structured documents. It is also the source of the scalability constraint: you need a document with structure to preserve.
One Hacker News commenter put it well: “a function called setup() might mask some really important thing … your input data structure could build bad summaries that the LLM misses with.” Poorly structured documents produce poor trees. Structural fidelity, the system’s greatest strength, becomes a liability when the structure is absent or misleading.
The real contribution
Strip away the “RAG killer” rhetoric and PageIndex makes a genuine, specific contribution: it demonstrates that LLM reasoning over document structure can substantially outperform similarity search for within-document retrieval on structured professional documents.
That is a narrower claim than “vector databases are obsolete.” It is also a more defensible one.
The practical future, as Do Rosario suggests, likely involves hybrid architectures: vector retrieval for document discovery across large corpora, tree-based reasoning for precise extraction within the top candidates. The idea is hardly radical: it is how human researchers work. You search the library catalogue to find the book, then you use the table of contents to find the chapter.
Darbit has argued that most RAG fails at the R, not the G. PageIndex is evidence for that thesis. It does not solve retrieval universally, but it shows how much accuracy is left on the table when retrieval ignores document structure. The 98.7% result is not proof that tree reasoning is the future of RAG. It is proof that chunking-and-embedding is leaving significant performance on the floor for an important class of documents.
That should be enough to take seriously, without pretending it is more than it is.
The measure of a research contribution is not whether it replaces everything that came before. It is whether it changes how we think about the problem. PageIndex does that. The field should engage with the idea, not the hype.
Designing a retrieval architecture that fits your documents, your queries, and your constraints rather than a benchmark? Interlusion builds RAG systems grounded in engineering tradeoffs, not hype cycles. Let’s talk.