GraphRAG: From Local to Global - A Graph RAG Approach to Query-Focused Summarization¶

User: "What does the policy doc say about remote work?"
  Vanilla RAG: Embed query -> top-k similar chunks -> LLM answers. WORKS.

User: "What are the five biggest themes across all 10,000 of our incident reports?"
  Vanilla RAG: Embed query -> top-k similar chunks -> LLM answers.
    Top-k chunks are about ONE theme (the most semantically similar one).
    The LLM never sees the other nine themes.
    Answer is incomplete, biased toward whichever theme has the most chunks.
  FAILS.

Vector RAG assumes: "the answer lives in a small handful of chunks"

But for queries like:
  - "Summarize the main findings of this 500-page report"
  - "What changed across all our quarterly earnings calls in 2023?"
  - "What are the recurring failure modes in this set of bug reports?"

...the answer is DISTRIBUTED across the entire corpus.

Top-k retrieval cannot retrieve "everything." Even k=100 misses 99%
of a 10,000-chunk corpus, and an LLM cannot fit 10,000 chunks in context.

Option A: Stuff everything into context (long-context LLM)
  - Costs scale linearly with corpus size
  - Lost-in-the-middle effects on 200K+ contexts
  - Doesn't scale past one book of content

Option B: Hierarchical summarization (RAPTOR-style)
  - Recursive map-reduce summarization works, but is corpus-shaped, not query-shaped
  - Still retrieves leaves, so global queries hit the same problem

Option C: Just map-reduce over all chunks for every query
  - Re-summarizing 10,000 chunks per query is prohibitive
  - Every user question pays the full corpus cost

We need an index that:
  1. Captures structure (which entities relate, which topics co-occur)
  2. Summarizes at multiple zoom levels (chunk -> community -> corpus)
  3. Lets queries efficiently route to the right zoom level
  4. Pays the expensive summarization cost ONCE at index time

                        INDEX TIME (one-time, expensive)
   +---------------------------------------------------------------+
   |                                                               |
   |  Source     Chunk      LLM extracts        Graph              |
   |  Docs   ->  text   ->  entities &      ->  nodes/edges        |
   |             splits     relationships        (with claims)     |
   |                                                               |
   |                            |                                  |
   |                            v                                  |
   |                                                               |
   |                    Leiden clustering                          |
   |                    (hierarchical communities)                 |
   |                                                               |
   |                            |                                  |
   |                            v                                  |
   |                                                               |
   |                    LLM summarizes each                        |
   |                    community at every level                   |
   |                    (root -> leaf hierarchy)                   |
   |                                                               |
   +---------------------------------------------------------------+

                        QUERY TIME (cheap, parallel)
   +---------------------------------------------------------------+
   |                                                               |
   |   User query                                                  |
   |       |                                                       |
   |       v                                                       |
   |   Pick community level (resolution)                           |
   |       |                                                       |
   |       v                                                       |
   |   MAP: each community summary -> partial answer + score       |
   |       |                                                       |
   |       v                                                       |
   |   REDUCE: combine top-scored partial answers -> final answer  |
   |                                                               |
   +---------------------------------------------------------------+

Standard chunking, but the chunk size is a real lever:

  Smaller chunks (300 tokens):
    + LLM extraction is more thorough (fewer entities missed)
    + Higher recall on entity references
    - More LLM calls, higher index cost

  Larger chunks (1200 tokens):
    + Fewer LLM calls
    + More context for relationship extraction
    - Misses some entities the LLM "skips over"

Paper finds: 600-token chunks with overlap is a reasonable default.

For each chunk, prompt the LLM to extract:

  Entities:
    (name, type, description)
    e.g. ("OpenAI", "ORGANIZATION", "AI research company...")

  Relationships:
    (source, target, description, strength)
    e.g. ("OpenAI", "Microsoft", "received $13B investment", 9)

  Claims (optional):
    (subject, object, type, status, period, description)
    e.g. ("OpenAI", "Microsoft", "INVESTMENT", "TRUE", "2023", "...")

Multi-pass "gleaning":
  After first extraction, ask LLM "did you miss any?"
  Repeat 1-2 times to catch entities buried in subordinate clauses.
  Empirically: 1 gleaning pass adds ~10-15% recall, 2nd pass diminishing returns.

Across all chunks:
  - Same entity may appear with slight name variations
    ("OpenAI", "Open AI", "OpenAI Inc.")
  - LLM-generated descriptions need to be merged
  - Edge weights = number of chunks where the relationship appears

Element summarization:
  For each entity (and each edge) with multiple descriptions,
  ask LLM to write ONE consolidated description.

Result: a single weighted graph where nodes = entities,
edges = relationships, with rich text descriptions on both.

Leiden algorithm (Traag et al. 2019) is the modern successor to Louvain:
  - Greedy modularity optimization
  - Guarantees well-connected communities
  - Hierarchical: produces communities at multiple resolutions

GraphRAG runs Leiden hierarchically:

  Level 0 (root):    [whole graph]
                          |
  Level 1:        [C1]   [C2]   [C3]
                   |      |      |
  Level 2:    [C1a][C1b] [C2a] [C3a][C3b][C3c]
                                          |
  Level 3:                            [C3c-i][C3c-ii]

At each level, every node belongs to exactly one community.
Each community at level N is a subset of some community at level N-1.

Why Leiden specifically:
  - Modularity-based (clusters dense subgraphs)
  - Guarantees connectedness (Louvain can produce disconnected communities)
  - Fast on million-node graphs
  - Hierarchical resolution comes for free

For each community at every level:

  Inputs:
    - All entity descriptions in the community
    - All relationship descriptions inside (and crossing into) the community
    - Optionally, all claims about those entities

  Prompt the LLM:
    "Given these entities and relationships, write a report-style
     summary of this community: what it's about, key entities,
     key findings, and how it relates to neighbors."

  Output:
    A structured report (title + summary + findings) per community.

Storage:
  Community summaries are the actual queryable index.
  They're the unit of retrieval at query time.

Cost shape:
  Index time = O(chunks * extraction_cost + communities * summary_cost)
    For a 1M-token corpus, expect ~$5-50 in LLM costs (GPT-4-class models).
  Query time = O(communities_at_level * map_cost + 1 * reduce_cost)
    Typically 10-200 LLM calls per query, runnable in parallel.

User asks: "What are the dominant themes in this dataset?"

Step 6a: Pick a community level.
  Level 0: too coarse (one answer about the whole graph)
  Level 1-2: typical sweet spot for global queries
  Higher levels: more detail, more LLM calls

Step 6b: MAP (parallel)
  For each community summary at the chosen level:
    Prompt: "Given this community summary, answer the user's query.
             Also rate how helpful this community is for the answer (0-100)."
    Output: (partial_answer, helpfulness_score)

Step 6c: REDUCE
  Filter out partial answers with score = 0.
  Sort by helpfulness, take top-N until token budget is hit.
  Prompt: "Combine these partial answers into a single coherent response."
  Output: final answer.

VANILLA RAG
   query -> embed -> top-k chunks -> LLM -> answer
   Good at: factoid lookup, "what does the doc say about X"
   Bad at:  global synthesis, "what are the main themes"

GRAPH RAG (Global)
   query -> community level -> MAP over all summaries at that level
                                        |
                                        v
                            (partial_answer, score) per community
                                        |
                                        v
                            REDUCE top-scored partials -> answer
   Good at: holistic queries, multi-doc synthesis, "themes/trends"
   Cost:    higher per query (many LLM calls), but parallelizable

GRAPH RAG (Local)
   query -> identify relevant entities -> expand to neighborhood
        -> gather attached chunks + relationships + claims
        -> LLM -> answer
   Good at: entity-centric questions, "tell me about X and its connections"

Pre-LLM era: knowledge graphs required NER models, relation extractors,
ontology design, schema engineering, and human curation. Months of work.

GraphRAG: a single prompt to a general LLM extracts a usable graph
from arbitrary text. No schema, no labeled training data.

Trade-off: the graph is "fuzzy" (entity descriptions are free text,
not typed properties), but it's good enough for retrieval.

Vector RAG retrieves chunks. GraphRAG retrieves community summaries.

A community summary captures:
  - What entities cluster together
  - How they relate
  - The "story" of that subgraph

This is exactly the right granularity for global queries:
not too small (chunks miss the forest), not too big (the corpus
is too much for one prompt).

Same query, different community levels:

  Level 1 (10 communities):  ~10 map calls, broad strokes answer
  Level 2 (50 communities):  ~50 map calls, more nuance
  Level 3 (200 communities): ~200 map calls, fine-grained but expensive

The user (or system) picks the resolution per query.
Vanilla RAG has no equivalent dial.

Naive map-reduce over a corpus is wasteful: most chunks aren't relevant.

GraphRAG's twist: every map step ALSO returns a 0-100 helpfulness score.
The reduce step ignores zeros and prioritizes high-scoring partials.

This makes the reduce step linear in actual signal, not corpus size.

The expensive work happens ONCE at index time:
  - Entity extraction (per chunk)
  - Element summarization (per node/edge)
  - Community summarization (per community at every level)

Query time only does:
  - Map across community summaries (cheap, parallel)
  - One reduce step

Two real-world corpora:
  - Podcast transcripts (Behind the Tech with Kevin Scott): ~1M tokens
  - News articles dataset: ~1.7M tokens

Both too large for context-stuffing, both with rich global structure.

"Sensemaking" questions generated by an LLM persona pipeline:
  - 125 questions per dataset
  - Designed to require global synthesis (not factoid lookup)
  - Examples: "What are the recurring criticisms of...?",
              "How has the narrative around X evolved...?"

Head-to-head LLM-as-judge comparisons across these axes:
  - Comprehensiveness (covers all relevant points)
  - Diversity (multiple perspectives)
  - Empowerment (helps the reader form their own view)
  - Directness (specific to the question, no waffle)

Read this as: GraphRAG produces more thorough, multi-faceted answers
on global questions, at the cost of being less terse than naive RAG.

For factoid questions, naive RAG is still the right tool.
GraphRAG wins specifically on the queries it was designed for.

Index cost (one-time):
  ~600K-1M tokens of LLM calls per 1M tokens of corpus
  Dominated by entity extraction; community summarization is smaller

Query cost:
  Naive RAG:    ~5-10K tokens per query
  GraphRAG L2:  ~30-100K tokens per query (parallel map calls)
  GraphRAG L1:  ~10-30K tokens per query

Conclusion: GraphRAG is 3-10x more expensive per query than naive RAG,
but produces qualitatively different answers on global questions.

1. Enterprise knowledge bases
   "What policies have changed across our employee handbook revisions?"

2. Legal discovery
   "What are the patterns of communication between these custodians?"

3. Scientific literature review
   "What competing hypotheses exist on topic X across these 500 papers?"

4. Investigative journalism / OSINT
   "Who connects to whom across these leaked documents?"

5. Customer support analytics
   "What are the recurring failure modes in support tickets this quarter?"

6. Compliance / audit
   "What risk themes appear across all incident reports in 2024?"

1. Factoid lookup ("What's the address in section 3?") - vanilla RAG wins
2. Real-time / fresh data - the index is expensive to rebuild
3. Small corpora - just stuff it into context, don't bother
4. Highly structured data (SQL tables, knowledge graphs) - use those directly
5. Cost-sensitive consumer apps - the per-query cost adds up

Microsoft uses GraphRAG (and variants) inside:
  - Azure AI Search "agentic retrieval" pipelines
  - Microsoft Research's internal sensemaking tools
  - Customer-facing offerings that need long-document QA

The open-source repo (microsoft/graphrag) is the reference implementation.
LazyGraphRAG (a 2024 follow-up from the same team) defers community
summarization to query time to cut index cost dramatically.

+-----------------------+--------------------+--------------------+
| Dimension             | Vector RAG         | GraphRAG           |
+-----------------------+--------------------+--------------------+
| Index unit            | Chunk + embedding  | Entity graph +     |
|                       |                    | community summary  |
| Index cost            | Low (embed only)   | High (LLM-heavy)   |
| Query cost            | Low (1 LLM call)   | Med-High (N calls) |
| Best for              | Local/factoid      | Global/holistic    |
| Worst for             | "Main themes" Qs   | "Quote section X"  |
| Freshness             | Easy (re-embed)    | Hard (re-extract)  |
| Explainability        | Cited chunks       | Cited communities  |
| Hallucination control | Source attribution | Source attribution |
| Multi-hop reasoning   | Weak               | Strong             |
+-----------------------+--------------------+--------------------+

The right architecture is usually BOTH:
  - Route factoid questions to vector RAG (cheap, fast)
  - Route holistic questions to GraphRAG (thorough)
  - A small classifier or LLM-as-router decides per query

Storage backends:
  - Local files (parquet)
  - Azure Blob Storage
  - CosmosDB

LLM backends:
  - Azure OpenAI
  - OpenAI
  - Any OpenAI-compatible endpoint (vLLM, Together, etc.)

The library has been forked dozens of times. Notable derivatives:
  - nano-graphrag: minimal Python implementation, ~1k LOC
  - LightRAG: cheaper variant with single-level communities
  - HippoRAG: graph + personalized PageRank for retrieval
  - KAG: knowledge-graph-augmented RAG for domain reasoning

Indexing a 10M-token corpus with GPT-4 can run $500-5000 in LLM costs.
For corpora that change weekly, this is a serious operational expense.
LazyGraphRAG and incremental indexing variants exist but are still maturing.

The entire downstream pipeline rests on entity/relationship extraction.
Domain-specific corpora (legal, medical) need prompt-tuning to extract
the right entity types, otherwise the graph is shallow.

LLM-written summaries can:
  - Miss long-tail entities
  - Hallucinate relationships under pressure
  - Lose grounding when communities are too large

Mitigations: cite back to source chunks, restrict to extracted facts.

Even map-reduce over communities is a sample of the corpus, not
the full thing. Pathological queries that require integrating
100% of the data still struggle.

LLM-as-judge on "comprehensiveness" is itself imperfect.
Real human evaluation on global queries is expensive and rare.
Leaderboard numbers should be read as directional, not precise.

Adding 1% new documents shouldn't require a 100% reindex,
but maintaining graph + community structure under streaming updates
is an active research problem.