Core Objects
SourceSpan
The smallest addressable source unit. Stores source kind, text, page range, optional section and form metadata, a stable hash, and optional bounding boxes.
DocumentSourceNode
The canonical retrieval and hierarchy unit. Groups spans into typed nodes: document, page_group, form, endorsement, section, schedule, clause, table, row, cell, and text.
PolicyOperationalProfile
A source-backed projection of product-critical facts — limits, retentions, lines of business, and effective dates — each tied to a quoted span.
SourceChunk
A compatibility retrieval window used for vector search and RAG pipelines. Produced by chunking spans with configurable overlap.
SourceStore persists spans and chunks and implements the SourceRetriever interface used by agents and pipelines.
Minimal Setup
The example below shows the complete path from raw page text to a grounded extraction result.result that carries a quoted value traces back to a span ID in sourceSpans. You can pass those IDs to sourceStore.getSourceSpan() to retrieve the original text at any point downstream.
Architecture
Design Rules
Following these rules keeps your evidence layer trustworthy across every workflow that reads from it.1
Keep spans stable
Span IDs should change only when the underlying source text changes. Stable IDs let downstream workflows cache evidence lookups and detect genuine document changes.
2
Keep source nodes parser-grounded
LLM organization may label or group existing node IDs, but must not invent text, pages, spans, or bounding boxes. Every node must trace to real parsed content.
3
Prefer title-derived section hierarchy
Build your section tree from document headings, not page-by-page outlines. Title-derived hierarchy produces more meaningful retrieval and better coverage gap detection.
4
Keep quote text verifiable
Quote text should be short enough to verify quickly, but long enough to uniquely identify the policy language. Aim for one to three sentences per span.
5
Treat operational profiles as projections
PolicyOperationalProfile values are materialized views of what the source nodes and spans say. The source is canonical — the profile is derived.What Connects Here
Source Spans
Learn how to build and structure SourceSpan objects from PDF pages, sections, tables, and free text.
Retrieval
Search spans and nodes using SourceRetriever with configurable retrieval modes.
Storage Overview
Persist spans, chunks, and documents using provider-agnostic storage interfaces.
API Reference
Full reference for source grounding builder functions and store factory.