HTTP API

The FastAPI service (spec_v2 §4): the app factory, the wire contract, SSE framing, dependencies, and the route modules. The HTTP-level contract (paths, bodies, event protocol) is documented as-built in the OpenAPI schema served at /openapi.json; these are the Python internals.

varagity.api

The HTTP API (spec_v2 §4.1): FastAPI over the same flows the CLI runs.

The API lives inside the package so it imports the pipeline directly — no network hop, and the "app is a client of every backing service" invariant holds with a second front door. async at the edge, sync flows underneath: endpoints run the synchronous pipeline in a threadpool while the event loop streams SSE frames out.

Import :func:create_app lazily from :mod:varagity.api.main (uvicorn's --factory target) — importing it pulls FastAPI and the whole pipeline.

varagity.api.main

FastAPI app factory: CORS, lifespan, structured errors, routers.

create_app is served by a single uvicorn worker (plan decision #11): uvicorn varagity.api.main:create_app --factory. The lifespan warms the registries, runs the idempotent migration runner (so an existing pgdata volume gains the v2 tables without down -v), and replays persisted runtime setting overrides (spec_v2 §4.7); every error response carries the {error: {code, message}} envelope the GUI banners on.

Importing this module imports :mod:varagity.pipeline (via the chat route), which exports PREFECT_API_URL before prefect loads — the same ordering invariant the CLI honors.

StructuredServerErrors

Render unhandled route exceptions as the structured 500 envelope.

Starlette's stack is ServerErrorMiddleware → user middleware (CORS) → ExceptionMiddleware → routes: an exception no handler catches propagates past the CORS middleware and becomes a bare text 500 without Access-Control-Allow-Origin — which a browser can only report as TypeError: Failed to fetch, hiding the real error (the exact failure mode of the first unwritable-./docs upload). This pure-ASGI middleware is registered inside CORS, so its enveloped 500 flows through the CORS send path and stays readable cross-origin — "errors are structured" (spec_v2 §4.1) holds for every failure.

A response whose headers already flushed (an SSE stream mid-flight) can't change status; those re-raise, and the stream protocols carry their own in-band error events instead.

Source code in varagity/api/main.py

class StructuredServerErrors:
    """Render unhandled route exceptions as the structured 500 envelope.

    Starlette's stack is ``ServerErrorMiddleware → user middleware (CORS)
    → ExceptionMiddleware → routes``: an exception no handler catches
    propagates *past* the CORS middleware and becomes a bare text 500
    **without** ``Access-Control-Allow-Origin`` — which a browser can only
    report as ``TypeError: Failed to fetch``, hiding the real error (the
    exact failure mode of the first unwritable-``./docs`` upload). This
    pure-ASGI middleware is registered *inside* CORS, so its enveloped 500
    flows through the CORS send path and stays readable cross-origin —
    "errors are structured" (spec_v2 §4.1) holds for every failure.

    A response whose headers already flushed (an SSE stream mid-flight)
    can't change status; those re-raise, and the stream protocols carry
    their own in-band ``error`` events instead.
    """

    def __init__(self, app: Callable[..., Any]) -> None:
        """Wrap the downstream ASGI app.

        Args:
            app: The next ASGI callable in the stack.
        """
        self.app = app

    async def __call__(
        self, scope: dict[str, Any], receive: Callable[..., Any], send: Callable[..., Any]
    ) -> None:
        """Serve one connection, enveloping any unhandled exception.

        Args:
            scope: The ASGI connection scope.
            receive: The ASGI receive callable.
            send: The ASGI send callable.
        """
        if scope["type"] != "http":
            await self.app(scope, receive, send)
            return
        response_started = False

        async def tracking_send(message: dict[str, Any]) -> None:
            nonlocal response_started
            if message["type"] == "http.response.start":
                response_started = True
            await send(message)

        try:
            await self.app(scope, receive, tracking_send)
        except Exception as error:
            if response_started:
                raise
            logger.exception("unhandled error serving %s", scope.get("path", "?"))
            response = JSONResponse(
                status_code=500,
                content=_error_payload("internal_error", f"{type(error).__name__}: {error}"),
            )
            await response(scope, receive, send)  # type: ignore[arg-type]

__call__(scope, receive, send) async

Serve one connection, enveloping any unhandled exception.

Parameters:

Name	Type	Description	Default
scope	dict[str, Any]	The ASGI connection scope.	required
receive	Callable[..., Any]	The ASGI receive callable.	required
send	Callable[..., Any]	The ASGI send callable.	required

Source code in varagity/api/main.py

async def __call__(
    self, scope: dict[str, Any], receive: Callable[..., Any], send: Callable[..., Any]
) -> None:
    """Serve one connection, enveloping any unhandled exception.

    Args:
        scope: The ASGI connection scope.
        receive: The ASGI receive callable.
        send: The ASGI send callable.
    """
    if scope["type"] != "http":
        await self.app(scope, receive, send)
        return
    response_started = False

    async def tracking_send(message: dict[str, Any]) -> None:
        nonlocal response_started
        if message["type"] == "http.response.start":
            response_started = True
        await send(message)

    try:
        await self.app(scope, receive, tracking_send)
    except Exception as error:
        if response_started:
            raise
        logger.exception("unhandled error serving %s", scope.get("path", "?"))
        response = JSONResponse(
            status_code=500,
            content=_error_payload("internal_error", f"{type(error).__name__}: {error}"),
        )
        await response(scope, receive, send)  # type: ignore[arg-type]

__init__(app)

Wrap the downstream ASGI app.

Parameters:

Name	Type	Description	Default
app	Callable[..., Any]	The next ASGI callable in the stack.	required

Source code in varagity/api/main.py

def __init__(self, app: Callable[..., Any]) -> None:
    """Wrap the downstream ASGI app.

    Args:
        app: The next ASGI callable in the stack.
    """
    self.app = app

create_app()

Build the configured FastAPI application.

Returns:

Type	Description
FastAPI	The app, ready for uvicorn (--factory).

Source code in varagity/api/main.py

def create_app() -> FastAPI:
    """Build the configured FastAPI application.

    Returns:
        The app, ready for uvicorn (``--factory``).
    """
    settings = get_settings()
    setup_logging(settings.LOG_LEVEL)
    try:
        version = metadata.version("varagity")
    except metadata.PackageNotFoundError:  # running from a raw checkout
        version = "0.0.0"
    app = FastAPI(
        title="Varagity API",
        description="Contextual-Retrieval RAG over local GPUs (spec_v2 §4).",
        version=version,
        lifespan=lifespan,
    )
    # Order matters: add_middleware prepends (last added = outermost), so
    # registering the error enveloper *before* CORS seats it inside — its
    # 500s pass through the CORS send path and stay browser-readable.
    app.add_middleware(StructuredServerErrors)
    app.add_middleware(
        CORSMiddleware,
        allow_origins=settings.cors_origin_list,
        allow_credentials=True,
        allow_methods=["*"],
        allow_headers=["*"],
    )
    app.add_exception_handler(StarletteHTTPException, _handle_http_exception)
    app.add_exception_handler(RequestValidationError, _handle_validation_error)
    app.include_router(system.router)
    app.include_router(conversations.router)
    app.include_router(groups.router)
    app.include_router(chat.router)
    app.include_router(settings_routes.router)
    app.include_router(documents.router)
    app.include_router(ingest.router)
    app.include_router(graph.router)
    app.include_router(backups.router)
    if settings.METRICS_ENABLED:
        # The corpus and graph gauges are collected at scrape time, so they
        # only need to exist when something can scrape them (spec_v3 §6.1a;
        # spec_graphrag §6). Neither touches its store on registration.
        register_corpus_collector()
        register_graph_collector()
        app.include_router(metrics.router)
    _install_sse_event_schemas(app)
    return app

lifespan(app) async

Run startup work before the app serves: registries, then migrations.

Parameters:

Name	Type	Description	Default
app	FastAPI	The FastAPI instance being started.	required

Yields:

Type	Description
AsyncIterator[None]	Nothing — control for the app's serving lifetime.

Source code in varagity/api/main.py

@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
    """Run startup work before the app serves: registries, then migrations.

    Args:
        app: The FastAPI instance being started.

    Yields:
        Nothing — control for the app's serving lifetime.
    """
    _warm_registries()
    await run_in_threadpool(_run_startup_migrations)
    # After migrations (the app_settings table must exist) and before the
    # first request: overrides survive an api restart (spec_v2 §4.7).
    await run_in_threadpool(_load_runtime_overrides)
    try:
        yield
    finally:
        await run_in_threadpool(_close_graph_session)

varagity.api.schemas

The HTTP wire contract — pydantic request/response models (spec_v2 §4.1).

Every request and response body is a model here; FastAPI renders them into the OpenAPI schema the web app's TypeScript types are generated from (openapi-typescript), so the contract cannot drift by hand-editing. SSE event payloads (spec_v2 §4.3) are models too — one per event type — because they cross the same wire.

BackupLogEvent

Bases: BaseModel

Payload of the backup-status SSE log event.

Relayed varagity.backup log records (what was exported or restored, a swept partial, an embedding-model drift warning, a graph workdir entry the export had to skip, a graph restore undone because the restored graph would not open), so the panel shows what the server log would.

Attributes:

Name	Type	Description
level	str	The log level name (INFO | WARNING | ERROR).
message	str	The formatted log message.

Source code in varagity/api/schemas.py

class BackupLogEvent(BaseModel):
    """Payload of the backup-status SSE ``log`` event.

    Relayed ``varagity.backup`` log records (what was exported or restored,
    a swept partial, an embedding-model drift warning, a graph workdir entry
    the export had to skip, a graph restore undone because the restored
    graph would not open), so the panel shows what the server log would.

    Attributes:
        level: The log level name (``INFO`` | ``WARNING`` | ``ERROR``).
        message: The formatted log message.
    """

    level: str
    message: str

BackupProgressEvent

Bases: BaseModel

Payload of the backup-status SSE progress event.

A RAG export walks snapshot (open one consistent read of the corpus) → documents → chunks (rows written, with current/total) → finalize (the vector matrix, the manifest, the atomic rename). A graph export walks snapshot (take the graph's write lock, list its workdir) → pack (the workdir's files copied in, counted in bytes) → finalize. A RAG restore walks validate (the bundle's verdict again) → load (the replacing transaction's binary COPY, chunks loaded so far) → index (the vector index rebuild and the commit — the restore takes effect) → bm25 (the BM25 index refilled from the restored rows, chunks so far). A graph restore walks validate → extract (the bundle's files into a staging folder beside the workdir, counted in bytes) → swap (the session closed, the folders renamed) → open (the restored graph opened and checked).

Attributes:

Name	Type	Description
stage	Literal['snapshot', 'documents', 'chunks', 'pack', 'finalize', 'validate', 'load', 'index', 'bm25', 'extract', 'swap', 'open']	snapshot | documents | chunks | finalize (a RAG export), snapshot | pack | finalize (a graph export), validate | load | index | bm25 (a RAG restore), or validate | extract | swap | open (a graph restore).
current	int | None	Rows (pack, extract: bytes) done so far in a counted stage.
total	int | None	The stage's row (pack, extract: byte) count.

Source code in varagity/api/schemas.py

class BackupProgressEvent(BaseModel):
    """Payload of the backup-status SSE ``progress`` event.

    A RAG export walks ``snapshot`` (open one consistent read of the corpus)
    → ``documents`` → ``chunks`` (rows written, with ``current``/``total``) →
    ``finalize`` (the vector matrix, the manifest, the atomic rename). A
    graph export walks ``snapshot`` (take the graph's write lock, list its
    workdir) → ``pack`` (the workdir's files copied in, counted in **bytes**)
    → ``finalize``. A RAG restore walks ``validate`` (the bundle's verdict
    again) → ``load`` (the replacing transaction's binary COPY, chunks
    loaded so far) → ``index`` (the vector index rebuild and the commit —
    the restore takes effect) → ``bm25`` (the BM25 index refilled from the
    restored rows, chunks so far). A graph restore walks ``validate`` →
    ``extract`` (the bundle's files into a staging folder beside the
    workdir, counted in **bytes**) → ``swap`` (the session closed, the
    folders renamed) → ``open`` (the restored graph opened and checked).

    Attributes:
        stage: ``snapshot`` | ``documents`` | ``chunks`` | ``finalize`` (a
            RAG export), ``snapshot`` | ``pack`` | ``finalize`` (a graph
            export), ``validate`` | ``load`` | ``index`` | ``bm25`` (a RAG
            restore), or ``validate`` | ``extract`` | ``swap`` | ``open`` (a
            graph restore).
        current: Rows (``pack``, ``extract``: bytes) done so far in a counted
            stage.
        total: The stage's row (``pack``, ``extract``: byte) count.
    """

    stage: Literal[
        "snapshot",
        "documents",
        "chunks",
        "pack",
        "finalize",
        "validate",
        "load",
        "index",
        "bm25",
        "extract",
        "swap",
        "open",
    ]
    current: int | None = None
    total: int | None = None

BackupRestoreRequest

Bases: BaseModel

Body of POST /api/backups/{corpus}/restore (ADR-019).

Attributes:

Name	Type	Description
name	str	The bundle to restore — a file name in the corpus's varagity-exports/ folder, as the backups list reports it.

Source code in varagity/api/schemas.py

class BackupRestoreRequest(BaseModel):
    """Body of ``POST /api/backups/{corpus}/restore`` (ADR-019).

    Attributes:
        name: The bundle to restore — a file name in the corpus's
            ``varagity-exports/`` folder, as the backups list reports it.
    """

    model_config = ConfigDict(extra="forbid")

    name: str = Field(min_length=1)

BackupRunOut

Bases: BaseModel

One backup run's state (POST /api/backups/{corpus}/export or /restore).

Attributes:

Name	Type	Description
run_id	str	The run handle.
corpus	Literal['rag', 'graph']	The corpus being backed up or restored.
operation	Literal['export', 'restore']	What the run does — export writes a new bundle, restore replaces the corpus with one.
state	Literal['running', 'completed', 'failed']	running | completed | failed.
bundle	str | None	An export's: the file it wrote (set once it is published — an export names nothing until its bundle is whole). A restore's: the bundle it restores, from the start.
saved_to	str | None	Where an export's bundle landed, relative to the checkout (docs/varagity-exports/<name>, or graph-docs/varagity-exports/<name> for the graph) — the path "Copy path" copies, right in both host and compose mode. None for a restore.
started_at	datetime	When the run started.
finished_at	datetime | None	When it reached a terminal state (None while running).
error	str | None	The run-level failure (failed only).
summary	BackupSummaryOut | None	The final counters (completed only).

Source code in varagity/api/schemas.py

class BackupRunOut(BaseModel):
    """One backup run's state (``POST /api/backups/{corpus}/export`` or ``/restore``).

    Attributes:
        run_id: The run handle.
        corpus: The corpus being backed up or restored.
        operation: What the run does — ``export`` writes a new bundle,
            ``restore`` replaces the corpus with one.
        state: ``running`` | ``completed`` | ``failed``.
        bundle: An export's: the file it wrote (set once it is published —
            an export names nothing until its bundle is whole). A restore's:
            the bundle it restores, from the start.
        saved_to: Where an export's bundle landed, relative to the checkout
            (``docs/varagity-exports/<name>``, or
            ``graph-docs/varagity-exports/<name>`` for the graph) — the path
            "Copy path" copies, right in both host and compose mode.
            ``None`` for a restore.
        started_at: When the run started.
        finished_at: When it reached a terminal state (``None`` while
            running).
        error: The run-level failure (``failed`` only).
        summary: The final counters (``completed`` only).
    """

    run_id: str
    corpus: Literal["rag", "graph"]
    operation: Literal["export", "restore"]
    state: Literal["running", "completed", "failed"]
    bundle: str | None = None
    saved_to: str | None = None
    started_at: datetime
    finished_at: datetime | None = None
    error: str | None = None
    summary: BackupSummaryOut | None = None

BackupStatusEvent

Bases: BaseModel

Payload of the backup-status SSE status event.

The stream's first frame (a snapshot on connect) and its last (the terminal state). run=None means no backup run has run for this corpus in this API process — the stream closes immediately after.

Attributes:

Name	Type	Description
run	BackupRunOut | None	The current (or last) run, if any.

Source code in varagity/api/schemas.py

class BackupStatusEvent(BaseModel):
    """Payload of the backup-status SSE ``status`` event.

    The stream's first frame (a snapshot on connect) and its last (the
    terminal state). ``run=None`` means no backup run has run for this
    corpus in this API process — the stream closes immediately after.

    Attributes:
        run: The current (or last) run, if any.
    """

    run: BackupRunOut | None = None

BackupSummaryOut

Bases: BaseModel

What a completed backup run did (terminal runs only).

Attributes:

Name	Type	Description
counts	dict[str, int]	An export's: what the bundle holds — documents and chunks for a RAG bundle; thread_days, messages and (when the graph reported them) entities and relations for a graph bundle. A RAG restore's: what it wrote — documents, chunks, and bm25 (the chunks the rebuilt index holds). A graph restore's: what the restored graph holds, by its bundle's counts.
size_bytes	int	The bundle's size on disk.
wall_clock_s	float	Seconds from the snapshot to the published bundle (an export), from validation to a searchable index (a RAG restore), or from validation to an open graph (a graph restore).
stale	bool | None	A restore's: whether it left the corpus flagged stale — for RAG, the bundle was stale at export or its rows were made under other ingest settings; for the graph, the bundle was stale at export. None for an export.

Source code in varagity/api/schemas.py

class BackupSummaryOut(BaseModel):
    """What a completed backup run did (terminal runs only).

    Attributes:
        counts: An export's: what the bundle holds — ``documents`` and
            ``chunks`` for a RAG bundle; ``thread_days``, ``messages`` and
            (when the graph reported them) ``entities`` and ``relations``
            for a graph bundle. A RAG restore's: what it wrote —
            ``documents``, ``chunks``, and ``bm25`` (the chunks the rebuilt
            index holds). A graph restore's: what the restored graph holds,
            by its bundle's counts.
        size_bytes: The bundle's size on disk.
        wall_clock_s: Seconds from the snapshot to the published bundle (an
            export), from validation to a searchable index (a RAG restore),
            or from validation to an open graph (a graph restore).
        stale: A restore's: whether it left the corpus flagged stale — for
            RAG, the bundle was stale at export or its rows were made under
            other ingest settings; for the graph, the bundle was stale at
            export. ``None`` for an export.
    """

    counts: dict[str, int]
    size_bytes: int
    wall_clock_s: float
    stale: bool | None = None

BundleManifestOut

Bases: BundleManifest

A backup bundle's bundle.json, as the backups list reports it (ADR-019).

The file format's own model (:class:varagity.backup.bundle.BundleManifest) on the wire unchanged — the panel shows what a bundle holds (counts) and what it was made under (the embedding model and dimension, a RAG bundle's ingest settings, a graph bundle's engine format, the stale flag) straight from the manifest.

Source code in varagity/api/schemas.py

class BundleManifestOut(BundleManifest):
    """A backup bundle's ``bundle.json``, as the backups list reports it (ADR-019).

    The file format's own model (:class:`varagity.backup.bundle.BundleManifest`)
    on the wire unchanged — the panel shows what a bundle holds (``counts``)
    and what it was made under (the embedding model and dimension, a RAG
    bundle's ingest settings, a graph bundle's engine format, the stale
    flag) straight from the manifest.
    """

BundleOut

Bases: BaseModel

One bundle file in GET /api/backups/{corpus} (ADR-019).

The verdict fields are the cheap check — the zip's directory and its manifest, never a hash (a restore hashes every member as it loads it, before anything takes effect). A bundle damaged inside an intact zip therefore lists as restorable until a restore fails on it; from then on, until its size or mtime changes, it lists as damaged (and a restore of it is refused up front).

Attributes:

Name	Type	Description
name	str	The bundle's file name in the corpus's varagity-exports/ folder — the key the download and restore routes take.
size_bytes	int	Its size on disk.
modified_at	datetime	Its mtime (UTC).
manifest	BundleManifestOut | None	Its bundle.json, or None when the file has none or it is unreadable — a zip copied into the folder by hand is listed, never hidden.
restorable	bool	Whether nothing the cheap check sees refuses a restore.
reason	str | None	Why the bundle can't be restored (None when it can): an unreadable manifest, another format version or corpus, another embedding model or dimension, a graph bundle another engine (or another major.minor version of this one) wrote, an unsafe or unexpected member, a member whose recorded size differs from the zip's, or a damage a restore already found.
drift	list[str]	The reingest-affecting settings the bundle's rows were made under that differ from this stack's — restoring it flags the corpus stale. Empty when they match (or there is no manifest).

Source code in varagity/api/schemas.py

class BundleOut(BaseModel):
    """One bundle file in ``GET /api/backups/{corpus}`` (ADR-019).

    The verdict fields are the *cheap* check — the zip's directory and its
    manifest, never a hash (a restore hashes every member as it loads it,
    before anything takes effect). A bundle damaged inside an intact zip
    therefore lists as restorable until a restore fails on it; from then
    on, until its size or mtime changes, it lists as damaged (and a
    restore of it is refused up front).

    Attributes:
        name: The bundle's file name in the corpus's ``varagity-exports/``
            folder — the key the download and restore routes take.
        size_bytes: Its size on disk.
        modified_at: Its mtime (UTC).
        manifest: Its ``bundle.json``, or ``None`` when the file has none or
            it is unreadable — a zip copied into the folder by hand is
            listed, never hidden.
        restorable: Whether nothing the cheap check sees refuses a restore.
        reason: Why the bundle can't be restored (``None`` when it can): an
            unreadable manifest, another format version or corpus, another
            embedding model or dimension, a graph bundle another engine (or
            another major.minor version of this one) wrote, an unsafe or
            unexpected member, a member whose recorded size differs from the
            zip's, or a damage a restore already found.
        drift: The reingest-affecting settings the bundle's rows were made
            under that differ from this stack's — restoring it flags the
            corpus stale. Empty when they match (or there is no manifest).
    """

    name: str
    size_bytes: int
    modified_at: datetime
    manifest: BundleManifestOut | None = None
    restorable: bool
    reason: str | None
    drift: list[str]

ChatOverrides

Bases: BaseModel

Per-request overrides of query-time settings (spec_v2 §4.2).

Unknown fields are rejected so a typo'd override fails loudly instead of silently running with defaults.

Attributes:

Name	Type	Description
retrieval_method	str | None	Registry name of the retrieval method for this question only (semantic | bm25 | hybrid | reranked | hyde).
top_k	int | None	Number of chunks retrieved for this question only.
chat_engine	str | None	Registry name of the chat engine for this question only (simple | condense_context — spec_v3 §4.2).

Source code in varagity/api/schemas.py

class ChatOverrides(BaseModel):
    """Per-request overrides of query-time settings (spec_v2 §4.2).

    Unknown fields are rejected so a typo'd override fails loudly instead
    of silently running with defaults.

    Attributes:
        retrieval_method: Registry name of the retrieval method for this
            question only (``semantic`` | ``bm25`` | ``hybrid`` |
            ``reranked`` | ``hyde``).
        top_k: Number of chunks retrieved for this question only.
        chat_engine: Registry name of the chat engine for this question
            only (``simple`` | ``condense_context`` — spec_v3 §4.2).
    """

    model_config = ConfigDict(extra="forbid")

    retrieval_method: str | None = None
    top_k: int | None = Field(default=None, ge=1)
    chat_engine: str | None = None

ChatRequest

Bases: BaseModel

Body of POST /api/chat (spec_v2 §4.2, spec_graphrag §4.2).

Attributes:

Name	Type	Description
query	str	The user's question.
conversation_id	str | None	Existing conversation to append the turn to; a new conversation is created when omitted.
overrides	ChatOverrides | None	Optional per-request setting overrides.
corpus	Literal['rag', 'graph'] | None	Which corpus to answer from — "rag" (the chunk corpus, the default when omitted) or "graph" (the message archive). The router-fillable field: automatic routing is out of scope (spec_graphrag §6), so today the composer sets it and later a router fills the same field, changing no contract.

Source code in varagity/api/schemas.py

class ChatRequest(BaseModel):
    """Body of ``POST /api/chat`` (spec_v2 §4.2, spec_graphrag §4.2).

    Attributes:
        query: The user's question.
        conversation_id: Existing conversation to append the turn to; a new
            conversation is created when omitted.
        overrides: Optional per-request setting overrides.
        corpus: Which corpus to answer from — ``"rag"`` (the chunk corpus,
            the default when omitted) or ``"graph"`` (the message archive).
            The **router-fillable** field: automatic routing is out of scope
            (spec_graphrag §6), so today the composer sets it and later a
            router fills the same field, changing no contract.
    """

    query: str = Field(min_length=1)
    conversation_id: str | None = None
    overrides: ChatOverrides | None = None
    corpus: Literal["rag", "graph"] | None = None

ConfigResponse

Bases: BaseModel

Response of GET /api/config — static capabilities (spec_v2 §4.2).

The GUI builds its settings controls from this, so a newly registered implementation (one file + one import) appears in the UI automatically.

Attributes:

Name	Type	Description
retrievers	list[str]	Registered retrieval method names.
chunkers	list[str]	Registered chunking strategy names.
ocr_engines	list[str]	Available OCR engine names.
model_types	list[str]	Valid get_model types.
llm_model_types	list[str]	The chat-capable subset of model_types — the CHAT_MODEL_TYPE vocabulary the composer quick-toggle offers.
ranges	dict[str, NumericRange]	Valid ranges for the numeric query-time knobs, keyed by setting name (lowercase).
upload_max_mb	int	Effective per-file upload cap (UPLOAD_MAX_MB) — the dropzone validates against it client-side.
allowed_extensions	list[str]	Effective ingestable extensions (ALLOWED_EXTENSIONS, normalized with leading dots), sorted.
preview_enabled	bool	Whether the page-preview endpoints are on (PREVIEW_ENABLED, read-only here — the knob is env-only) — off, the GUI skips preview eligibility entirely.

Source code in varagity/api/schemas.py

class ConfigResponse(BaseModel):
    """Response of ``GET /api/config`` — static capabilities (spec_v2 §4.2).

    The GUI builds its settings controls from this, so a newly registered
    implementation (one file + one import) appears in the UI automatically.

    Attributes:
        retrievers: Registered retrieval method names.
        chunkers: Registered chunking strategy names.
        ocr_engines: Available OCR engine names.
        model_types: Valid ``get_model`` types.
        llm_model_types: The chat-capable subset of ``model_types`` — the
            ``CHAT_MODEL_TYPE`` vocabulary the composer quick-toggle offers.
        ranges: Valid ranges for the numeric query-time knobs, keyed by
            setting name (lowercase).
        upload_max_mb: Effective per-file upload cap (``UPLOAD_MAX_MB``) —
            the dropzone validates against it client-side.
        allowed_extensions: Effective ingestable extensions
            (``ALLOWED_EXTENSIONS``, normalized with leading dots), sorted.
        preview_enabled: Whether the page-preview endpoints are on
            (``PREVIEW_ENABLED``, read-only here — the knob is env-only) —
            off, the GUI skips preview eligibility entirely.
    """

    retrievers: list[str]
    chunkers: list[str]
    ocr_engines: list[str]
    model_types: list[str]
    llm_model_types: list[str]
    ranges: dict[str, NumericRange]
    upload_max_mb: int
    allowed_extensions: list[str]
    preview_enabled: bool

ConversationCreateRequest

Bases: BaseModel

Body of POST /api/conversations.

Attributes:

Name	Type	Description
title	str | None	Optional explicit title; defaults to the placeholder the first chat turn auto-replaces.

Source code in varagity/api/schemas.py

class ConversationCreateRequest(BaseModel):
    """Body of ``POST /api/conversations``.

    Attributes:
        title: Optional explicit title; defaults to the placeholder the
            first chat turn auto-replaces.
    """

    title: str | None = Field(default=None, min_length=1, max_length=200)

ConversationDetailOut

Bases: BaseModel

Response of GET /api/conversations/{id} — the full transcript.

Attributes:

Name	Type	Description
conversation_id	str	The app-generated id.
title	str	Current title.
created_at	datetime	Creation timestamp.
updated_at	datetime	Last-turn timestamp.
messages	list[MessageOut]	All messages, oldest first, each with its sources.

Source code in varagity/api/schemas.py

class ConversationDetailOut(BaseModel):
    """Response of ``GET /api/conversations/{id}`` — the full transcript.

    Attributes:
        conversation_id: The app-generated id.
        title: Current title.
        created_at: Creation timestamp.
        updated_at: Last-turn timestamp.
        messages: All messages, oldest first, each with its sources.
    """

    conversation_id: str
    title: str
    created_at: datetime
    updated_at: datetime
    messages: list[MessageOut]

ConversationSummaryOut

Bases: BaseModel

One conversation in the GET /api/conversations list.

Attributes:

Name	Type	Description
conversation_id	str	The app-generated id.
title	str	Current title.
created_at	datetime	Creation timestamp.
updated_at	datetime	Last-turn timestamp (list ordering key).
message_count	int	Number of persisted messages.
group_id	str | None	Sidebar group the conversation is filed under (None = ungrouped).

Source code in varagity/api/schemas.py

class ConversationSummaryOut(BaseModel):
    """One conversation in the ``GET /api/conversations`` list.

    Attributes:
        conversation_id: The app-generated id.
        title: Current title.
        created_at: Creation timestamp.
        updated_at: Last-turn timestamp (list ordering key).
        message_count: Number of persisted messages.
        group_id: Sidebar group the conversation is filed under (``None``
            = ungrouped).
    """

    conversation_id: str
    title: str
    created_at: datetime
    updated_at: datetime
    message_count: int
    group_id: str | None = None

ConversationUpdateRequest

Bases: BaseModel

Body of PATCH /api/conversations/{conversation_id}.

Unknown fields are rejected so a typo'd patch fails loudly instead of silently doing nothing.

Attributes:

Name	Type	Description
group_id	str | None	Sidebar group to file the conversation under, or None to ungroup it. Required — it is the route's only mutable field, so an empty patch is a validation error, not a silent no-op.

Source code in varagity/api/schemas.py

class ConversationUpdateRequest(BaseModel):
    """Body of ``PATCH /api/conversations/{conversation_id}``.

    Unknown fields are rejected so a typo'd patch fails loudly instead of
    silently doing nothing.

    Attributes:
        group_id: Sidebar group to file the conversation under, or ``None``
            to ungroup it. Required — it is the route's only mutable field,
            so an empty patch is a validation error, not a silent no-op.
    """

    model_config = ConfigDict(extra="forbid")

    group_id: str | None

DeltaEvent

Bases: BaseModel

Payload of the SSE reasoning and token events.

Attributes:

Name	Type	Description
delta	str	The next text fragment, in stream order.

Source code in varagity/api/schemas.py

class DeltaEvent(BaseModel):
    """Payload of the SSE ``reasoning`` and ``token`` events.

    Attributes:
        delta: The next text fragment, in stream order.
    """

    delta: str

DocumentBulkDeleteRequest

Bases: BaseModel

Body of POST /api/documents/delete (spec_v2 §4.2).

Attributes:

Name	Type	Description
doc_ids	list[str]	The documents to remove, at least one. Duplicates collapse and unknown ids are reported rather than fatal, so the corpus table's selection can be posted as-is.

Source code in varagity/api/schemas.py

class DocumentBulkDeleteRequest(BaseModel):
    """Body of ``POST /api/documents/delete`` (spec_v2 §4.2).

    Attributes:
        doc_ids: The documents to remove, at least one. Duplicates collapse
            and unknown ids are reported rather than fatal, so the corpus
            table's selection can be posted as-is.
    """

    model_config = ConfigDict(extra="forbid")

    doc_ids: list[str] = Field(min_length=1)

DocumentBulkDeleteResponse

Bases: BaseModel

Response of POST /api/documents/delete (spec_v2 §4.2).

Attributes:

Name	Type	Description
deleted	list[DocumentDeleteResponse]	Per-document outcomes, in requested order (duplicates collapsed) — the same shape a single delete returns.
not_found	list[str]	Requested ids that no longer had a documents row. Reported, never fatal: a concurrent delete (another tab, a reingest) must not fail the rest of the batch.

Source code in varagity/api/schemas.py

class DocumentBulkDeleteResponse(BaseModel):
    """Response of ``POST /api/documents/delete`` (spec_v2 §4.2).

    Attributes:
        deleted: Per-document outcomes, in requested order (duplicates
            collapsed) — the same shape a single delete returns.
        not_found: Requested ids that no longer had a ``documents`` row.
            Reported, never fatal: a concurrent delete (another tab, a
            reingest) must not fail the rest of the batch.
    """

    deleted: list[DocumentDeleteResponse]
    not_found: list[str]

DocumentDeleteResponse

Bases: BaseModel

Response of DELETE /api/documents/{doc_id} (spec_v2 §4.2).

Also one entry of a bulk delete's deleted list.

Attributes:

Name	Type	Description
doc_id	str	The deleted document.
chunks_deleted	int	Chunks removed (the pgvector count; the Elasticsearch delete_by_query removes the same identity- addressed set).
file_removed	bool	Whether the source file was also deleted from DOCS_PATH (requested via ?remove_file=true and only honored inside the corpus directory).

Source code in varagity/api/schemas.py

class DocumentDeleteResponse(BaseModel):
    """Response of ``DELETE /api/documents/{doc_id}`` (spec_v2 §4.2).

    Also one entry of a bulk delete's ``deleted`` list.

    Attributes:
        doc_id: The deleted document.
        chunks_deleted: Chunks removed (the pgvector count; the
            Elasticsearch ``delete_by_query`` removes the same identity-
            addressed set).
        file_removed: Whether the source file was also deleted from
            ``DOCS_PATH`` (requested via ``?remove_file=true`` and only
            honored inside the corpus directory).
    """

    doc_id: str
    chunks_deleted: int
    file_removed: bool

DocumentOut

Bases: BaseModel

One corpus document in the GET /api/documents list (spec_v2 §4.2).

Attributes:

Name	Type	Description
doc_id	str	The document's stable id.
file_name	str	Base name of the source file.
source	str	Absolute file path recorded at ingest time.
relative_path	str | None	The source's path relative to the corpus directory (DOCS_PATH), POSIX-separated — folder uploads (spec_v3 §5.2) keep their structure, and this is the GUI's grouping key for folding the corpus table back into those folders. None when the source doesn't resolve inside DOCS_PATH (a document ingested from elsewhere lists flat).
file_type	str	File extension without the dot (pdf, docx, …).
content_hash	str	sha256 of the source file's bytes at ingest time.
n_chunks	int	Chunks ingested (0 = no extractable text).
ingested_at	datetime	When the document (last) landed in the stores.
extraction_mix	dict[str, int]	Chunk count per extraction method (text / ocr / ocr_fallback).

Source code in varagity/api/schemas.py

class DocumentOut(BaseModel):
    """One corpus document in the ``GET /api/documents`` list (spec_v2 §4.2).

    Attributes:
        doc_id: The document's stable id.
        file_name: Base name of the source file.
        source: Absolute file path recorded at ingest time.
        relative_path: The source's path relative to the corpus directory
            (``DOCS_PATH``), POSIX-separated — folder uploads (spec_v3
            §5.2) keep their structure, and this is the GUI's grouping key
            for folding the corpus table back into those folders. ``None``
            when the source doesn't resolve inside ``DOCS_PATH`` (a
            document ingested from elsewhere lists flat).
        file_type: File extension without the dot (``pdf``, ``docx``, …).
        content_hash: sha256 of the source file's bytes at ingest time.
        n_chunks: Chunks ingested (``0`` = no extractable text).
        ingested_at: When the document (last) landed in the stores.
        extraction_mix: Chunk count per extraction method (``text`` /
            ``ocr`` / ``ocr_fallback``).
    """

    doc_id: str
    file_name: str
    source: str
    relative_path: str | None = None
    file_type: str
    content_hash: str
    n_chunks: int
    ingested_at: datetime
    extraction_mix: dict[str, int]

DoneEvent

Bases: BaseModel

Payload of the terminal SSE done event (spec_v2 §4.3).

Attributes:

Name	Type	Description
message_id	str	Persisted assistant message id.
conversation_id	str	The conversation the turn was persisted into (the one just created, when the request named none).
answer	str	The full, <think>-stripped answer (authoritative — the streamed deltas are best-effort display).
usage	UsageInfo	Token usage and per-stage latency.

Source code in varagity/api/schemas.py

class DoneEvent(BaseModel):
    """Payload of the terminal SSE ``done`` event (spec_v2 §4.3).

    Attributes:
        message_id: Persisted assistant message id.
        conversation_id: The conversation the turn was persisted into (the
            one just created, when the request named none).
        answer: The full, ``<think>``-stripped answer (authoritative — the
            streamed deltas are best-effort display).
        usage: Token usage and per-stage latency.
    """

    message_id: str
    conversation_id: str
    answer: str
    usage: UsageInfo

ErrorBody

Bases: BaseModel

The machine-readable error inside the envelope (spec_v2 §4.1).

Attributes:

Name	Type	Description
code	str	Stable, machine-readable code (e.g. "es_unreachable") the GUI maps to an actionable banner.
message	str	Human-readable detail.

Source code in varagity/api/schemas.py

class ErrorBody(BaseModel):
    """The machine-readable error inside the envelope (spec_v2 §4.1).

    Attributes:
        code: Stable, machine-readable code (e.g. ``"es_unreachable"``) the
            GUI maps to an actionable banner.
        message: Human-readable detail.
    """

    code: str
    message: str

ErrorEvent

Bases: BaseModel

Payload of the in-band SSE error event.

Emitted when the pipeline fails after the stream already opened (the 200 headers are gone — status can't change; spec_v2 plan decision #7).

Attributes:

Name	Type	Description
code	str	Stable, machine-readable code.
message	str	Human-readable detail.

Source code in varagity/api/schemas.py

class ErrorEvent(BaseModel):
    """Payload of the in-band SSE ``error`` event.

    Emitted when the pipeline fails after the stream already opened (the
    200 headers are gone — status can't change; spec_v2 plan decision #7).

    Attributes:
        code: Stable, machine-readable code.
        message: Human-readable detail.
    """

    code: str
    message: str

ErrorResponse

Bases: BaseModel

The structured error envelope every non-2xx response carries.

Attributes:

Name	Type	Description
error	ErrorBody	The error body.

Source code in varagity/api/schemas.py

class ErrorResponse(BaseModel):
    """The structured error envelope every non-2xx response carries.

    Attributes:
        error: The error body.
    """

    error: ErrorBody

GraphBuildLogEvent

Bases: BaseModel

Payload of the graph-build SSE log event.

Relayed varagity.graph log records (the parse summaries, the build diff's new/changed/unchanged counts, per-document failures), so the browser sees what the server log would show.

Attributes:

Name	Type	Description
level	str	The log level name (INFO | WARNING | ERROR).
message	str	The formatted log message.

Source code in varagity/api/schemas.py

class GraphBuildLogEvent(BaseModel):
    """Payload of the graph-build SSE ``log`` event.

    Relayed ``varagity.graph`` log records (the parse summaries, the build
    diff's new/changed/unchanged counts, per-document failures), so the
    browser sees what the server log would show.

    Attributes:
        level: The log level name (``INFO`` | ``WARNING`` | ``ERROR``).
        message: The formatted log message.
    """

    level: str
    message: str

GraphBuildProgressEvent

Bases: BaseModel

Payload of the graph-build SSE progress event.

Two phases, because a graph build has two very different clocks: scan/parse walk the corpus directory in seconds, then index/process hand the merged messages to the engine and sample its own document statuses for hours (the extraction pass — ADR-017's 7.17 s/message).

Attributes:

Name	Type	Description
stage	str	scan | parse | bound | reset | index | process.
file	str | None	The file being parsed (None outside parse).
current	int | None	Intra-stage progress (files parsed so far; messages kept by bound).
total	int | None	Stage denominator (files found, messages parsed).
docs_done	int | None	Documents the engine reports as processed (process).
docs_total	int | None	Documents the engine holds in any status (process).

Source code in varagity/api/schemas.py

class GraphBuildProgressEvent(BaseModel):
    """Payload of the graph-build SSE ``progress`` event.

    Two phases, because a graph build has two very different clocks:
    ``scan``/``parse`` walk the corpus directory in seconds, then
    ``index``/``process`` hand the merged messages to the engine and sample
    its own document statuses for hours (the extraction pass — ADR-017's
    7.17 s/message).

    Attributes:
        stage: ``scan`` | ``parse`` | ``bound`` | ``reset`` | ``index`` |
            ``process``.
        file: The file being parsed (``None`` outside ``parse``).
        current: Intra-stage progress (files parsed so far; messages kept by
            ``bound``).
        total: Stage denominator (files found, messages parsed).
        docs_done: Documents the engine reports as processed (``process``).
        docs_total: Documents the engine holds in any status (``process``).
    """

    stage: str
    file: str | None = None
    current: int | None = None
    total: int | None = None
    docs_done: int | None = None
    docs_total: int | None = None

GraphBuildRequest

Bases: BaseModel

Body of POST /api/graph/build (spec_graphrag §5.2).

Attributes:

Name	Type	Description
reingest	bool	Wipe the engine's working directory and index from scratch. The only thing that clears the graph-stale flag (stage-2 decision #16) — and the only reason to pay a full backfill twice.
message_limit	int | None	Index at most this many messages, newest first — the bounded escape hatch that makes a minutes-long spot check possible on an archive whose full build runs for days. A bounded build never prunes: its render is partial by construction, so documents it does not mention are kept, not deleted.
since	date | None	Ignore messages older than this date (the other half of the bound; same no-prune rule).

Source code in varagity/api/schemas.py

class GraphBuildRequest(BaseModel):
    """Body of ``POST /api/graph/build`` (spec_graphrag §5.2).

    Attributes:
        reingest: Wipe the engine's working directory and index from
            scratch. The **only** thing that clears the graph-stale flag
            (stage-2 decision #16) — and the only reason to pay a full
            backfill twice.
        message_limit: Index at most this many messages, newest first — the
            bounded escape hatch that makes a minutes-long spot check
            possible on an archive whose full build runs for days. A bounded
            build never prunes: its render is partial by construction, so
            documents it does not mention are kept, not deleted.
        since: Ignore messages older than this date (the other half of the
            bound; same no-prune rule).
    """

    model_config = ConfigDict(extra="forbid")

    reingest: bool = False
    message_limit: int | None = Field(default=None, ge=1)
    since: date | None = None

GraphBuildRunOut

Bases: BaseModel

One graph build run's state (POST /api/graph/build + its stream).

Attributes:

Name	Type	Description
run_id	str	The run handle.
state	str	running | completed | failed.
reingest	bool	Whether the run wiped the working directory first.
bounded	bool	Whether message_limit/since narrowed the render (a bounded run deliberately skips the removal pass).
message_limit	int | None	The requested message cap, if any.
since	date | None	The requested date floor, if any.
started_at	datetime	When the run started.
finished_at	datetime | None	When it reached a terminal state (None while running).
summary	GraphBuildSummaryOut | None	The final counters (terminal states only; a run that died before the engine returned carries None).
error	str | None	The run-level failure (failed only; per-document failures ride in summary.failures and the document statuses).

Source code in varagity/api/schemas.py

class GraphBuildRunOut(BaseModel):
    """One graph build run's state (``POST /api/graph/build`` + its stream).

    Attributes:
        run_id: The run handle.
        state: ``running`` | ``completed`` | ``failed``.
        reingest: Whether the run wiped the working directory first.
        bounded: Whether ``message_limit``/``since`` narrowed the render (a
            bounded run deliberately skips the removal pass).
        message_limit: The requested message cap, if any.
        since: The requested date floor, if any.
        started_at: When the run started.
        finished_at: When it reached a terminal state (``None`` while
            running).
        summary: The final counters (terminal states only; a run that died
            before the engine returned carries ``None``).
        error: The run-level failure (``failed`` only; per-document failures
            ride in ``summary.failures`` and the document statuses).
    """

    run_id: str
    state: str
    reingest: bool
    bounded: bool
    message_limit: int | None = None
    since: date | None = None
    started_at: datetime
    finished_at: datetime | None = None
    summary: GraphBuildSummaryOut | None = None
    error: str | None = None

GraphBuildStatusEvent

Bases: BaseModel

Payload of the graph-build SSE status event.

The stream's first frame (a snapshot on connect) and its last (the terminal state). run=None means no build has run in this API process — the stream closes immediately after.

Attributes:

Name	Type	Description
run	GraphBuildRunOut | None	The current (or last) run, if any.

Source code in varagity/api/schemas.py

class GraphBuildStatusEvent(BaseModel):
    """Payload of the graph-build SSE ``status`` event.

    The stream's first frame (a snapshot on connect) and its last (the
    terminal state). ``run=None`` means no build has run in this API
    process — the stream closes immediately after.

    Attributes:
        run: The current (or last) run, if any.
    """

    run: GraphBuildRunOut | None = None

GraphBuildSummaryOut

Bases: BaseModel

The graph build's counters (terminal runs only).

Attributes:

Name	Type	Description
files_scanned	int	Files found under GRAPH_DOCS_PATH.
files_parsed	int	Of those, files a registered message source claimed.
files_failed	int	Files that raised while parsing (the run continued).
messages_parsed	int	Messages recovered from every parsed file.
messages_indexed	int	Messages handed to the engine after the bounds were applied (equal to messages_parsed for an unbounded build, before guid-merging collapses overlaps).
messages_seen	int	What the engine reported after guid-merging the batches — the upsert grain.
wall_clock_s	float	Seconds the engine spent indexing.
resumed	int	Documents already in flight (or failed) when this run started — the work a killed build left behind and this one picked up.
documents	dict[str, int]	The engine's document counts by status when the run finished.
failures	list[str]	Per-step failures the build caught and continued past.

Source code in varagity/api/schemas.py

class GraphBuildSummaryOut(BaseModel):
    """The graph build's counters (terminal runs only).

    Attributes:
        files_scanned: Files found under ``GRAPH_DOCS_PATH``.
        files_parsed: Of those, files a registered message source claimed.
        files_failed: Files that raised while parsing (the run continued).
        messages_parsed: Messages recovered from every parsed file.
        messages_indexed: Messages handed to the engine after the bounds
            were applied (equal to ``messages_parsed`` for an unbounded
            build, before guid-merging collapses overlaps).
        messages_seen: What the engine reported after guid-merging the
            batches — the upsert grain.
        wall_clock_s: Seconds the engine spent indexing.
        resumed: Documents already in flight (or failed) when this run
            started — the work a killed build left behind and this one
            picked up.
        documents: The engine's document counts by status when the run
            finished.
        failures: Per-step failures the build caught and continued past.
    """

    files_scanned: int
    files_parsed: int
    files_failed: int
    messages_parsed: int
    messages_indexed: int
    messages_seen: int
    wall_clock_s: float
    resumed: int
    documents: dict[str, int] = {}
    failures: list[str] = []

GraphDocumentDeleteResponse

Bases: BaseModel

Response of DELETE /api/graph/documents/{name}.

Attributes:

Name	Type	Description
relative_path	str	The removed file's path under GRAPH_DOCS_PATH.
graph_stale	bool	Whether the graph is now flagged stale — it keeps the deleted file's messages until the next rebuild, and saying so is the honest alternative to silently diverging (stage-2 decision 16). False only when the flag could not be persisted.

Source code in varagity/api/schemas.py

class GraphDocumentDeleteResponse(BaseModel):
    """Response of ``DELETE /api/graph/documents/{name}``.

    Attributes:
        relative_path: The removed file's path under ``GRAPH_DOCS_PATH``.
        graph_stale: Whether the graph is now flagged stale — it keeps the
            deleted file's messages until the next rebuild, and saying so is
            the honest alternative to silently diverging (stage-2 decision
            #16). ``False`` only when the flag could not be persisted.
    """

    relative_path: str
    graph_stale: bool

GraphDocumentOut

Bases: BaseModel

One file in the GET /api/graph/documents list (spec_graphrag §10.2).

Unlike the chunk-RAG corpus list — which reports the documents table — this is a directory listing: the graph's own record of what it indexed lives in the engine's workdir, and the tab's job is to show what has been dropped in. Non-source files (a contacts list, the -wal / -shm sidecars a chat.db needs) are listed too, with a None doc_id: they are on disk and the user put them there.

Attributes:

Name	Type	Description
name	str	Base name of the file.
relative_path	str	Its POSIX path relative to GRAPH_DOCS_PATH (the delete route's key).
size_bytes	int	Size on disk.
modified_at	datetime	The file's mtime.
doc_id	str | None	The stable id a build derives for it (path relative to GRAPH_DOCS_PATH + byte hash), or None when no build has scanned this file yet — or when no registered message source claims it.
parse	GraphParseSummary | None	What the last scan parsed out of it, or None when no build has scanned it (parsing a multi-gigabyte database is a build's job, not a list request's).

Source code in varagity/api/schemas.py

class GraphDocumentOut(BaseModel):
    """One file in the ``GET /api/graph/documents`` list (spec_graphrag §10.2).

    Unlike the chunk-RAG corpus list — which reports the ``documents``
    *table* — this is a directory listing: the graph's own record of what it
    indexed lives in the engine's workdir, and the tab's job is to show what
    has been dropped in. Non-source files (a contacts list, the ``-wal`` /
    ``-shm`` sidecars a ``chat.db`` needs) are listed too, with a ``None``
    ``doc_id``: they are on disk and the user put them there.

    Attributes:
        name: Base name of the file.
        relative_path: Its POSIX path relative to ``GRAPH_DOCS_PATH`` (the
            delete route's key).
        size_bytes: Size on disk.
        modified_at: The file's mtime.
        doc_id: The stable id a build derives for it (path relative to
            ``GRAPH_DOCS_PATH`` + byte hash), or ``None`` when no build has
            scanned this file yet — or when no registered message source
            claims it.
        parse: What the last scan parsed out of it, or ``None`` when no
            build has scanned it (parsing a multi-gigabyte database is a
            build's job, not a list request's).
    """

    name: str
    relative_path: str
    size_bytes: int
    modified_at: datetime
    doc_id: str | None = None
    parse: GraphParseSummary | None = None

GraphEntityDetailOut

Bases: BaseModel

Response of GET /api/graph/entities/{name} — the inspect panel.

Attributes:

Name	Type	Description
entity	GraphExportNodeOut	The entity itself, with the summary the engine merged from every mention.
relations	list[GraphExportEdgeOut]	Every edge touching it in a depth-1 slice, i.e. its immediate neighbourhood.
sources	list[GraphTranscriptRefOut]	The transcript days it was extracted from — the drill-down from a name in the graph back to the conversations behind it. Empty when the engine recorded no document provenance.

Source code in varagity/api/schemas.py

class GraphEntityDetailOut(BaseModel):
    """Response of ``GET /api/graph/entities/{name}`` — the inspect panel.

    Attributes:
        entity: The entity itself, with the summary the engine merged from
            every mention.
        relations: Every edge touching it in a depth-1 slice, i.e. its
            immediate neighbourhood.
        sources: The transcript days it was extracted from — the drill-down
            from a name in the graph back to the conversations behind it.
            Empty when the engine recorded no document provenance.
    """

    entity: GraphExportNodeOut
    relations: list[GraphExportEdgeOut] = []
    sources: list[GraphTranscriptRefOut] = []

GraphEntityOut

Bases: BaseModel

One entity the graph retrieval surfaced (spec_graphrag §10.2).

Attributes:

Name	Type	Description
name	str	The entity's canonical name, as the engine resolved it.
type	str | None	The engine's entity type/category (None when untyped) — what the evidence chips and the graph view color by, LightRAG having no community tier (ADR-017).
summary	str | None	The engine's description of the entity.

Source code in varagity/api/schemas.py

class GraphEntityOut(BaseModel):
    """One entity the graph retrieval surfaced (spec_graphrag §10.2).

    Attributes:
        name: The entity's canonical name, as the engine resolved it.
        type: The engine's entity type/category (``None`` when untyped) —
            what the evidence chips and the graph view color by, LightRAG
            having no community tier (ADR-017).
        summary: The engine's description of the entity.
    """

    name: str
    type: str | None = None
    summary: str | None = None

GraphExportEdgeOut

Bases: BaseModel

One relation in the drawable graph slice (spec_graphrag §4.4).

Attributes:

Name	Type	Description
id	str	Stable edge identity within the slice (source-target).
source	str	Name of the edge's source entity.
target	str	Name of the edge's target entity.
label	str | None	Short relation label/keywords (None when unlabelled).
description	str | None	The engine's longer description of the relation.

Source code in varagity/api/schemas.py

class GraphExportEdgeOut(BaseModel):
    """One relation in the drawable graph slice (spec_graphrag §4.4).

    Attributes:
        id: Stable edge identity within the slice (``source-target``).
        source: Name of the edge's source entity.
        target: Name of the edge's target entity.
        label: Short relation label/keywords (``None`` when unlabelled).
        description: The engine's longer description of the relation.
    """

    id: str
    source: str
    target: str
    label: str | None = None
    description: str | None = None

GraphExportNodeOut

Bases: BaseModel

One entity in the drawable graph slice (spec_graphrag §4.4).

Attributes:

Name	Type	Description
id	str	The entity's canonical name — also the node's identity in the view and the key the entity-detail route takes.
entity_type	str | None	The engine's type/category. This is what the view colors and groups by: LightRAG builds no community tier, so type is the honest clustering rather than a faked one (ADR-017). None for an untyped node.
description	str | None	The engine's merged description of the entity.
degree	int	How many of this slice's edges touch the node — the size channel in the view. A subgraph degree, not a whole-graph one: the export is capped, so the number describes the picture drawn.

Source code in varagity/api/schemas.py

class GraphExportNodeOut(BaseModel):
    """One entity in the drawable graph slice (spec_graphrag §4.4).

    Attributes:
        id: The entity's canonical name — also the node's identity in the
            view and the key the entity-detail route takes.
        entity_type: The engine's type/category. **This is what the view
            colors and groups by**: LightRAG builds no community tier, so
            type is the honest clustering rather than a faked one (ADR-017).
            ``None`` for an untyped node.
        description: The engine's merged description of the entity.
        degree: How many of *this slice's* edges touch the node — the size
            channel in the view. A subgraph degree, not a whole-graph one:
            the export is capped, so the number describes the picture drawn.
    """

    id: str
    entity_type: str | None = None
    description: str | None = None
    degree: int = 0

GraphExportOut

Bases: BaseModel

Response of GET /api/graph/export — what the graph view draws.

Attributes:

Name	Type	Description
nodes	list[GraphExportNodeOut]	The exported entities, engine order (degree-descending for a whole-graph export, so the cap keeps the most connected).
edges	list[GraphExportEdgeOut]	Every edge between exported nodes.
truncated	bool	Whether the cap hid nodes the graph actually holds. The view must surface this rather than implying it drew everything — an honest partial picture, per §4.4's scale ceiling.

Source code in varagity/api/schemas.py

class GraphExportOut(BaseModel):
    """Response of ``GET /api/graph/export`` — what the graph view draws.

    Attributes:
        nodes: The exported entities, engine order (degree-descending for a
            whole-graph export, so the cap keeps the most connected).
        edges: Every edge between exported nodes.
        truncated: Whether the cap hid nodes the graph actually holds. The
            view **must** surface this rather than implying it drew
            everything — an honest partial picture, per §4.4's scale ceiling.
    """

    nodes: list[GraphExportNodeOut] = []
    edges: list[GraphExportEdgeOut] = []
    truncated: bool = False

GraphParseSummary

Bases: BaseModel

What the last build's scan found inside one graph source file.

Attributes:

Name	Type	Description
messages	int	Recoverable messages parsed from the file.
threads	int	Distinct conversations they belong to.
first	datetime | None	Oldest message timestamp (None for an empty parse).
last	datetime | None	Newest message timestamp (None for an empty parse).

Source code in varagity/api/schemas.py

class GraphParseSummary(BaseModel):
    """What the last build's scan found inside one graph source file.

    Attributes:
        messages: Recoverable messages parsed from the file.
        threads: Distinct conversations they belong to.
        first: Oldest message timestamp (``None`` for an empty parse).
        last: Newest message timestamp (``None`` for an empty parse).
    """

    messages: int
    threads: int
    first: datetime | None = None
    last: datetime | None = None

GraphRelationOut

Bases: BaseModel

One relation the graph retrieval surfaced (spec_graphrag §10.2).

Attributes:

Name	Type	Description
source	str	Name of the edge's source entity.
target	str	Name of the edge's target entity.
label	str | None	Short relation label/keywords (None when unlabelled).
description	str | None	The engine's longer description — the fact the answer grounds on.

Source code in varagity/api/schemas.py

class GraphRelationOut(BaseModel):
    """One relation the graph retrieval surfaced (spec_graphrag §10.2).

    Attributes:
        source: Name of the edge's source entity.
        target: Name of the edge's target entity.
        label: Short relation label/keywords (``None`` when unlabelled).
        description: The engine's longer description — the *fact* the answer
            grounds on.
    """

    source: str
    target: str
    label: str | None = None
    description: str | None = None

GraphRetrievalPayload

Bases: BaseModel

The graph half of the SSE retrieval event (spec_graphrag §4.3).

Attributes:

Name	Type	Description
mode	str	The engine query mode the turn retrieved with (GRAPH_QUERY_MODE).
entities	list[GraphEntityOut]	Entities the retrieval surfaced.
relations	list[GraphRelationOut]	Relations the retrieval surfaced.
transcripts	list[GraphTranscriptOut]	The cited transcript days, best first — the cards a citation chip resolves to, labelled "{thread} ({span})".

Source code in varagity/api/schemas.py

class GraphRetrievalPayload(BaseModel):
    """The graph half of the SSE ``retrieval`` event (spec_graphrag §4.3).

    Attributes:
        mode: The engine query mode the turn retrieved with
            (``GRAPH_QUERY_MODE``).
        entities: Entities the retrieval surfaced.
        relations: Relations the retrieval surfaced.
        transcripts: The cited transcript days, best first — the cards a
            citation chip resolves to, labelled ``"{thread} ({span})"``.
    """

    mode: str
    entities: list[GraphEntityOut] = []
    relations: list[GraphRelationOut] = []
    transcripts: list[GraphTranscriptOut] = []

GraphStatusOut

Bases: BaseModel

Response of GET /api/graph/status — the graph tab's poll surface.

Every size field is optional because "nothing has been built here yet" and "the engine would not say" are both real states, and reporting them as 0 would read like an empty graph (the honesty rule :class:~varagity.graph.records.GraphStats sets).

Attributes:

Name	Type	Description
enabled	bool	The GRAPH_ENABLED kill switch.
stale	bool	Whether a source file was deleted since the last rebuild.
building	bool	Whether a build is running in this API process.
documents	dict[str, int]	The engine's document counts by status (its own vocabulary: pending, processing, processed, failed, …); empty when nothing has been indexed.
entities	int | None	Entity count from the workdir's summary sidecar.
relations	int | None	Relation count from the same.
message_guids	int | None	Messages the workdir's manifest accounts for.
workdir_files	int	Files in the engine's working directory that a backup would copy (ADR-019) — 0 when nothing has been built. The backup panel's "nothing built yet": the export route's own nothing_to_export rule, so the two can never disagree (an unopenable graph, or a first build killed before its summary, still has files to back up and to replace).
last_build	GraphBuildRunOut | None	The current (or last) build run in this API process.

Source code in varagity/api/schemas.py

class GraphStatusOut(BaseModel):
    """Response of ``GET /api/graph/status`` — the graph tab's poll surface.

    Every size field is optional because "nothing has been built here yet"
    and "the engine would not say" are both real states, and reporting them
    as ``0`` would read like an empty graph (the honesty rule
    :class:`~varagity.graph.records.GraphStats` sets).

    Attributes:
        enabled: The ``GRAPH_ENABLED`` kill switch.
        stale: Whether a source file was deleted since the last rebuild.
        building: Whether a build is running in this API process.
        documents: The engine's document counts by status (its own
            vocabulary: ``pending``, ``processing``, ``processed``,
            ``failed``, …); empty when nothing has been indexed.
        entities: Entity count from the workdir's summary sidecar.
        relations: Relation count from the same.
        message_guids: Messages the workdir's manifest accounts for.
        workdir_files: Files in the engine's working directory that a
            backup would copy (ADR-019) — ``0`` when nothing has been built.
            The backup panel's "nothing built yet": the export route's own
            ``nothing_to_export`` rule, so the two can never disagree (an
            unopenable graph, or a first build killed before its summary,
            still has files to back up and to replace).
        last_build: The current (or last) build run in this API process.
    """

    enabled: bool
    stale: bool
    building: bool
    documents: dict[str, int] = {}
    entities: int | None = None
    relations: int | None = None
    message_guids: int | None = None
    workdir_files: int = 0
    last_build: GraphBuildRunOut | None = None

GraphTranscriptOut

Bases: BaseModel

One cited transcript day of a graph answer (spec_graphrag §4.3).

Evidence is day-grain, not message-grain: that is what the engines can honestly attribute (ADR-017's priced regret), so this is the card the evidence panel draws and the label the answer cites.

Attributes:

Name	Type	Description
doc_key	str	The transcript document key — the join back to the corpus.
thread_name	str	Human-facing thread label.
span	str	The document's day span (YYYY-MM-DD or first..last).
excerpt	str	The retrieved passage, capped for the wire.
message_count	int	Messages the graph's manifest accounts for in that document (0 when it cannot say — never a claim of empty).

Source code in varagity/api/schemas.py

class GraphTranscriptOut(BaseModel):
    """One cited transcript day of a graph answer (spec_graphrag §4.3).

    Evidence is **day-grain, not message-grain**: that is what the engines
    can honestly attribute (ADR-017's priced regret), so this is the card
    the evidence panel draws and the label the answer cites.

    Attributes:
        doc_key: The transcript document key — the join back to the corpus.
        thread_name: Human-facing thread label.
        span: The document's day span (``YYYY-MM-DD`` or ``first..last``).
        excerpt: The retrieved passage, capped for the wire.
        message_count: Messages the graph's manifest accounts for in that
            document (``0`` when it cannot say — never a claim of empty).
    """

    doc_key: str
    thread_name: str
    span: str
    excerpt: str
    message_count: int = 0

GraphTranscriptRefOut

Bases: BaseModel

One transcript day an entity was extracted from (the drill-down).

The evidence panel's :class:GraphTranscriptOut without the retrieved text: this is a pointer into the corpus ("Bob was talked about on these days"), not a retrieval hit, so there is no excerpt to show.

Attributes:

Name	Type	Description
doc_key	str	The transcript document key — the join back to the corpus.
thread_name	str	Human-facing thread label, resolved through the workdir manifest (falling back to the key's own thread id).
span	str	The document's day span (YYYY-MM-DD or first..last).
message_count	int	Messages the manifest accounts for in that document (0 when the manifest does not know the key — never a claim that the day was empty).

Source code in varagity/api/schemas.py

class GraphTranscriptRefOut(BaseModel):
    """One transcript day an entity was extracted from (the drill-down).

    The evidence panel's :class:`GraphTranscriptOut` without the retrieved
    text: this is a *pointer* into the corpus ("Bob was talked about on
    these days"), not a retrieval hit, so there is no excerpt to show.

    Attributes:
        doc_key: The transcript document key — the join back to the corpus.
        thread_name: Human-facing thread label, resolved through the
            workdir manifest (falling back to the key's own thread id).
        span: The document's day span (``YYYY-MM-DD`` or ``first..last``).
        message_count: Messages the manifest accounts for in that document
            (``0`` when the manifest does not know the key — never a claim
            that the day was empty).
    """

    doc_key: str
    thread_name: str
    span: str
    message_count: int = 0

GraphUploadResponse

Bases: BaseModel

Response of POST /api/graph/documents.

Attributes:

Name	Type	Description
files	list[UploadedFileOut]	Per-file outcomes, in upload order (the corpus upload's shape, plus its unsupported_graph_source rejection).

Source code in varagity/api/schemas.py

class GraphUploadResponse(BaseModel):
    """Response of ``POST /api/graph/documents``.

    Attributes:
        files: Per-file outcomes, in upload order (the corpus upload's
            shape, plus its ``unsupported_graph_source`` rejection).
    """

    files: list[UploadedFileOut]

GroupCreateRequest

Bases: BaseModel

Body of POST /api/groups.

Attributes:

Name	Type	Description
name	str	Display name of the new group.

Source code in varagity/api/schemas.py

class GroupCreateRequest(BaseModel):
    """Body of ``POST /api/groups``.

    Attributes:
        name: Display name of the new group.
    """

    model_config = ConfigDict(extra="forbid")

    name: str = Field(min_length=1, max_length=200)

GroupOut

Bases: BaseModel

One sidebar conversation group (a user-created folder).

Attributes:

Name	Type	Description
group_id	str	The app-generated id.
name	str	Display name.
created_at	datetime	Creation timestamp.

Source code in varagity/api/schemas.py

class GroupOut(BaseModel):
    """One sidebar conversation group (a user-created folder).

    Attributes:
        group_id: The app-generated id.
        name: Display name.
        created_at: Creation timestamp.
    """

    group_id: str
    name: str
    created_at: datetime

HealthResponse

Bases: BaseModel

Response of GET /api/health.

The endpoint answers 200 whenever the API process itself is alive — per-dependency state lives in the body, so a flapping backing service doesn't mark the api container unhealthy.

Attributes:

Name	Type	Description
ok	bool	True when every dependency probe succeeded.
services	dict[str, ServiceHealth]	Probe result per service name (llamacpp, infinity, postgres, elasticsearch, prefect).

Source code in varagity/api/schemas.py

class HealthResponse(BaseModel):
    """Response of ``GET /api/health``.

    The endpoint answers ``200`` whenever the API process itself is alive —
    per-dependency state lives in the body, so a flapping backing service
    doesn't mark the ``api`` container unhealthy.

    Attributes:
        ok: ``True`` when every dependency probe succeeded.
        services: Probe result per service name (``llamacpp``,
            ``infinity``, ``postgres``, ``elasticsearch``, ``prefect``).
    """

    ok: bool
    services: dict[str, ServiceHealth]

IngestLogEvent

Bases: BaseModel

Payload of the ingest-status SSE log event.

Relayed varagity.ingest log records (the per-file outcome lines the terminal shows: skipped-unchanged, no-text, failure tracebacks' heads), so the browser progress view mirrors the CLI run.

Attributes:

Name	Type	Description
level	str	The log level name (INFO | WARNING | ERROR).
message	str	The formatted log message.

Source code in varagity/api/schemas.py

class IngestLogEvent(BaseModel):
    """Payload of the ingest-status SSE ``log`` event.

    Relayed ``varagity.ingest`` log records (the per-file outcome lines the
    terminal shows: skipped-unchanged, no-text, failure tracebacks' heads),
    so the browser progress view mirrors the CLI run.

    Attributes:
        level: The log level name (``INFO`` | ``WARNING`` | ``ERROR``).
        message: The formatted log message.
    """

    level: str
    message: str

IngestProgressEvent

Bases: BaseModel

Payload of the ingest-status SSE progress event.

One frame per pipeline stage transition, mirroring the CLI's rich display: discover (with total files), then per file parse → chunk → contextualize (with per-chunk current/total ticks) → embed → store, and a file_done frame carrying the file's outcome plus the run-level progress counters.

Attributes:

Name	Type	Description
stage	str	discover | parse | chunk | contextualize | embed | store | file_done.
file	str | None	The file being processed (None for discover).
outcome	str | None	file_done only — ingested | skipped | no_text | failed.
current	int | None	Intra-stage progress (contextualized chunks so far).
total	int | None	Stage denominator (files for discover, chunks for chunk/contextualize, texts for embed).
files_done	int | None	Files finished so far (file_done only).
files_total	int | None	Files discovered (file_done only).

Source code in varagity/api/schemas.py

class IngestProgressEvent(BaseModel):
    """Payload of the ingest-status SSE ``progress`` event.

    One frame per pipeline stage transition, mirroring the CLI's ``rich``
    display: ``discover`` (with ``total`` files), then per file ``parse`` →
    ``chunk`` → ``contextualize`` (with per-chunk ``current``/``total``
    ticks) → ``embed`` → ``store``, and a ``file_done`` frame carrying the
    file's outcome plus the run-level progress counters.

    Attributes:
        stage: ``discover`` | ``parse`` | ``chunk`` | ``contextualize`` |
            ``embed`` | ``store`` | ``file_done``.
        file: The file being processed (``None`` for ``discover``).
        outcome: ``file_done`` only — ``ingested`` | ``skipped`` |
            ``no_text`` | ``failed``.
        current: Intra-stage progress (contextualized chunks so far).
        total: Stage denominator (files for ``discover``, chunks for
            ``chunk``/``contextualize``, texts for ``embed``).
        files_done: Files finished so far (``file_done`` only).
        files_total: Files discovered (``file_done`` only).
    """

    stage: str
    file: str | None = None
    outcome: str | None = None
    current: int | None = None
    total: int | None = None
    files_done: int | None = None
    files_total: int | None = None

IngestRunOut

Bases: BaseModel

One ingest run's state (POST /api/ingest + the status stream).

Attributes:

Name	Type	Description
run_id	str	The run handle.
state	str	running | completed | failed.
reingest	bool	Whether the run re-processes unchanged files.
started_at	datetime	When the run started.
finished_at	datetime | None	When it reached a terminal state (None while running).
summary	IngestSummaryOut | None	The final counters (terminal states only; failed runs may carry None when the flow died before summarizing).
error	str | None	The flow-level failure (failed only; per-file failures ride in summary.failed and the log events instead).

Source code in varagity/api/schemas.py

class IngestRunOut(BaseModel):
    """One ingest run's state (``POST /api/ingest`` + the status stream).

    Attributes:
        run_id: The run handle.
        state: ``running`` | ``completed`` | ``failed``.
        reingest: Whether the run re-processes unchanged files.
        started_at: When the run started.
        finished_at: When it reached a terminal state (``None`` while
            running).
        summary: The final counters (terminal states only; ``failed`` runs
            may carry ``None`` when the flow died before summarizing).
        error: The flow-level failure (``failed`` only; per-file failures
            ride in ``summary.failed`` and the ``log`` events instead).
    """

    run_id: str
    state: str
    reingest: bool
    started_at: datetime
    finished_at: datetime | None = None
    summary: IngestSummaryOut | None = None
    error: str | None = None

IngestStartRequest

Bases: BaseModel

Body of POST /api/ingest.

Attributes:

Name	Type	Description
reingest	bool	Delete each discovered document's previous ingest and re-process it (required after reingest-affecting setting changes — content hashes don't change, so unchanged files are otherwise skipped).

Source code in varagity/api/schemas.py

class IngestStartRequest(BaseModel):
    """Body of ``POST /api/ingest``.

    Attributes:
        reingest: Delete each discovered document's previous ingest and
            re-process it (required after reingest-affecting setting
            changes — content hashes don't change, so unchanged files are
            otherwise skipped).
    """

    reingest: bool = False

IngestStatusEvent

Bases: BaseModel

Payload of the ingest-status SSE status event.

The stream's first frame (a snapshot on connect) and its last (the terminal state). run=None means no ingest has run in this API process — the stream closes immediately after.

Attributes:

Name	Type	Description
run	IngestRunOut | None	The current (or last) run, if any.

Source code in varagity/api/schemas.py

class IngestStatusEvent(BaseModel):
    """Payload of the ingest-status SSE ``status`` event.

    The stream's first frame (a snapshot on connect) and its last (the
    terminal state). ``run=None`` means no ingest has run in this API
    process — the stream closes immediately after.

    Attributes:
        run: The current (or last) run, if any.
    """

    run: IngestRunOut | None = None

IngestSummaryOut

Bases: BaseModel

The ingest run counters (mirrors the loader's IngestSummary).

Attributes:

Name	Type	Description
discovered	int	Files found in the corpus buckets.
ingested	int	Files parsed, chunked, embedded, and stored this run.
skipped	int	Unchanged files skipped via the idempotency check.
no_text	int	Files with no extractable text.
unsupported	int	Files whose bucket has no registered parser.
failed	int	Files that raised during ingestion (run continued).
chunks	int	Total chunks stored this run.

Source code in varagity/api/schemas.py

class IngestSummaryOut(BaseModel):
    """The ingest run counters (mirrors the loader's ``IngestSummary``).

    Attributes:
        discovered: Files found in the corpus buckets.
        ingested: Files parsed, chunked, embedded, and stored this run.
        skipped: Unchanged files skipped via the idempotency check.
        no_text: Files with no extractable text.
        unsupported: Files whose bucket has no registered parser.
        failed: Files that raised during ingestion (run continued).
        chunks: Total chunks stored this run.
    """

    discovered: int
    ingested: int
    skipped: int
    no_text: int
    unsupported: int
    failed: int
    chunks: int

MessageOut

Bases: BaseModel

One persisted message in a transcript.

Attributes:

Name	Type	Description
message_id	str	The app-generated id.
role	str	"user" or "assistant".
content	str	The question or the generated answer.
created_at	datetime	Persistence timestamp.
retrieval_method	str | None	Retrieval method of an assistant turn.
latency_ms	dict[str, Any] | None	Per-stage timings of an assistant turn.
reasoning	str | None	Captured <think> stream, if any.
condensed_query	str | None	The standalone search query that drove an assistant turn's retrieval (spec_v3 §8 — snapshot semantics, like the sources: it explains a historical answer). None when the search used the user's words verbatim.
chat_engine	str | None	Registry name of the chat engine that produced an assistant turn (None for user turns and pre-v3 history).
target_corpus	str | None	The corpus the turn asked for ("rag" | "graph"); None for user turns and pre-stage-2 history, which clients read as chunk RAG. Recorded even when the graph turn degraded — the request is part of the record (ADR-017).
graph_evidence	dict[str, Any] | None	The graph retrieval's mode/entities/ relations snapshot; None for chunk turns and for a degraded graph turn, whose evidence is chunk-shaped in sources.
sources	list[MessageSourceOut]	The turn's snapshotted evidence, rank order.

Source code in varagity/api/schemas.py

class MessageOut(BaseModel):
    """One persisted message in a transcript.

    Attributes:
        message_id: The app-generated id.
        role: ``"user"`` or ``"assistant"``.
        content: The question or the generated answer.
        created_at: Persistence timestamp.
        retrieval_method: Retrieval method of an assistant turn.
        latency_ms: Per-stage timings of an assistant turn.
        reasoning: Captured ``<think>`` stream, if any.
        condensed_query: The standalone search query that drove an
            assistant turn's retrieval (spec_v3 §8 — snapshot semantics,
            like the sources: it explains a historical answer). ``None``
            when the search used the user's words verbatim.
        chat_engine: Registry name of the chat engine that produced an
            assistant turn (``None`` for user turns and pre-v3 history).
        target_corpus: The corpus the turn asked for (``"rag"`` |
            ``"graph"``); ``None`` for user turns and pre-stage-2 history,
            which clients read as chunk RAG. Recorded even when the graph
            turn degraded — the request is part of the record (ADR-017).
        graph_evidence: The graph retrieval's ``mode``/``entities``/
            ``relations`` snapshot; ``None`` for chunk turns and for a
            degraded graph turn, whose evidence is chunk-shaped in
            ``sources``.
        sources: The turn's snapshotted evidence, rank order.
    """

    message_id: str
    role: str
    content: str
    created_at: datetime
    retrieval_method: str | None = None
    latency_ms: dict[str, Any] | None = None
    reasoning: str | None = None
    condensed_query: str | None = None
    chat_engine: str | None = None
    target_corpus: str | None = None
    graph_evidence: dict[str, Any] | None = None
    sources: list[MessageSourceOut] = []

MessageSourceOut

Bases: BaseModel

One snapshotted evidence row of a persisted assistant turn.

Attributes:

Name	Type	Description
rank	int	Final rank in the answer's evidence (1-based).
chunk_id	str	Soft reference to the producing chunk — or, on a graph turn, the cited transcript day's doc_key.
trace	dict[str, Any]	The spec_v2 §9.1 snapshot: score, content, context, source provenance, and the serialized retrieval trace. A graph turn's rows carry the other shape (kind="graph_transcript" plus thread/span/excerpt/guids), so clients branch on kind.

Source code in varagity/api/schemas.py

class MessageSourceOut(BaseModel):
    """One snapshotted evidence row of a persisted assistant turn.

    Attributes:
        rank: Final rank in the answer's evidence (1-based).
        chunk_id: Soft reference to the producing chunk — or, on a graph
            turn, the cited transcript day's ``doc_key``.
        trace: The spec_v2 §9.1 snapshot: score, content, context, source
            provenance, and the serialized retrieval trace. A graph turn's
            rows carry the other shape (``kind="graph_transcript"`` plus
            thread/span/excerpt/guids), so clients branch on ``kind``.
    """

    rank: int
    chunk_id: str
    trace: dict[str, Any]

NumericRange

Bases: BaseModel

Valid range of one numeric setting, for GUI control generation.

Attributes:

Name	Type	Description
min	float | None	Inclusive lower bound (None = unbounded).
max	float | None	Inclusive upper bound (None = unbounded).

Source code in varagity/api/schemas.py

class NumericRange(BaseModel):
    """Valid range of one numeric setting, for GUI control generation.

    Attributes:
        min: Inclusive lower bound (``None`` = unbounded).
        max: Inclusive upper bound (``None`` = unbounded).
    """

    min: float | None = None
    max: float | None = None

PreviewLocateRequest

Bases: BaseModel

Body of POST /api/documents/{doc_id}/preview/locate (ADR-010).

Attributes:

Name	Type	Description
text	str	The chunk content to locate — both wire shapes (the live retrieval event and persisted message_sources snapshots) already deliver it to the client, so history previews work without any migration.

Source code in varagity/api/schemas.py

class PreviewLocateRequest(BaseModel):
    """Body of ``POST /api/documents/{doc_id}/preview/locate`` (ADR-010).

    Attributes:
        text: The chunk content to locate — both wire shapes (the live
            ``retrieval`` event and persisted ``message_sources`` snapshots)
            already deliver it to the client, so history previews work
            without any migration.
    """

    text: str = Field(min_length=1, max_length=20_000)

PreviewLocateResponse

Bases: BaseModel

Where a chunk's text lives in its source document (ADR-010).

available=False + reason covers every degradable condition (preview_disabled | unsupported_type | file_missing | file_changed | conversion_unavailable | conversion_failed | no_match) — the GUI falls back to the full-text view on any of them, never a dead panel.

Attributes:

Name	Type	Description
available	bool	Whether a page was located (the fields below are set).
reason	str | None	The degradable condition when available is false.
page	int | None	Best-matching page, 1-based.
page_count	int | None	Total pages in the document (also set on no_match).
rects	list[PreviewRect]	Highlight rectangles for the chunk's text on that page.
coverage	float | None	The winning page's containment score in [0, 1] (also set on no_match — the score that stayed below the floor).

Source code in varagity/api/schemas.py

class PreviewLocateResponse(BaseModel):
    """Where a chunk's text lives in its source document (ADR-010).

    ``available=False`` + ``reason`` covers every degradable condition
    (``preview_disabled`` | ``unsupported_type`` | ``file_missing`` |
    ``file_changed`` | ``conversion_unavailable`` | ``conversion_failed`` |
    ``no_match``) — the GUI falls back to the full-text view on any of
    them, never a dead panel.

    Attributes:
        available: Whether a page was located (the fields below are set).
        reason: The degradable condition when ``available`` is false.
        page: Best-matching page, 1-based.
        page_count: Total pages in the document (also set on ``no_match``).
        rects: Highlight rectangles for the chunk's text on that page.
        coverage: The winning page's containment score in ``[0, 1]`` (also
            set on ``no_match`` — the score that stayed below the floor).
    """

    available: bool
    reason: str | None = None
    page: int | None = None
    page_count: int | None = None
    rects: list[PreviewRect] = []
    coverage: float | None = None

PreviewRect

Bases: BaseModel

One highlight rectangle, normalized to the page ([0, 1], top-left origin).

Y-flipped from PDF coordinates server-side, so the client positions overlay divs with bare percentages and no coordinate math.

Attributes:

Name	Type	Description
x0	float	Left edge.
y0	float	Top edge.
x1	float	Right edge (> x0).
y1	float	Bottom edge (> y0).

Source code in varagity/api/schemas.py

class PreviewRect(BaseModel):
    """One highlight rectangle, normalized to the page (``[0, 1]``, top-left origin).

    Y-flipped from PDF coordinates server-side, so the client positions
    overlay divs with bare percentages and no coordinate math.

    Attributes:
        x0: Left edge.
        y0: Top edge.
        x1: Right edge (``> x0``).
        y1: Bottom edge (``> y0``).
    """

    x0: float
    y0: float
    x1: float
    y1: float

RetrievalEvent

Bases: BaseModel

Payload of the SSE retrieval event — the provenance panel's data.

Emitted once retrieval (+rerank) completes, before any answer token (spec_v2 §4.3): the browser gets the evidence before the prose. One event name covers both corpora (spec_graphrag §4.3): a graph turn fills graph and leaves chunks empty, and a graph turn that degraded to a chunk answer is visibly exactly that — corpus="graph" with graph=null and chunks present (ADR-017's degrade semantics).

Attributes:

Name	Type	Description
chunks	list[RetrievedChunk]	The retrieved chunks, best first, each carrying its full metadata record and (when the method fills it) the :class:~varagity.stores.records.RetrievalTrace. Empty for a graph turn.
method	str	What actually retrieved, in the corpus's own vocabulary: the retrieval-method registry name for a chunk turn, the engine query mode for a graph one (also in graph.mode).
top_k	int	Chunks requested from the retriever — a chunk-retrieval knob, reported unchanged on a graph turn, which bounds its evidence engine-side instead.
reranked_to	int | None	RERANK_TOP_N when the reranked method narrowed the list; None otherwise.
condensed_query	str | None	The standalone search query the chat engine rewrote this turn into (spec_v3 §4.7) — retrieval metadata, like method: it names what was actually searched. None whenever the search used the user's words verbatim (the simple engine, a first turn, the kill switch, or the condense fallback). Applies to both corpora: the condensed words are what the graph engine searched too.
corpus	str	The corpus the turn asked for ("rag" | "graph").
graph	GraphRetrievalPayload | None	The graph evidence, present only on a graph turn that the engine actually answered.

Source code in varagity/api/schemas.py

class RetrievalEvent(BaseModel):
    """Payload of the SSE ``retrieval`` event — the provenance panel's data.

    Emitted once retrieval (+rerank) completes, **before** any answer token
    (spec_v2 §4.3): the browser gets the evidence before the prose. One
    event name covers both corpora (spec_graphrag §4.3): a graph turn fills
    ``graph`` and leaves ``chunks`` empty, and a graph turn that *degraded*
    to a chunk answer is visibly exactly that — ``corpus="graph"`` with
    ``graph=null`` and chunks present (ADR-017's degrade semantics).

    Attributes:
        chunks: The retrieved chunks, best first, each carrying its full
            metadata record and (when the method fills it) the
            :class:`~varagity.stores.records.RetrievalTrace`. Empty for a
            graph turn.
        method: What actually retrieved, in the corpus's own vocabulary: the
            retrieval-method registry name for a chunk turn, the engine
            query mode for a graph one (also in ``graph.mode``).
        top_k: Chunks requested from the retriever — a chunk-retrieval knob,
            reported unchanged on a graph turn, which bounds its evidence
            engine-side instead.
        reranked_to: ``RERANK_TOP_N`` when the ``reranked`` method narrowed
            the list; ``None`` otherwise.
        condensed_query: The standalone search query the chat engine
            rewrote this turn into (spec_v3 §4.7) — retrieval metadata,
            like ``method``: it names what was actually searched.
            ``None`` whenever the search used the user's words verbatim
            (the ``simple`` engine, a first turn, the kill switch, or the
            condense fallback). Applies to both corpora: the condensed
            words are what the graph engine searched too.
        corpus: The corpus the turn asked for (``"rag"`` | ``"graph"``).
        graph: The graph evidence, present only on a graph turn that the
            engine actually answered.
    """

    chunks: list[RetrievedChunk]
    method: str
    top_k: int
    reranked_to: int | None = None
    condensed_query: str | None = None
    corpus: str = "rag"
    graph: GraphRetrievalPayload | None = None

ServiceHealth

Bases: BaseModel

Reachability of one backing service (spec_v2 §4.2).

Attributes:

Name	Type	Description
ok	bool	Whether the service answered its probe.
detail	str | None	Failure detail when it didn't (None when healthy).

Source code in varagity/api/schemas.py

class ServiceHealth(BaseModel):
    """Reachability of one backing service (spec_v2 §4.2).

    Attributes:
        ok: Whether the service answered its probe.
        detail: Failure detail when it didn't (``None`` when healthy).
    """

    ok: bool
    detail: str | None = None

SettingOut

Bases: BaseModel

One effective setting in the GET /api/settings catalog (spec_v2 §4.7).

Attributes:

Name	Type	Description
name	str	The Settings field name (e.g. "RETRIEVAL_METHOD").
value	SettingValue	The effective value (env defaults merged with any override).
group	str	Drawer group — retrieval | generation | ingestion.
overridden	bool	Whether a persisted runtime override is in effect (the drawer shows a reset affordance).
reingest_affecting	bool	Whether changing it marks the corpus stale (it doesn't change content hashes — the surfaced v1 footgun).
choices	list[str] | None	Valid values for enum-like settings (registry-derived, so a new implementation appears automatically); None for numeric and free-form settings (ranges live in GET /api/config).

Source code in varagity/api/schemas.py

class SettingOut(BaseModel):
    """One effective setting in the ``GET /api/settings`` catalog (spec_v2 §4.7).

    Attributes:
        name: The ``Settings`` field name (e.g. ``"RETRIEVAL_METHOD"``).
        value: The effective value (env defaults merged with any override).
        group: Drawer group — ``retrieval`` | ``generation`` | ``ingestion``.
        overridden: Whether a persisted runtime override is in effect (the
            drawer shows a reset affordance).
        reingest_affecting: Whether changing it marks the corpus stale (it
            doesn't change content hashes — the surfaced v1 footgun).
        choices: Valid values for enum-like settings (registry-derived, so a
            new implementation appears automatically); ``None`` for numeric
            and free-form settings (ranges live in ``GET /api/config``).
    """

    name: str
    value: SettingValue
    group: str
    overridden: bool
    reingest_affecting: bool
    choices: list[str] | None = None

SettingsPatchRequest

Bases: BaseModel

Body of PATCH /api/settings.

Attributes:

Name	Type	Description
overrides	dict[str, SettingValue | None]	Setting name → new value, or None to clear that override (reverting to the env value). Linked settings (the fusion weight pair) must be patched together — validation runs on the merged whole.

Source code in varagity/api/schemas.py

class SettingsPatchRequest(BaseModel):
    """Body of ``PATCH /api/settings``.

    Attributes:
        overrides: Setting name → new value, or ``None`` to clear that
            override (reverting to the env value). Linked settings (the
            fusion weight pair) must be patched together — validation runs
            on the merged whole.
    """

    model_config = ConfigDict(extra="forbid")

    overrides: dict[str, SettingValue | None] = Field(min_length=1)

SettingsResponse

Bases: BaseModel

Response of GET /api/settings and PATCH /api/settings.

Attributes:

Name	Type	Description
settings	list[SettingOut]	The full overridable catalog with effective values.
corpus_stale	bool	Whether a reingest-affecting setting changed since the corpus was last (re)ingested — the "Re-ingest to apply" banner.
graph_stale	bool	Whether the graph corpus changed since the graph was last (re)built — the graph tab's "Re-build to apply" banner (stage-2 decision #16).

Source code in varagity/api/schemas.py

class SettingsResponse(BaseModel):
    """Response of ``GET /api/settings`` and ``PATCH /api/settings``.

    Attributes:
        settings: The full overridable catalog with effective values.
        corpus_stale: Whether a reingest-affecting setting changed since the
            corpus was last (re)ingested — the "Re-ingest to apply" banner.
        graph_stale: Whether the graph corpus changed since the graph was
            last (re)built — the graph tab's "Re-build to apply" banner
            (stage-2 decision #16).
    """

    settings: list[SettingOut]
    corpus_stale: bool
    graph_stale: bool = False

StatsEvent

Bases: BaseModel

Payload of the SSE stats event — live decode throughput.

Emitted while generation runs, interleaved with the reasoning/ token deltas and throttled server-side (the model server reports these counters on every chunk; a frame per token would be noise). Purely additive to the protocol: the event only exists when the model server reports its own timings — llama.cpp does, so the readout is llama.cpp-only by construction rather than by configuration.

Attributes:

Name	Type	Description
tokens_per_second	float	Decode throughput so far, averaged over the whole generation (cumulative, not instantaneous — it settles rather than jitters).
completion_tokens	int	Tokens decoded so far, the denominator behind that average.

Source code in varagity/api/schemas.py

class StatsEvent(BaseModel):
    """Payload of the SSE ``stats`` event — live decode throughput.

    Emitted while generation runs, interleaved with the ``reasoning``/
    ``token`` deltas and throttled server-side (the model server reports
    these counters on *every* chunk; a frame per token would be noise).
    Purely additive to the protocol: the event only exists when the model
    server reports its own timings — llama.cpp does, so the readout is
    llama.cpp-only by construction rather than by configuration.

    Attributes:
        tokens_per_second: Decode throughput so far, averaged over the
            whole generation (cumulative, not instantaneous — it settles
            rather than jitters).
        completion_tokens: Tokens decoded so far, the denominator behind
            that average.
    """

    tokens_per_second: float
    completion_tokens: int

UploadResponse

Bases: BaseModel

Response of POST /api/documents.

Attributes:

Name	Type	Description
files	list[UploadedFileOut]	Per-file outcomes, in upload order.

Source code in varagity/api/schemas.py

class UploadResponse(BaseModel):
    """Response of ``POST /api/documents``.

    Attributes:
        files: Per-file outcomes, in upload order.
    """

    files: list[UploadedFileOut]

UploadedFileOut

Bases: BaseModel

Outcome for one file of a POST /api/documents upload.

Attributes:

Name	Type	Description
file_name	str	The stored (sanitized) file name (rejections echo the client-supplied name/path they rejected).
size_bytes	int	Bytes written (0 when rejected).
stored	bool	Whether the file landed in DOCS_PATH.
replaced	bool	Whether an existing file at the same target was overwritten (a re-upload; the next ingest re-processes it under a new hash).
reason	str | None	Rejection reason when stored is false (extension_not_allowed | file_too_large | invalid_filename | invalid_path | path_too_deep | write_failed — the last is a server-side problem, escalated to a structured 500 when no file in the batch landed). Graph corpus uploads add unsupported_graph_source: the file landed, no registered message source claimed it, and it was removed again (spec_graphrag §10.2). Both corpora refuse reserved_path: a path under — or a flat file named — the reserved varagity-exports backup folder (ADR-019).
relative_path	str | None	The stored path relative to DOCS_PATH when the upload declared one (folder uploads, spec_v3 §5.2); None for flat uploads and rejections.

Source code in varagity/api/schemas.py

class UploadedFileOut(BaseModel):
    """Outcome for one file of a ``POST /api/documents`` upload.

    Attributes:
        file_name: The stored (sanitized) file name (rejections echo the
            client-supplied name/path they rejected).
        size_bytes: Bytes written (``0`` when rejected).
        stored: Whether the file landed in ``DOCS_PATH``.
        replaced: Whether an existing file at the same target was
            overwritten (a re-upload; the next ingest re-processes it under
            a new hash).
        reason: Rejection reason when ``stored`` is false
            (``extension_not_allowed`` | ``file_too_large`` |
            ``invalid_filename`` | ``invalid_path`` | ``path_too_deep`` |
            ``write_failed`` — the last is a server-side problem, escalated
            to a structured ``500`` when no file in the batch landed).
            Graph corpus uploads add ``unsupported_graph_source``: the file
            landed, no registered message source claimed it, and it was
            removed again (spec_graphrag §10.2). Both corpora refuse
            ``reserved_path``: a path under — or a flat file named — the
            reserved ``varagity-exports`` backup folder (ADR-019).
        relative_path: The stored path relative to ``DOCS_PATH`` when the
            upload declared one (folder uploads, spec_v3 §5.2); ``None``
            for flat uploads and rejections.
    """

    file_name: str
    size_bytes: int
    stored: bool
    replaced: bool = False
    reason: str | None = None
    relative_path: str | None = None

UsageInfo

Bases: BaseModel

Token usage and per-stage latency reported by the done event.

Attributes:

Name	Type	Description
prompt_tokens	int | None	Server-reported prompt tokens (None when the model server reports no usage).
completion_tokens	int | None	Server-reported completion tokens (None when unreported).
latency_ms	dict[str, int]	Wall-clock milliseconds per stage: retrieval, generation, total.
tokens_per_second	float | None	Final decode throughput as the model server measured it, or None when it reports no timings. Distinct from completion_tokens / latency_ms["generation"], which would smear queueing and prompt eval into the rate.

Source code in varagity/api/schemas.py

class UsageInfo(BaseModel):
    """Token usage and per-stage latency reported by the ``done`` event.

    Attributes:
        prompt_tokens: Server-reported prompt tokens (``None`` when the
            model server reports no usage).
        completion_tokens: Server-reported completion tokens (``None`` when
            unreported).
        latency_ms: Wall-clock milliseconds per stage: ``retrieval``,
            ``generation``, ``total``.
        tokens_per_second: Final decode throughput as the model server
            measured it, or ``None`` when it reports no timings. Distinct
            from ``completion_tokens / latency_ms["generation"]``, which
            would smear queueing and prompt eval into the rate.
    """

    prompt_tokens: int | None = None
    completion_tokens: int | None = None
    latency_ms: dict[str, int]
    tokens_per_second: float | None = None

varagity.api.streaming

SSE event framing and the flow→stream bridge (spec_v2 §4.3).

Two concerns live here:

• Typed framing — one constructor per event in the chat protocol (retrieval, reasoning, token, stats, done, error), each pairing the event name with its :mod:varagity.api.schemas payload, so routes can't emit a misnamed or mistyped frame.
• The bridge — the query pipeline is synchronous and runs in a worker thread (spec_v2 §4.1: async at the edge, sync flows underneath), while the SSE response is an async generator on the event loop. :class:EventBridge carries frames from the flow's callbacks to the generator (thread-safe, order-preserving) and carries the client's disconnect back to the flow as an abort flag polled between tokens.

EventBridge

Order-preserving conduit between a worker-thread flow and an SSE stream.

Built on the running event loop; the flow's callbacks call :meth:emit from their thread (marshalled with call_soon_threadsafe), the async generator drains :meth:events, and the stream side signals the flow to stop with :meth:abort (polled via :meth:should_abort between tokens — how a closed tab frees the GPU, spec_v2 §4.3 cancellation).

Must be constructed on the event loop thread.

Source code in varagity/api/streaming.py

class EventBridge:
    """Order-preserving conduit between a worker-thread flow and an SSE stream.

    Built on the running event loop; the flow's callbacks call :meth:`emit`
    from their thread (marshalled with ``call_soon_threadsafe``), the async
    generator drains :meth:`events`, and the stream side signals the flow to
    stop with :meth:`abort` (polled via :meth:`should_abort` between
    tokens — how a closed tab frees the GPU, spec_v2 §4.3 cancellation).

    Must be constructed on the event loop thread.
    """

    def __init__(self) -> None:
        """Capture the running loop and start with an empty queue."""
        self._loop = asyncio.get_running_loop()
        self._queue: asyncio.Queue[ServerSentEvent | None] = asyncio.Queue()
        self._abort = threading.Event()

    def emit(self, event: str, payload: BaseModel) -> None:
        """Enqueue one frame from any thread.

        Args:
            event: The SSE event name.
            payload: The event's typed payload.
        """
        self._send(ServerSentEvent(event=event, data=payload))

    def emit_frame(self, frame: ServerSentEvent) -> None:
        """Enqueue an already-framed event from any thread.

        Args:
            frame: The framed event.
        """
        self._send(frame)

    def close(self) -> None:
        """Signal end-of-stream; :meth:`events` finishes after the backlog."""
        self._send(None)

    def abort(self) -> None:
        """Ask the producing flow to stop at its next between-tokens check."""
        self._abort.set()

    def should_abort(self) -> bool:
        """Whether :meth:`abort` was called (the flow's poll seam).

        Returns:
            ``True`` once the stream side requested a stop.
        """
        return self._abort.is_set()

    async def events(self) -> AsyncIterator[ServerSentEvent]:
        """Drain frames in emission order until :meth:`close`.

        Yields:
            Each enqueued frame.
        """
        while (frame := await self._queue.get()) is not None:
            yield frame

    def _send(self, item: ServerSentEvent | None) -> None:
        """Marshal one queue item onto the loop, tolerating shutdown races.

        Args:
            item: The frame, or ``None`` as the end-of-stream sentinel.
        """
        try:
            self._loop.call_soon_threadsafe(self._queue.put_nowait, item)
        except RuntimeError:  # loop already closed (shutdown) — nothing to deliver to
            logger.debug("dropped an SSE frame emitted after the event loop closed")

__init__()

Capture the running loop and start with an empty queue.

Source code in varagity/api/streaming.py

def __init__(self) -> None:
    """Capture the running loop and start with an empty queue."""
    self._loop = asyncio.get_running_loop()
    self._queue: asyncio.Queue[ServerSentEvent | None] = asyncio.Queue()
    self._abort = threading.Event()

abort()

Ask the producing flow to stop at its next between-tokens check.

Source code in varagity/api/streaming.py

def abort(self) -> None:
    """Ask the producing flow to stop at its next between-tokens check."""
    self._abort.set()

close()

Signal end-of-stream; :meth:events finishes after the backlog.

Source code in varagity/api/streaming.py

def close(self) -> None:
    """Signal end-of-stream; :meth:`events` finishes after the backlog."""
    self._send(None)

emit(event, payload)

Enqueue one frame from any thread.

Parameters:

Name	Type	Description	Default
event	str	The SSE event name.	required
payload	BaseModel	The event's typed payload.	required

Source code in varagity/api/streaming.py

def emit(self, event: str, payload: BaseModel) -> None:
    """Enqueue one frame from any thread.

    Args:
        event: The SSE event name.
        payload: The event's typed payload.
    """
    self._send(ServerSentEvent(event=event, data=payload))

emit_frame(frame)

Enqueue an already-framed event from any thread.

Parameters:

Name	Type	Description	Default
frame	ServerSentEvent	The framed event.	required

Source code in varagity/api/streaming.py

def emit_frame(self, frame: ServerSentEvent) -> None:
    """Enqueue an already-framed event from any thread.

    Args:
        frame: The framed event.
    """
    self._send(frame)

events() async

Drain frames in emission order until :meth:close.

Yields:

Type	Description
AsyncIterator[ServerSentEvent]	Each enqueued frame.

Source code in varagity/api/streaming.py

async def events(self) -> AsyncIterator[ServerSentEvent]:
    """Drain frames in emission order until :meth:`close`.

    Yields:
        Each enqueued frame.
    """
    while (frame := await self._queue.get()) is not None:
        yield frame

should_abort()

Whether :meth:abort was called (the flow's poll seam).

Returns:

Type	Description
bool	True once the stream side requested a stop.

Source code in varagity/api/streaming.py

def should_abort(self) -> bool:
    """Whether :meth:`abort` was called (the flow's poll seam).

    Returns:
        ``True`` once the stream side requested a stop.
    """
    return self._abort.is_set()

delta_event(kind, text)

Frame one classified text fragment as reasoning or token.

Parameters:

Name	Type	Description	Default
kind	str	The splitter's classification — "reasoning" maps to the reasoning event, "answer" to token.	required
text	str	The fragment.	required

Returns:

Type	Description
ServerSentEvent	The framed event.

Source code in varagity/api/streaming.py

def delta_event(kind: str, text: str) -> ServerSentEvent:
    """Frame one classified text fragment as ``reasoning`` or ``token``.

    Args:
        kind: The splitter's classification — ``"reasoning"`` maps to the
            ``reasoning`` event, ``"answer"`` to ``token``.
        text: The fragment.

    Returns:
        The framed event.
    """
    event = EVENT_REASONING if kind == "reasoning" else EVENT_TOKEN
    return ServerSentEvent(event=event, data=DeltaEvent(delta=text))

done_event(payload)

Frame the terminal done event.

Parameters:

Name	Type	Description	Default
payload	DoneEvent	Message/conversation ids, the full answer, and usage.	required

Returns:

Type	Description
ServerSentEvent	The framed event.

Source code in varagity/api/streaming.py

def done_event(payload: DoneEvent) -> ServerSentEvent:
    """Frame the terminal ``done`` event.

    Args:
        payload: Message/conversation ids, the full answer, and usage.

    Returns:
        The framed event.
    """
    return ServerSentEvent(event=EVENT_DONE, data=payload)

error_event(code, message)

Frame an in-band error event (mid-stream failure).

Parameters:

Name	Type	Description	Default
code	str	Stable, machine-readable code.	required
message	str	Human-readable detail.	required

Returns:

Type	Description
ServerSentEvent	The framed event.

Source code in varagity/api/streaming.py

def error_event(code: str, message: str) -> ServerSentEvent:
    """Frame an in-band ``error`` event (mid-stream failure).

    Args:
        code: Stable, machine-readable code.
        message: Human-readable detail.

    Returns:
        The framed event.
    """
    return ServerSentEvent(event=EVENT_ERROR, data=ErrorEvent(code=code, message=message))

retrieval_event(payload)

Frame the provenance payload as the retrieval event.

Parameters:

Name	Type	Description	Default
payload	RetrievalEvent	The retrieved chunks and method metadata.	required

Returns:

Type	Description
ServerSentEvent	The framed event.

Source code in varagity/api/streaming.py

def retrieval_event(payload: RetrievalEvent) -> ServerSentEvent:
    """Frame the provenance payload as the ``retrieval`` event.

    Args:
        payload: The retrieved chunks and method metadata.

    Returns:
        The framed event.
    """
    return ServerSentEvent(event=EVENT_RETRIEVAL, data=payload)

stats_event(payload)

Frame one live-throughput reading as the stats event.

Parameters:

Name	Type	Description	Default
payload	StatsEvent	The decode throughput so far and its token count.	required

Returns:

Type	Description
ServerSentEvent	The framed event.

Source code in varagity/api/streaming.py

def stats_event(payload: StatsEvent) -> ServerSentEvent:
    """Frame one live-throughput reading as the ``stats`` event.

    Args:
        payload: The decode throughput so far and its token count.

    Returns:
        The framed event.
    """
    return ServerSentEvent(event=EVENT_STATS, data=payload)

varagity.api.deps

FastAPI dependencies: providers, health probes, and the chat preflight.

Providers are deliberately thin functions so tests swap them via app.dependency_overrides — the same seam the flows expose with their injectable retriever/llm parameters. The health probes back both GET /api/health (report) and the chat preflight (clean 503 with a machine-readable code before the SSE stream opens — once the 200 headers flush, status can't change; spec_v2 plan decision #7).

It is also where a corpus's write gate turns into HTTP (ADR-019): the one 409 corpus_busy envelope every start and delete route answers with, and the 503 corpus_restoring a reader gets while a backup restore replaces the corpus it reads — :func:require_rag_readable is that refusal as a dependency, for every route that queries documents/chunks.

check_services(names) async

Probe the named backing services concurrently.

Parameters:

Name	Type	Description	Default
names	tuple[str, ...]	Service names among llamacpp, infinity, postgres, elasticsearch, prefect.	required

Returns:

Type	Description
dict[str, ServiceHealth]	Probe outcome per requested name.

Raises:

Type	Description
KeyError	If a name is not a known service.

Source code in varagity/api/deps.py

async def check_services(names: tuple[str, ...]) -> dict[str, ServiceHealth]:
    """Probe the named backing services concurrently.

    Args:
        names: Service names among ``llamacpp``, ``infinity``, ``postgres``,
            ``elasticsearch``, ``prefect``.

    Returns:
        Probe outcome per requested name.

    Raises:
        KeyError: If a name is not a known service.
    """
    settings = get_settings()
    # /v1/models is the OpenAI surface's cheapest GET; llama.cpp's and
    # infinity's root /health endpoints sit outside the configured /v1
    # base URLs, so probing through them avoids URL surgery.
    urls: dict[str, tuple[str, dict[str, str] | None]] = {
        "llamacpp": (f"{settings.BASE_MODEL_API_URL.rstrip('/')}/models", None),
        "infinity": (
            f"{settings.EMBEDDING_API_URL.rstrip('/')}/models",
            {"Authorization": f"Bearer {settings.EMBEDDING_API_KEY}"},
        ),
        "elasticsearch": (f"{settings.ELASTICSEARCH_URL.rstrip('/')}/_cluster/health", None),
        "prefect": (f"{settings.PREFECT_API_URL.rstrip('/')}/health", None),
    }

    async def run_one(name: str, client: httpx.AsyncClient) -> ServiceHealth:
        if name == "postgres":
            return await asyncio.to_thread(_probe_postgres)
        url, headers = urls[name]
        return await _probe_http(client, url, headers=headers)

    async with httpx.AsyncClient(timeout=PROBE_TIMEOUT_SECONDS) as client:
        results = await asyncio.gather(*(run_one(name, client) for name in names))
    statuses = dict(zip(names, results, strict=True))
    # Every probe refreshes the reachability gauge (spec_v2 §6.2); in
    # compose the api container's own healthcheck calls /api/health every
    # 15 s, so the gauge stays current without a dedicated poller.
    for name, health in statuses.items():
        metrics.set_dependency_up(name, health.ok)
    return statuses

corpus_busy_error(error)

Build the structured 409 corpus_busy for a corpus another operation holds.

The one envelope every route uses — an ingest or build start, a backup start, a document or graph-source delete — so the code and the message (which names the holder) never drift between them.

Parameters:

Name	Type	Description	Default
error	CorpusBusy	The gate's refusal.	required

Returns:

Type	Description
HTTPException	The 409 corpus_busy to raise.

Source code in varagity/api/deps.py

def corpus_busy_error(error: CorpusBusy) -> HTTPException:
    """Build the structured ``409 corpus_busy`` for a corpus another operation holds.

    The one envelope every route uses — an ingest or build start, a backup
    start, a document or graph-source delete — so the code and the message
    (which names the holder) never drift between them.

    Args:
        error: The gate's refusal.

    Returns:
        The ``409 corpus_busy`` to raise.
    """
    return HTTPException(status_code=409, detail={"code": "corpus_busy", "message": str(error)})

get_app_settings_store()

Provide a per-request app-settings store, closed after the response.

Yields:

Type	Description
AppSettingsStore	A connected store.

Raises:

Type	Description
HTTPException	503 postgres_unreachable when the database is down.

Source code in varagity/api/deps.py

def get_app_settings_store() -> Iterator[AppSettingsStore]:
    """Provide a per-request app-settings store, closed after the response.

    Yields:
        A connected store.

    Raises:
        HTTPException: ``503 postgres_unreachable`` when the database is
            down.
    """
    try:
        store = AppSettingsStore()
    except psycopg.OperationalError as error:
        raise _unreachable("postgres", str(error)) from error
    try:
        yield store
    finally:
        store.close()

get_backup_preflight()

Provide the backup reachability preflight (override seam for tests).

Returns:

Type	Description
Callable[[str, str], Awaitable[None]]	func:require_backup_services, called with (corpus, operation).

Source code in varagity/api/deps.py

def get_backup_preflight() -> Callable[[str, str], Awaitable[None]]:
    """Provide the backup reachability preflight (override seam for tests).

    Returns:
        :func:`require_backup_services`, called with ``(corpus, operation)``.
    """
    return require_backup_services

get_bm25_store()

Provide a per-request BM25 store, closed after the response.

The Elasticsearch client performs no I/O at construction, so unreachability surfaces on the operation itself — routes map those failures to the structured 503 es_unreachable.

Yields:

Type	Description
ElasticsearchBM25	A configured store.

Source code in varagity/api/deps.py

def get_bm25_store() -> Iterator[ElasticsearchBM25]:
    """Provide a per-request BM25 store, closed after the response.

    The Elasticsearch client performs no I/O at construction, so
    unreachability surfaces on the operation itself — routes map those
    failures to the structured ``503 es_unreachable``.

    Yields:
        A configured store.
    """
    store = ElasticsearchBM25()
    try:
        yield store
    finally:
        store.close()

get_chat_engine_resolver()

Provide the chat-engine resolver (override seam for tests).

Returns:

Type	Description
Callable[[str], ChatEngine]	func:varagity.chat.get_chat_engine.

Source code in varagity/api/deps.py

def get_chat_engine_resolver() -> Callable[[str], ChatEngine]:
    """Provide the chat-engine resolver (override seam for tests).

    Returns:
        :func:`varagity.chat.get_chat_engine`.
    """
    return get_chat_engine

get_conversation_store()

Provide a per-request conversation store, closed after the response.

Yields:

Type	Description
ConversationStore	A connected store.

Raises:

Type	Description
HTTPException	503 postgres_unreachable when the database is down (the structured-error shape the GUI banners on).

Source code in varagity/api/deps.py

def get_conversation_store() -> Iterator[ConversationStore]:
    """Provide a per-request conversation store, closed after the response.

    Yields:
        A connected store.

    Raises:
        HTTPException: ``503 postgres_unreachable`` when the database is
            down (the structured-error shape the GUI banners on).
    """
    try:
        store = ConversationStore()
    except psycopg.OperationalError as error:
        raise _unreachable("postgres", str(error)) from error
    try:
        yield store
    finally:
        store.close()

get_conversation_store_factory()

Provide a store factory for routes that connect off the event loop.

The chat route persists from a worker thread after the stream ends — a per-request dependency handle would pin the connection open for the stream's whole lifetime instead of the write's.

Returns:

Type	Description
Callable[[], ConversationStore]	A zero-argument :class:ConversationStore constructor.

Source code in varagity/api/deps.py

def get_conversation_store_factory() -> Callable[[], ConversationStore]:
    """Provide a store *factory* for routes that connect off the event loop.

    The chat route persists from a worker thread after the stream ends —
    a per-request dependency handle would pin the connection open for the
    stream's whole lifetime instead of the write's.

    Returns:
        A zero-argument :class:`ConversationStore` constructor.
    """
    return ConversationStore

get_graph_build_preflight()

Provide the graph-build reachability preflight (override seam for tests).

Returns:

Type	Description
Callable[[], Awaitable[None]]	func:require_graph_build_services.

Source code in varagity/api/deps.py

def get_graph_build_preflight() -> Callable[[], Awaitable[None]]:
    """Provide the graph-build reachability preflight (override seam for tests).

    Returns:
        :func:`require_graph_build_services`.
    """
    return require_graph_build_services

get_graph_chat_preflight()

Provide the graph-chat reachability preflight (override seam for tests).

Returns:

Type	Description
Callable[[], Awaitable[None]]	func:require_graph_chat_services.

Source code in varagity/api/deps.py

def get_graph_chat_preflight() -> Callable[[], Awaitable[None]]:
    """Provide the graph-chat reachability preflight (override seam for tests).

    Returns:
        :func:`require_graph_chat_services`.
    """
    return require_graph_chat_services

get_ingest_preflight()

Provide the ingest reachability preflight (override seam for tests).

Returns:

Type	Description
Callable[[], Awaitable[None]]	func:require_ingest_services.

Source code in varagity/api/deps.py

def get_ingest_preflight() -> Callable[[], Awaitable[None]]:
    """Provide the ingest reachability preflight (override seam for tests).

    Returns:
        :func:`require_ingest_services`.
    """
    return require_ingest_services

get_llm()

Provide the chat LLM client (override seam for tests).

Returns:

Type	Description
LLMClient	The registry client for settings.CHAT_MODEL_TYPE (an LLM alias —
LLMClient	the config validator guarantees it's never embedding/rerank,
LLMClient	so this is always an :class:LLMClient).

Source code in varagity/api/deps.py

def get_llm() -> LLMClient:
    """Provide the chat LLM client (override seam for tests).

    Returns:
        The registry client for ``settings.CHAT_MODEL_TYPE`` (an LLM alias —
        the config validator guarantees it's never ``embedding``/``rerank``,
        so this is always an :class:`LLMClient`).
    """
    client = get_model(get_settings().CHAT_MODEL_TYPE)
    assert isinstance(client, LLMClient)  # CHAT_MODEL_TYPE is validated to the LLM aliases
    return client

get_retriever_resolver()

Provide the retrieval-method resolver (override seam for tests).

Returns:

Type	Description
Callable[[str], Retriever]	func:varagity.retrieval.get_retriever.

Source code in varagity/api/deps.py

def get_retriever_resolver() -> Callable[[str], Retriever]:
    """Provide the retrieval-method resolver (override seam for tests).

    Returns:
        :func:`varagity.retrieval.get_retriever`.
    """
    return get_retriever

get_services_preflight()

Provide the chat reachability preflight (override seam for tests).

Returns:

Type	Description
Callable[[], Awaitable[None]]	func:require_chat_services.

Source code in varagity/api/deps.py

def get_services_preflight() -> Callable[[], Awaitable[None]]:
    """Provide the chat reachability preflight (override seam for tests).

    Returns:
        :func:`require_chat_services`.
    """
    return require_chat_services

get_vector_store()

Provide a per-request vector store, closed after the response.

Backs the corpus routes (document list/count/delete — spec_v2 §4.2).

Yields:

Type	Description
ContextualVectorDB	A connected store.

Raises:

Type	Description
HTTPException	503 postgres_unreachable when the database is down.

Source code in varagity/api/deps.py

def get_vector_store() -> Iterator[ContextualVectorDB]:
    """Provide a per-request vector store, closed after the response.

    Backs the corpus routes (document list/count/delete — spec_v2 §4.2).

    Yields:
        A connected store.

    Raises:
        HTTPException: ``503 postgres_unreachable`` when the database is
            down.
    """
    try:
        store = ContextualVectorDB()
    except psycopg.OperationalError as error:
        raise _unreachable("postgres", str(error)) from error
    try:
        yield store
    finally:
        store.close()

refuse_corpus_busy(gate, operations)

Answer 409 corpus_busy while one of operations holds the corpus.

For routes that write outside any run (the deletes) and so conflict with only some of the gate's holders; checked before any store is touched, without claiming the corpus.

Parameters:

Name	Type	Description	Default
gate	CorpusGate	The corpus's write gate.	required
operations	Collection[Operation]	The holders that refuse the request.	required

Raises:

Type	Description
HTTPException	409 corpus_busy, naming the holder.

Source code in varagity/api/deps.py

def refuse_corpus_busy(gate: CorpusGate, operations: Collection[Operation]) -> None:
    """Answer ``409 corpus_busy`` while one of ``operations`` holds the corpus.

    For routes that write outside any run (the deletes) and so conflict
    with only some of the gate's holders; checked before any store is
    touched, without claiming the corpus.

    Args:
        gate: The corpus's write gate.
        operations: The holders that refuse the request.

    Raises:
        HTTPException: ``409 corpus_busy``, naming the holder.
    """
    try:
        gate.ensure_not_held_by(operations)
    except CorpusBusy as error:
        raise corpus_busy_error(error) from error

refuse_corpus_restoring(gate)

Answer 503 corpus_restoring while a backup restore holds the corpus.

A wait, not a fault: until the restore commits (RAG) or swaps the graph in, the corpus is empty, half-loaded, or about to be replaced — and a RAG restore's TRUNCATE holds both tables, so a query would hang for the whole load instead of answering.

Parameters:

Name	Type	Description	Default
gate	CorpusGate	The write gate of the corpus the request would read.	required

Raises:

Type	Description
HTTPException	503 corpus_restoring, naming the corpus.

Source code in varagity/api/deps.py

def refuse_corpus_restoring(gate: CorpusGate) -> None:
    """Answer ``503 corpus_restoring`` while a backup restore holds the corpus.

    A wait, not a fault: until the restore commits (RAG) or swaps the graph
    in, the corpus is empty, half-loaded, or about to be replaced — and a
    RAG restore's ``TRUNCATE`` holds both tables, so a query would hang for
    the whole load instead of answering.

    Args:
        gate: The write gate of the corpus the request would read.

    Raises:
        HTTPException: ``503 corpus_restoring``, naming the corpus.
    """
    if gate.holder == "restore":
        noun = _CORPUS_NOUNS.get(gate.corpus, gate.corpus)
        raise HTTPException(
            status_code=503,
            detail={
                "code": "corpus_restoring",
                "message": (
                    f"the {noun} is being restored from a backup — "
                    "ask again when the restore finishes"
                ),
            },
        )

require_backup_services(corpus, operation) async

Backup preflight: 503 if a store the operation needs is down.

Runs before POST /api/backups/{corpus}/… spawns the run, so an outage is a clean structured error instead of an instantly failed run.

Parameters:

Name	Type	Description	Default
corpus	str	The corpus being backed up (rag | graph).	required
operation	str	The backup operation (export | restore).	required

Raises:

Type	Description
HTTPException	503 naming the first unreachable service.
KeyError	If the (corpus, operation) pair has no requirements on record (a route/contract mismatch — never a user error).

Source code in varagity/api/deps.py

async def require_backup_services(corpus: str, operation: str) -> None:
    """Backup preflight: 503 if a store the operation needs is down.

    Runs before ``POST /api/backups/{corpus}/…`` spawns the run, so an
    outage is a clean structured error instead of an instantly failed run.

    Args:
        corpus: The corpus being backed up (``rag`` | ``graph``).
        operation: The backup operation (``export`` | ``restore``).

    Raises:
        HTTPException: ``503`` naming the first unreachable service.
        KeyError: If the ``(corpus, operation)`` pair has no requirements on
            record (a route/contract mismatch — never a user error).
    """
    required = BACKUP_REQUIRED_SERVICES[(corpus, operation)]
    statuses = await check_services(required)
    for name in required:
        status = statuses[name]
        if not status.ok:
            raise _unreachable(name, status.detail or "no detail")

require_chat_services() async

Chat preflight: 503 with a machine-readable code if a dependency is down.

Runs before POST /api/chat opens its stream, so an outage surfaces as a clean structured error instead of a half-stream.

Raises:

Type	Description
HTTPException	503 naming the first unreachable service.

Source code in varagity/api/deps.py

async def require_chat_services() -> None:
    """Chat preflight: 503 with a machine-readable code if a dependency is down.

    Runs before ``POST /api/chat`` opens its stream, so an outage surfaces
    as a clean structured error instead of a half-stream.

    Raises:
        HTTPException: ``503`` naming the first unreachable service.
    """
    statuses = await check_services(CHAT_REQUIRED_SERVICES)
    for name in CHAT_REQUIRED_SERVICES:
        status = statuses[name]
        if not status.ok:
            raise _unreachable(name, status.detail or "no detail")

require_graph_build_services() async

Graph-build preflight: 503 with a machine-readable code if a model service is down.

A graph build talks to the two GPU services and nothing else — the engine self-stores in its working directory, so neither postgres nor Elasticsearch is required (only the stale flag touches postgres, and that failure is logged rather than raised). Runs before POST /api/graph/build spawns a run that would otherwise fail hours in.

Raises:

Type	Description
HTTPException	503 naming the first unreachable service.

Source code in varagity/api/deps.py

async def require_graph_build_services() -> None:
    """Graph-build preflight: 503 with a machine-readable code if a model service is down.

    A graph build talks to the two GPU services and nothing else — the
    engine self-stores in its working directory, so neither postgres nor
    Elasticsearch is required (only the stale flag touches postgres, and
    that failure is logged rather than raised). Runs before ``POST
    /api/graph/build`` spawns a run that would otherwise fail hours in.

    Raises:
        HTTPException: ``503`` naming the first unreachable service.
    """
    required: tuple[str, ...] = ("llamacpp", "infinity")
    statuses = await check_services(required)
    for name in required:
        status = statuses[name]
        if not status.ok:
            raise _unreachable(name, status.detail or "no detail")

require_graph_chat_services() async

Graph-chat preflight: 503 if a service the graph turn needs is down.

Raises:

Type	Description
HTTPException	503 naming the first unreachable service among :data:GRAPH_CHAT_REQUIRED_SERVICES.

Source code in varagity/api/deps.py

async def require_graph_chat_services() -> None:
    """Graph-chat preflight: 503 if a service the graph turn needs is down.

    Raises:
        HTTPException: ``503`` naming the first unreachable service among
            :data:`GRAPH_CHAT_REQUIRED_SERVICES`.
    """
    statuses = await check_services(GRAPH_CHAT_REQUIRED_SERVICES)
    for name in GRAPH_CHAT_REQUIRED_SERVICES:
        status = statuses[name]
        if not status.ok:
            raise _unreachable(name, status.detail or "no detail")

require_ingest_services() async

Ingest preflight: 503 with a machine-readable code if a dependency is down.

Ingestion needs both stores and the embedder; the chat LLM joins only when CONTEXTUALIZE is on (the identity path never calls it). Runs before POST /api/ingest spawns the background run, so an outage is a clean structured error instead of an instantly failed run.

Raises:

Type	Description
HTTPException	503 naming the first unreachable service.

Source code in varagity/api/deps.py

async def require_ingest_services() -> None:
    """Ingest preflight: 503 with a machine-readable code if a dependency is down.

    Ingestion needs both stores and the embedder; the chat LLM joins only
    when ``CONTEXTUALIZE`` is on (the identity path never calls it). Runs
    before ``POST /api/ingest`` spawns the background run, so an outage is
    a clean structured error instead of an instantly failed run.

    Raises:
        HTTPException: ``503`` naming the first unreachable service.
    """
    required: tuple[str, ...] = ("postgres", "elasticsearch", "infinity")
    if get_settings().CONTEXTUALIZE:
        required += ("llamacpp",)
    statuses = await check_services(required)
    for name in required:
        status = statuses[name]
        if not status.ok:
            raise _unreachable(name, status.detail or "no detail")

require_rag_readable(gates)

Dependency: refuse a documents/chunks read while a RAG restore runs.

Attached to every route that queries those tables (the document list, the page previews), so none of them parks a worker thread on the restore's table lock for the length of the load.

Parameters:

Name	Type	Description	Default
gates	Annotated[dict[str, CorpusGate], Depends(get_corpus_gates)]	The process's per-corpus write gates.	required

Raises:

Type	Description
HTTPException	503 corpus_restoring while a restore holds the document corpus.

Source code in varagity/api/deps.py

def require_rag_readable(
    gates: Annotated[dict[str, CorpusGate], Depends(get_corpus_gates)],
) -> None:
    """Dependency: refuse a ``documents``/``chunks`` read while a RAG restore runs.

    Attached to every route that queries those tables (the document list,
    the page previews), so none of them parks a worker thread on the
    restore's table lock for the length of the load.

    Args:
        gates: The process's per-corpus write gates.

    Raises:
        HTTPException: ``503 corpus_restoring`` while a restore holds the
            document corpus.
    """
    refuse_corpus_restoring(gates["rag"])

varagity.api.routes.chat

POST /api/chat — the streaming question endpoint (spec_v2 §4.2, §4.3).

Reuses the pipeline, never reimplements it: the request runs :func:~varagity.pipeline.query_flow.query_stream_flow (every stage a tracked Prefect task, exactly like the CLI) in a worker thread, while this module's async generator relays the flow's callback events to the client as typed SSE frames — retrieval first (the evidence before the prose), then reasoning/token deltas, then done after the turn persists.

corpus="graph" (spec_graphrag §4.2 — the field a router will later fill) swaps the flow for :func:~varagity.pipeline.graph_flow.graph_query_stream_flow and nothing else: same protocol, same event order, same persistence, with the retrieval event carrying graph evidence instead of chunks. Whether the graph can answer is settled in :func:prepare_chat, because ADR-017's degrade (kill switch off, engine unavailable → answer from the chunk corpus, requested corpus still recorded) has to be decided before any evidence is on the wire.

Failure surfaces split by stream state: dependency outages and bad references are caught by dependencies before the stream opens (clean structured 503/404/422) — and so is a corpus mid-restore, which answers 503 corpus_restoring rather than retrieve from tables that are being replaced, or open a graph whose workdir is being swapped (ADR-019); anything after the 200 flushed is an in-band error event. A client disconnect cancels this generator, whose cleanup flips the bridge's abort flag; the flow notices between tokens and closes the LLM stream, freeing the GPU (spec_v2 §4.3 cancellation).

ChatPlan dataclass

A validated, ready-to-run chat request.

Assembled by :func:prepare_chat so every failure that can be detected before streaming is raised while the response status can still express it — including the graph degrade decision, which is settled here rather than mid-stream (ADR-017's semantics say a degraded turn answers from the chunk corpus, and by the time evidence has been emitted it is too late to change corpus).

Attributes:

Name	Type	Description
payload	ChatRequest	The validated request body.
retriever	Retriever	The resolved retrieval method.
method	str	Its registry name (recorded on the turn).
top_k	int	Chunks to retrieve (override or settings default).
reranked_to	int | None	RERANK_TOP_N when the reranked method narrows the list; None otherwise.
engine	ChatEngine	The resolved chat engine (override or settings.CHAT_ENGINE).
engine_name	str	Its registry name (recorded on the turn, like method).
history	list[Turn]	The conversation's recent turns, oldest first (empty for a fresh conversation) — the engine's condense input.
llm	LLMClient	The chat client.
store_factory	Callable[[], ConversationStore]	Conversation-store constructor (persistence runs in a worker thread with its own short-lived connection).
corpus	str	The corpus the request asked for ("rag" | "graph"), persisted on the turn whatever actually answered it.
graph_service	GraphService | None	The opened graph service when a graph turn will really run on the graph; None for a chunk turn and for a degraded graph turn (kill switch off, engine unavailable) — which is what makes graph_service is None the single "answer from chunks" test in the streaming body.
graph_mode	str	The engine query mode a graph turn retrieves with, resolved once here so a mid-turn settings PATCH cannot make the event and the persisted record disagree.

Source code in varagity/api/routes/chat.py

@dataclass
class ChatPlan:
    """A validated, ready-to-run chat request.

    Assembled by :func:`prepare_chat` so every failure that *can* be
    detected before streaming is raised while the response status can
    still express it — including the graph degrade decision, which is
    settled here rather than mid-stream (ADR-017's semantics say a degraded
    turn answers from the chunk corpus, and by the time evidence has been
    emitted it is too late to change corpus).

    Attributes:
        payload: The validated request body.
        retriever: The resolved retrieval method.
        method: Its registry name (recorded on the turn).
        top_k: Chunks to retrieve (override or settings default).
        reranked_to: ``RERANK_TOP_N`` when the ``reranked`` method narrows
            the list; ``None`` otherwise.
        engine: The resolved chat engine (override or
            ``settings.CHAT_ENGINE``).
        engine_name: Its registry name (recorded on the turn, like
            ``method``).
        history: The conversation's recent turns, oldest first (empty for
            a fresh conversation) — the engine's condense input.
        llm: The chat client.
        store_factory: Conversation-store constructor (persistence runs in
            a worker thread with its own short-lived connection).
        corpus: The corpus the request asked for (``"rag"`` | ``"graph"``),
            persisted on the turn whatever actually answered it.
        graph_service: The opened graph service when a graph turn will
            really run on the graph; ``None`` for a chunk turn **and** for a
            degraded graph turn (kill switch off, engine unavailable) —
            which is what makes ``graph_service is None`` the single "answer
            from chunks" test in the streaming body.
        graph_mode: The engine query mode a graph turn retrieves with,
            resolved once here so a mid-turn settings PATCH cannot make the
            event and the persisted record disagree.
    """

    payload: ChatRequest
    retriever: Retriever
    method: str
    top_k: int
    reranked_to: int | None
    engine: ChatEngine
    engine_name: str
    history: list[Turn]
    llm: LLMClient
    store_factory: Callable[[], ConversationStore]
    corpus: str = "rag"
    graph_service: GraphService | None = None
    graph_mode: str = ""

    @property
    def retrieval_label(self) -> str:
        """What actually retrieved, in the corpus's own vocabulary.

        Returns:
            The graph query mode for a live graph turn, else the retrieval
            method's registry name — including for a *degraded* graph turn,
            which really did retrieve chunks (:class:`RetrievalEvent`'s
            ``method`` and the persisted ``retrieval_method`` both use this).
        """
        return self.graph_mode if self.graph_service is not None else self.method

retrieval_label property

What actually retrieved, in the corpus's own vocabulary.

Returns:

Type	Description
str	The graph query mode for a live graph turn, else the retrieval
str	method's registry name — including for a degraded graph turn,
str	which really did retrieve chunks (:class:RetrievalEvent's
str	method and the persisted retrieval_method both use this).

RecordingEngine

Per-request engine wrapper that records the prepare outcome.

The SSE retrieval event carries condensed_query (spec_v3 §4.7), but the flow's on_retrieved callback only delivers chunks and the flow state only returns after the stream ends — too late. The condense stage runs strictly before retrieval in the same worker thread, so a delegating wrapper observes the :class:~varagity.chat.base.PreparedQuery exactly when the retrieval callback needs it, with no flow-signature change. Per-request by construction: registry engines are shared singletons and must not carry request state.

Source code in varagity/api/routes/chat.py

class RecordingEngine:
    """Per-request engine wrapper that records the prepare outcome.

    The SSE ``retrieval`` event carries ``condensed_query`` (spec_v3
    §4.7), but the flow's ``on_retrieved`` callback only delivers chunks
    and the flow state only returns after the stream ends — too late. The
    condense stage runs strictly before retrieval in the same worker
    thread, so a delegating wrapper observes the
    :class:`~varagity.chat.base.PreparedQuery` exactly when the retrieval
    callback needs it, with no flow-signature change. Per-request by
    construction: registry engines are shared singletons and must not
    carry request state.
    """

    def __init__(self, inner: ChatEngine) -> None:
        """Wrap one resolved engine for one request.

        Args:
            inner: The registry engine that does the actual preparing.
        """
        self._inner = inner
        self.prepared: PreparedQuery | None = None

    def prepare(
        self,
        query: str,
        *,
        history: Sequence[Turn],
        llm: LLMClient | None,
        verbose: int,
    ) -> PreparedQuery:
        """Delegate to the wrapped engine, keeping its outcome readable.

        Args:
            query: The user's question, verbatim.
            history: Prior turns, oldest first.
            llm: Chat client for engines that condense.
            verbose: Validated console verbosity.

        Returns:
            The wrapped engine's split, unchanged.
        """
        self.prepared = self._inner.prepare(query, history=history, llm=llm, verbose=verbose)
        return self.prepared

    @property
    def condensed_query(self) -> str | None:
        """The rewritten search query, or ``None`` when none was used.

        Returns:
            ``search_query`` when the engine condensed this turn; ``None``
            before the condense stage ran and whenever the search used the
            user's words verbatim.
        """
        if self.prepared is None or not self.prepared.condensed:
            return None
        return self.prepared.search_query

condensed_query property

The rewritten search query, or None when none was used.

Returns:

Type	Description
str | None	search_query when the engine condensed this turn; None
str | None	before the condense stage ran and whenever the search used the
str | None	user's words verbatim.

__init__(inner)

Wrap one resolved engine for one request.

Parameters:

Name	Type	Description	Default
inner	ChatEngine	The registry engine that does the actual preparing.	required

Source code in varagity/api/routes/chat.py

def __init__(self, inner: ChatEngine) -> None:
    """Wrap one resolved engine for one request.

    Args:
        inner: The registry engine that does the actual preparing.
    """
    self._inner = inner
    self.prepared: PreparedQuery | None = None

prepare(query, *, history, llm, verbose)

Delegate to the wrapped engine, keeping its outcome readable.

Parameters:

Name	Type	Description	Default
query	str	The user's question, verbatim.	required
history	Sequence[Turn]	Prior turns, oldest first.	required
llm	LLMClient | None	Chat client for engines that condense.	required
verbose	int	Validated console verbosity.	required

Returns:

Type	Description
PreparedQuery	The wrapped engine's split, unchanged.

Source code in varagity/api/routes/chat.py

def prepare(
    self,
    query: str,
    *,
    history: Sequence[Turn],
    llm: LLMClient | None,
    verbose: int,
) -> PreparedQuery:
    """Delegate to the wrapped engine, keeping its outcome readable.

    Args:
        query: The user's question, verbatim.
        history: Prior turns, oldest first.
        llm: Chat client for engines that condense.
        verbose: Validated console verbosity.

    Returns:
        The wrapped engine's split, unchanged.
    """
    self.prepared = self._inner.prepare(query, history=history, llm=llm, verbose=verbose)
    return self.prepared

chat(plan) async

Answer one question as a typed SSE stream (spec_v2 §4.3).

Event order: retrieval (the provenance payload) → reasoning/ token deltas, interleaved with throttled stats frames while the model server reports throughput → done (ids, full answer, usage + per-stage latency). A failure after the stream opened emits an in-band error event instead. Aborted turns (client disconnect) persist nothing — the turn is persisted at done.

stats is optional in the protocol: it appears only when the model server reports its own decode counters (llama.cpp does; see :class:~varagity.models.llm.GenerationTimings), so clients must treat zero stats frames as normal rather than as a stalled stream.

Parameters:

Name	Type	Description	Default
plan	Annotated[ChatPlan, Depends(prepare_chat)]	The validated request plan.	required

Yields:

Type	Description
AsyncIterator[ServerSentEvent]	The framed SSE events, in protocol order.

Source code in varagity/api/routes/chat.py

@router.post(
    "/api/chat",
    response_class=EventSourceResponse,
    responses={
        404: {"model": ErrorResponse},
        422: {"model": ErrorResponse},
        503: {"model": ErrorResponse},
    },
)
async def chat(plan: Annotated[ChatPlan, Depends(prepare_chat)]) -> AsyncIterator[ServerSentEvent]:
    """Answer one question as a typed SSE stream (spec_v2 §4.3).

    Event order: ``retrieval`` (the provenance payload) → ``reasoning``/
    ``token`` deltas, interleaved with throttled ``stats`` frames while the
    model server reports throughput → ``done`` (ids, full answer, usage +
    per-stage latency). A failure after the stream opened emits an in-band
    ``error`` event instead. Aborted turns (client disconnect) persist
    nothing — the turn is persisted *at* ``done``.

    ``stats`` is optional in the protocol: it appears only when the model
    server reports its own decode counters (llama.cpp does; see
    :class:`~varagity.models.llm.GenerationTimings`), so clients must treat
    zero ``stats`` frames as normal rather than as a stalled stream.

    Args:
        plan: The validated request plan.

    Yields:
        The framed SSE events, in protocol order.
    """
    bridge = EventBridge()
    started = time.monotonic()
    timings: dict[str, int] = {}
    # Observes the condense outcome for the retrieval event: the condense
    # stage runs before retrieval in the flow's worker thread, so the
    # recorder is filled by the time _on_retrieved fires.
    engine = RecordingEngine(plan.engine)

    def _elapsed_ms() -> int:
        return int((time.monotonic() - started) * 1000)

    def _emit_retrieval(
        chunks: list[RetrievedChunk], graph: GraphRetrievalPayload | None = None
    ) -> None:
        timings["retrieval"] = _elapsed_ms()
        bridge.emit_frame(
            retrieval_event(
                RetrievalEvent(
                    chunks=chunks,
                    method=plan.retrieval_label,
                    top_k=plan.top_k,
                    reranked_to=plan.reranked_to,
                    condensed_query=engine.condensed_query,
                    corpus=plan.corpus,
                    graph=graph,
                )
            )
        )

    def _on_retrieved(chunks: list[RetrievedChunk]) -> None:
        _emit_retrieval(chunks)

    def _on_graph_retrieved(retrieval: GraphRetrieval) -> None:
        _emit_retrieval([], graph_retrieval_payload(retrieval))

    def _on_delta(kind: Kind, text: str) -> None:
        bridge.emit_frame(delta_event(kind, text))

    last_stats_at = 0.0

    def _on_stats(timings: GenerationTimings) -> None:
        # Called from the flow's worker thread, once per model-server
        # chunk. Both gates live here rather than in the client: the
        # transport reports what the server said, the edge decides what is
        # worth a frame.
        nonlocal last_stats_at
        rate = timings.tokens_per_second
        if rate is None or timings.predicted_n < _STATS_WARMUP_TOKENS:
            return
        now = time.monotonic()
        if now - last_stats_at < _STATS_MIN_INTERVAL_S:
            return
        last_stats_at = now
        bridge.emit_frame(
            stats_event(StatsEvent(tokens_per_second=rate, completion_tokens=timings.predicted_n))
        )

    def _run_flow() -> _TurnOutcome:
        try:
            if plan.graph_service is not None:
                return _TurnOutcome.from_graph(
                    graph_query_stream_flow(
                        plan.payload.query,
                        plan.graph_service,
                        history=plan.history,
                        engine=engine,
                        llm=plan.llm,
                        mode=plan.graph_mode,
                        verbose=0,
                        on_retrieved=_on_graph_retrieved,
                        on_delta=_on_delta,
                        should_abort=bridge.should_abort,
                        on_stats=_on_stats,
                    )
                )
            return _TurnOutcome.from_chunks(
                query_stream_flow(
                    plan.payload.query,
                    history=plan.history,
                    engine=engine,
                    retriever=plan.retriever,
                    llm=plan.llm,
                    k=plan.top_k,
                    verbose=0,
                    on_retrieved=_on_retrieved,
                    on_delta=_on_delta,
                    should_abort=bridge.should_abort,
                    on_stats=_on_stats,
                )
            )
        finally:
            bridge.close()

    flow_future = asyncio.ensure_future(run_in_threadpool(_run_flow))
    flow_future.add_done_callback(_swallow_outcome)
    try:
        async for frame in bridge.events():
            yield frame
        outcome = await flow_future
        if outcome.aborted:
            logger.info("chat turn aborted by the client; nothing persisted")
            return

        timings["generation"] = _elapsed_ms() - timings.get("retrieval", 0)

        prepared = outcome.prepared
        retrieval = outcome.retrieval

        def _persist() -> tuple[str, str]:
            with plan.store_factory() as store:
                conversation_id = plan.payload.conversation_id
                if conversation_id is None:
                    conversation_id = store.create_conversation().conversation_id
                store.append_message(conversation_id, "user", plan.payload.query)
                message_id = store.append_message(
                    conversation_id,
                    "assistant",
                    outcome.answer,
                    retrieval_method=plan.retrieval_label,
                    latency_ms=dict(timings),
                    reasoning=outcome.reasoning or None,
                    # NULL = searched verbatim; the engine NAME is recorded
                    # even then, so a degraded condense_context turn (kill
                    # switch, fallback) stays attributable — spec_v3 §8.
                    condensed_query=(prepared.search_query if prepared.condensed else None),
                    chat_engine=plan.engine_name,
                    sources=outcome.chunks,
                    # Always the *requested* corpus, so a degraded graph turn
                    # stays attributable the same way (ADR-017); the NULL
                    # graph_evidence beside it is what says it degraded.
                    target_corpus=plan.corpus,
                    graph_evidence=None if retrieval is None else graph_evidence(retrieval),
                    graph_sources=() if retrieval is None else graph_snapshots(retrieval),
                )
                return conversation_id, message_id

        conversation_id, message_id = await run_in_threadpool(_persist)
        _auto_title_in_background(plan, conversation_id)
        timings["total"] = _elapsed_ms()
        usage = outcome.usage or {}
        yield done_event(
            DoneEvent(
                message_id=message_id,
                conversation_id=conversation_id,
                answer=outcome.answer,
                usage=UsageInfo(
                    prompt_tokens=usage.get("prompt_tokens"),
                    completion_tokens=usage.get("completion_tokens"),
                    latency_ms=dict(timings),
                    tokens_per_second=outcome.tokens_per_second,
                ),
            )
        )
    except Exception as error:
        # The 200 already flushed — failures from here are in-band frames.
        logger.exception("chat stream failed mid-flight")
        yield error_event("pipeline_error", f"{type(error).__name__}: {error}")
    finally:
        bridge.abort()

graph_evidence(retrieval)

Build the persisted messages.graph_evidence snapshot.

The entities and relations, plus the mode that found them — the same shape the live event carries, minus the transcripts, which persist as message_sources rows instead (stage-2 decision #12).

Parameters:

Name	Type	Description	Default
retrieval	GraphRetrieval	What the graph returned.	required

Returns:

Type	Description
dict[str, Any]	The JSONB-ready snapshot.

Source code in varagity/api/routes/chat.py

def graph_evidence(retrieval: GraphRetrieval) -> dict[str, Any]:
    """Build the persisted ``messages.graph_evidence`` snapshot.

    The entities and relations, plus the mode that found them — the same
    shape the live event carries, minus the transcripts, which persist as
    ``message_sources`` rows instead (stage-2 decision #12).

    Args:
        retrieval: What the graph returned.

    Returns:
        The JSONB-ready snapshot.
    """
    payload = graph_retrieval_payload(retrieval)
    return payload.model_dump(mode="json", exclude={"transcripts"})

graph_retrieval_payload(retrieval)

Map a graph retrieval onto the SSE retrieval event's graph half.

Two things are dropped on the way out, both deliberately: :attr:~varagity.graph.records.GraphEvidence.raw (the engine-native payload, kept for autopsies, not for browsers) and everything past :data:_EXCERPT_MAX_CHARS of each passage — a transcript day can run to thousands of characters and the panel shows a card, not a file.

Parameters:

Name	Type	Description	Default
retrieval	GraphRetrieval	What the graph returned.	required

Returns:

Type	Description
GraphRetrievalPayload	The wire payload.

Source code in varagity/api/routes/chat.py

def graph_retrieval_payload(retrieval: GraphRetrieval) -> GraphRetrievalPayload:
    """Map a graph retrieval onto the SSE ``retrieval`` event's graph half.

    Two things are dropped on the way out, both deliberately:
    :attr:`~varagity.graph.records.GraphEvidence.raw` (the engine-native
    payload, kept for autopsies, not for browsers) and everything past
    :data:`_EXCERPT_MAX_CHARS` of each passage — a transcript day can run to
    thousands of characters and the panel shows a card, not a file.

    Args:
        retrieval: What the graph returned.

    Returns:
        The wire payload.
    """
    return GraphRetrievalPayload(
        mode=retrieval.mode,
        entities=[
            GraphEntityOut(name=entity.name, type=entity.type, summary=entity.summary)
            for entity in retrieval.evidence.entities
        ],
        relations=[
            GraphRelationOut(
                source=relation.source,
                target=relation.target,
                label=relation.label,
                description=relation.description,
            )
            for relation in retrieval.evidence.relations
        ],
        transcripts=[
            GraphTranscriptOut(
                doc_key=excerpt.doc_key,
                thread_name=excerpt.thread_name,
                span=excerpt.span,
                excerpt=_clip(excerpt.text),
                message_count=len(excerpt.message_guids),
            )
            for excerpt in retrieval.excerpts
        ],
    )

graph_snapshots(retrieval)

Build the persisted evidence rows for the cited transcript days.

Parameters:

Name	Type	Description	Default
retrieval	GraphRetrieval	What the graph returned.	required

Returns:

Type	Description
list[GraphSourceSnapshot]	One snapshot per cited transcript day, best first.

Source code in varagity/api/routes/chat.py

def graph_snapshots(retrieval: GraphRetrieval) -> list[GraphSourceSnapshot]:
    """Build the persisted evidence rows for the cited transcript days.

    Args:
        retrieval: What the graph returned.

    Returns:
        One snapshot per cited transcript day, best first.
    """
    return [
        GraphSourceSnapshot(
            doc_key=excerpt.doc_key,
            thread_name=excerpt.thread_name,
            span=excerpt.span,
            excerpt=_clip(excerpt.text),
            message_guids=list(excerpt.message_guids),
        )
        for excerpt in retrieval.excerpts
    ]

prepare_chat(payload, llm, resolve_retriever, resolve_engine, store_factory, services_preflight, graph_service, graph_preflight, gates) async

Validate the request and resolve everything the stream will need.

Checks run cheapest-first: body shape (FastAPI, 422 — an unknown corpus never gets past it) → retrieval-method resolution (422) → chat-engine resolution (override or settings, 422) → a graph turn while the graph is mid-restore (503, before the session is opened — a lazy open could otherwise wait out the swap, or read a graph about to be replaced) → the graph degrade decision, which picks which corpus answers and so which dependencies the turn even needs → that corpus mid-restore (503) → dependency reachability (503, before the stream opens) → conversation existence (404), whose store round-trip doubles as the history load for the chat engine (bounded by CONDENSE_HISTORY_TURNS).

A graph turn resolves the chunk retriever too: it is cheap, and it is what answers the turn if the graph degrades.

Parameters:

Name	Type	Description	Default
payload	ChatRequest	The request body (FastAPI shares the parse with the route).	required
llm	Annotated[LLMClient, Depends(get_llm)]	The chat client provider.	required
resolve_retriever	Annotated[Callable[[str], Retriever], Depends(get_retriever_resolver)]	The retrieval-method resolver.	required
resolve_engine	Annotated[Callable[[str], ChatEngine], Depends(get_chat_engine_resolver)]	The chat-engine resolver.	required
store_factory	Annotated[Callable[[], ConversationStore], Depends(get_conversation_store_factory)]	The conversation-store factory.	required
services_preflight	Annotated[Callable[[], Awaitable[None]], Depends(get_services_preflight)]	The awaitable reachability check (raises the structured 503).	required
graph_service	Annotated[GraphService, Depends(get_graph_service)]	The process-wide graph service.	required
graph_preflight	Annotated[Callable[[], Awaitable[None]], Depends(get_graph_chat_preflight)]	The graph turn's reachability check (Elasticsearch is not on that path).	required
gates	Annotated[dict[str, CorpusGate], Depends(get_corpus_gates)]	The process's per-corpus write gates (a restore holds its corpus's gate for the whole run).	required

Returns:

Type	Description
ChatPlan	The assembled plan.

Raises:

Type

Description

HTTPException

422 unknown_retrieval_method / 422 unknown_chat_engine for an override naming no registered implementation; 503 corpus_restoring while a backup restore replaces the corpus that would answer (a graph turn the graph answers is unaffected by a document restore, and a document turn by a graph restore); 503 <service>_unreachable for a down dependency; 404 conversation_not_found for an unknown conversation_id.

Source code in varagity/api/routes/chat.py

async def prepare_chat(
    payload: ChatRequest,
    llm: Annotated[LLMClient, Depends(get_llm)],
    resolve_retriever: Annotated[Callable[[str], Retriever], Depends(get_retriever_resolver)],
    resolve_engine: Annotated[Callable[[str], ChatEngine], Depends(get_chat_engine_resolver)],
    store_factory: Annotated[
        Callable[[], ConversationStore], Depends(get_conversation_store_factory)
    ],
    services_preflight: Annotated[Callable[[], Awaitable[None]], Depends(get_services_preflight)],
    graph_service: Annotated[GraphService, Depends(get_graph_service)],
    graph_preflight: Annotated[Callable[[], Awaitable[None]], Depends(get_graph_chat_preflight)],
    gates: Annotated[dict[str, CorpusGate], Depends(get_corpus_gates)],
) -> ChatPlan:
    """Validate the request and resolve everything the stream will need.

    Checks run cheapest-first: body shape (FastAPI, 422 — an unknown
    ``corpus`` never gets past it) → retrieval-method resolution (422) →
    chat-engine resolution (override or settings, 422) → a graph turn while
    the graph is mid-restore (503, before the session is opened — a lazy
    open could otherwise wait out the swap, or read a graph about to be
    replaced) → the graph degrade decision, which picks *which* corpus
    answers and so which dependencies the turn even needs → that corpus
    mid-restore (503) → dependency reachability (503, *before* the stream
    opens) → conversation existence (404), whose store round-trip doubles as
    the history load for the chat engine (bounded by
    ``CONDENSE_HISTORY_TURNS``).

    A graph turn resolves the chunk retriever too: it is cheap, and it is
    what answers the turn if the graph degrades.

    Args:
        payload: The request body (FastAPI shares the parse with the route).
        llm: The chat client provider.
        resolve_retriever: The retrieval-method resolver.
        resolve_engine: The chat-engine resolver.
        store_factory: The conversation-store factory.
        services_preflight: The awaitable reachability check (raises the
            structured 503).
        graph_service: The process-wide graph service.
        graph_preflight: The graph turn's reachability check (Elasticsearch
            is not on that path).
        gates: The process's per-corpus write gates (a restore holds its
            corpus's gate for the whole run).

    Returns:
        The assembled plan.

    Raises:
        HTTPException: ``422 unknown_retrieval_method`` /
            ``422 unknown_chat_engine`` for an override naming no
            registered implementation; ``503 corpus_restoring`` while a
            backup restore replaces the corpus that would answer (a graph
            turn the graph answers is unaffected by a document restore, and
            a document turn by a graph restore); ``503
            <service>_unreachable`` for a down dependency; ``404
            conversation_not_found`` for an unknown ``conversation_id``.
    """
    settings = get_settings()
    overrides = payload.overrides or ChatOverrides()
    method = overrides.retrieval_method or settings.RETRIEVAL_METHOD
    try:
        retriever = resolve_retriever(method)
    except KeyError as error:
        raise HTTPException(
            status_code=422,
            detail={"code": "unknown_retrieval_method", "message": str(error)},
        ) from error
    engine_name = overrides.chat_engine or settings.CHAT_ENGINE
    try:
        engine = resolve_engine(engine_name)
    except KeyError as error:
        # Reachable only through the override: the settings value is
        # validator-gated to registered names at config load.
        raise HTTPException(
            status_code=422,
            detail={"code": "unknown_chat_engine", "message": str(error)},
        ) from error

    corpus = payload.corpus or "rag"
    # A graph turn while the graph restores is refused before the session is
    # touched: the restore closes it, swaps the workdir and reopens it, and a
    # lazy open here would either wait out the swap or read a graph that is
    # about to be replaced. (With the kill switch off the turn degrades to
    # the documents without touching the graph, so only they matter then.)
    if corpus == "graph" and settings.GRAPH_ENABLED:
        refuse_corpus_restoring(gates["graph"])
    # The degrade decision comes next because it decides *what* to preflight:
    # a graph turn never touches Elasticsearch, a degraded one always does.
    # Opening the session dials no service — it initializes local storages —
    # so nothing here depends on the check that follows it.
    graph = await _open_graph(graph_service) if corpus == "graph" else None
    # Whichever corpus will actually answer — a degraded graph turn answers
    # from the documents — must not be mid-restore: its tables are empty or
    # half-loaded until the restore commits, and retrieval would either wait
    # behind the restore or answer from nothing.
    refuse_corpus_restoring(gates["graph" if graph is not None else "rag"])
    await (graph_preflight() if graph is not None else services_preflight())

    history: list[Turn] = []
    if payload.conversation_id is not None:
        conversation_id = payload.conversation_id
        history_turns = settings.CONDENSE_HISTORY_TURNS

        def _load_history() -> list[tuple[str, str]] | None:
            with store_factory() as store:
                if not store.conversation_exists(conversation_id):
                    return None
                return store.recent_turns(conversation_id, limit=history_turns)

        turns = await run_in_threadpool(_load_history)
        if turns is None:
            raise HTTPException(
                status_code=404,
                detail={
                    "code": "conversation_not_found",
                    "message": f"No conversation with id {conversation_id!r}",
                },
            )
        history = [Turn(role=role, content=content) for role, content in turns]

    return ChatPlan(
        payload=payload,
        retriever=retriever,
        method=method,
        top_k=overrides.top_k or settings.TOP_K,
        reranked_to=settings.RERANK_TOP_N if method == "reranked" else None,
        engine=engine,
        engine_name=engine_name,
        history=history,
        llm=llm,
        store_factory=store_factory,
        corpus=corpus,
        graph_service=graph,
        graph_mode=settings.GRAPH_QUERY_MODE,
    )

varagity.api.routes.conversations

Conversation CRUD routes (spec_v2 §4.2, §4.4).

Thin adapters over :class:~varagity.stores.conversation_store .ConversationStore — the endpoints are synchronous on purpose (FastAPI runs them in its threadpool), matching the sync psycopg store underneath.

create_conversation(payload, store)

Start a conversation.

Parameters:

Name	Type	Description	Default
payload	ConversationCreateRequest	Optional explicit title (the first chat turn auto-titles otherwise).	required
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
ConversationSummaryOut	The created conversation's summary.

Source code in varagity/api/routes/conversations.py

@router.post("/api/conversations", status_code=201)
def create_conversation(
    payload: ConversationCreateRequest, store: StoreDep
) -> ConversationSummaryOut:
    """Start a conversation.

    Args:
        payload: Optional explicit title (the first chat turn auto-titles
            otherwise).
        store: The per-request conversation store.

    Returns:
        The created conversation's summary.
    """
    return ConversationSummaryOut(**store.create_conversation(payload.title).model_dump())

delete_conversation(conversation_id, store)

Delete a conversation; messages and sources cascade.

Parameters:

Name	Type	Description	Default
conversation_id	str	The conversation to delete.	required
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
Response	An empty 204 response.

Raises:

Type	Description
HTTPException	404 conversation_not_found for an unknown id.

Source code in varagity/api/routes/conversations.py

@router.delete("/api/conversations/{conversation_id}", status_code=204)
def delete_conversation(conversation_id: str, store: StoreDep) -> Response:
    """Delete a conversation; messages and sources cascade.

    Args:
        conversation_id: The conversation to delete.
        store: The per-request conversation store.

    Returns:
        An empty ``204`` response.

    Raises:
        HTTPException: ``404 conversation_not_found`` for an unknown id.
    """
    if store.delete_conversation(conversation_id) == 0:
        raise _not_found(conversation_id)
    return Response(status_code=204)

get_conversation(conversation_id, store)

Fetch a full transcript: messages plus each answer's stored sources.

Parameters:

Name	Type	Description	Default
conversation_id	str	The conversation to fetch.	required
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
ConversationDetailOut	The transcript, oldest message first.

Raises:

Type	Description
HTTPException	404 conversation_not_found for an unknown id.

Source code in varagity/api/routes/conversations.py

@router.get("/api/conversations/{conversation_id}")
def get_conversation(conversation_id: str, store: StoreDep) -> ConversationDetailOut:
    """Fetch a full transcript: messages plus each answer's stored sources.

    Args:
        conversation_id: The conversation to fetch.
        store: The per-request conversation store.

    Returns:
        The transcript, oldest message first.

    Raises:
        HTTPException: ``404 conversation_not_found`` for an unknown id.
    """
    detail = store.get_conversation(conversation_id)
    if detail is None:
        raise _not_found(conversation_id)
    return ConversationDetailOut(**detail.model_dump())

list_conversations(store)

List every conversation, most recently updated first.

Parameters:

Name	Type	Description	Default
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
list[ConversationSummaryOut]	Summaries with message counts.

Source code in varagity/api/routes/conversations.py

@router.get("/api/conversations")
def list_conversations(store: StoreDep) -> list[ConversationSummaryOut]:
    """List every conversation, most recently updated first.

    Args:
        store: The per-request conversation store.

    Returns:
        Summaries with message counts.
    """
    return [
        ConversationSummaryOut(**summary.model_dump()) for summary in store.list_conversations()
    ]

update_conversation(conversation_id, payload, store)

Move a conversation into a sidebar group, or out of one.

Deliberately not a re-order: the move never bumps updated_at, so the recency-sorted list keeps its order.

Parameters:

Name	Type	Description	Default
conversation_id	str	The conversation to move.	required
payload	ConversationUpdateRequest	The target group (group_id: null ungroups).	required
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
Response	An empty 204 response.

Raises:

Type	Description
HTTPException	404 group_not_found for an unknown target group; 404 conversation_not_found for an unknown conversation.

Source code in varagity/api/routes/conversations.py

@router.patch("/api/conversations/{conversation_id}", status_code=204)
def update_conversation(
    conversation_id: str, payload: ConversationUpdateRequest, store: StoreDep
) -> Response:
    """Move a conversation into a sidebar group, or out of one.

    Deliberately not a re-order: the move never bumps ``updated_at``, so
    the recency-sorted list keeps its order.

    Args:
        conversation_id: The conversation to move.
        payload: The target group (``group_id: null`` ungroups).
        store: The per-request conversation store.

    Returns:
        An empty ``204`` response.

    Raises:
        HTTPException: ``404 group_not_found`` for an unknown target group;
            ``404 conversation_not_found`` for an unknown conversation.
    """
    if payload.group_id is not None and not store.group_exists(payload.group_id):
        raise group_not_found(payload.group_id)
    if store.set_conversation_group(conversation_id, payload.group_id) == 0:
        raise _not_found(conversation_id)
    return Response(status_code=204)

varagity.api.routes.groups

Conversation-group CRUD routes.

Sidebar groups are user-created folders over the conversation list. The endpoints mirror :mod:varagity.api.routes.conversations — synchronous thin adapters (FastAPI runs them in its threadpool) over the same per-request :class:~varagity.stores.conversation_store.ConversationStore. There is no rename: like conversations, a group's identity is its id, and the GUI offers create/delete only.

create_group(payload, store)

Create a conversation group.

Parameters:

Name	Type	Description	Default
payload	GroupCreateRequest	The new group's display name.	required
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
GroupOut	The created group.

Source code in varagity/api/routes/groups.py

@router.post("/api/groups", status_code=201)
def create_group(payload: GroupCreateRequest, store: StoreDep) -> GroupOut:
    """Create a conversation group.

    Args:
        payload: The new group's display name.
        store: The per-request conversation store.

    Returns:
        The created group.
    """
    return GroupOut(**store.create_group(payload.name).model_dump())

delete_group(group_id, store)

Delete a group; its conversations survive, ungrouped.

Parameters:

Name	Type	Description	Default
group_id	str	The group to delete.	required
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
Response	An empty 204 response.

Raises:

Type	Description
HTTPException	404 group_not_found for an unknown id.

Source code in varagity/api/routes/groups.py

@router.delete("/api/groups/{group_id}", status_code=204)
def delete_group(group_id: str, store: StoreDep) -> Response:
    """Delete a group; its conversations survive, ungrouped.

    Args:
        group_id: The group to delete.
        store: The per-request conversation store.

    Returns:
        An empty ``204`` response.

    Raises:
        HTTPException: ``404 group_not_found`` for an unknown id.
    """
    if store.delete_group(group_id) == 0:
        raise group_not_found(group_id)
    return Response(status_code=204)

group_not_found(group_id)

Build the structured 404 for an unknown group.

Shared with the conversations router, whose move endpoint validates its target group against the same vocabulary.

Parameters:

Name	Type	Description	Default
group_id	str	The id that failed to resolve.	required

Returns:

Type	Description
HTTPException	The exception carrying the {code, message} detail dict.

Source code in varagity/api/routes/groups.py

def group_not_found(group_id: str) -> HTTPException:
    """Build the structured 404 for an unknown group.

    Shared with the conversations router, whose move endpoint validates its
    target group against the same vocabulary.

    Args:
        group_id: The id that failed to resolve.

    Returns:
        The exception carrying the ``{code, message}`` detail dict.
    """
    return HTTPException(
        status_code=404,
        detail={
            "code": "group_not_found",
            "message": f"No conversation group with id {group_id!r}",
        },
    )

list_groups(store)

List every conversation group, name order.

Parameters:

Name	Type	Description	Default
store	StoreDep	The per-request conversation store.	required

Returns:

Type	Description
list[GroupOut]	All groups, including empty ones.

Source code in varagity/api/routes/groups.py

@router.get("/api/groups")
def list_groups(store: StoreDep) -> list[GroupOut]:
    """List every conversation group, name order.

    Args:
        store: The per-request conversation store.

    Returns:
        All groups, including empty ones.
    """
    return [GroupOut(**group.model_dump()) for group in store.list_groups()]

varagity.api.routes.system

System routes: liveness/dependency health and static capabilities.

GET /api/health reports per-service reachability; GET /api/config exposes the registries and valid ranges the GUI builds its controls from (spec_v2 §4.2) — a newly registered retriever/chunker/parser appears here, and therefore in the UI, with zero API edits.

config()

Expose the registered capabilities and valid ranges.

Mostly static registry contents; the two upload constraints (upload_max_mb, allowed_extensions) are read from the effective settings per request so the dropzone always validates against what the server enforces (ALLOWED_EXTENSIONS is runtime-overridable).

Returns:

Type	Description
ConfigResponse	Registry contents (retrievers, chunkers, OCR engines, model types),
ConfigResponse	the numeric knobs' valid ranges, and the upload constraints.

Source code in varagity/api/routes/system.py

@router.get("/api/config")
def config() -> ConfigResponse:
    """Expose the registered capabilities and valid ranges.

    Mostly static registry contents; the two upload constraints
    (``upload_max_mb``, ``allowed_extensions``) are read from the effective
    settings per request so the dropzone always validates against what the
    server enforces (``ALLOWED_EXTENSIONS`` is runtime-overridable).

    Returns:
        Registry contents (retrievers, chunkers, OCR engines, model types),
        the numeric knobs' valid ranges, and the upload constraints.
    """
    settings = get_settings()
    return ConfigResponse(
        retrievers=sorted(RETRIEVER_REGISTRY),
        chunkers=sorted(CHUNKER_REGISTRY),
        ocr_engines=sorted(OCR_ENGINE_FACTORIES),
        model_types=list(MODEL_TYPES),
        llm_model_types=list(LLM_MODEL_TYPES),
        ranges=_RANGES,
        upload_max_mb=settings.UPLOAD_MAX_MB,
        allowed_extensions=sorted(settings.allowed_extension_set),
        preview_enabled=settings.PREVIEW_ENABLED,
    )

health() async

Report API liveness and per-dependency reachability.

Always 200 while the API process is alive — the compose healthcheck gates on the process, and per-dependency state lives in the body so a flapping backing service doesn't flap the api container.

Returns:

Type	Description
HealthResponse	The aggregate flag and each service's probe outcome.

Source code in varagity/api/routes/system.py

@router.get("/api/health")
async def health() -> HealthResponse:
    """Report API liveness and per-dependency reachability.

    Always ``200`` while the API process is alive — the compose healthcheck
    gates on the process, and per-dependency state lives in the body so a
    flapping backing service doesn't flap the ``api`` container.

    Returns:
        The aggregate flag and each service's probe outcome.
    """
    services = await check_services(ALL_SERVICES)
    return HealthResponse(ok=all(s.ok for s in services.values()), services=services)

varagity.api.routes.metrics

GET /metrics — the Prometheus scrape target (spec_v2 §6).

Serves the process-wide default registry (the spec_v2 §6.2 catalog in :mod:varagity.observability.metrics) in the Prometheus text exposition format. A plain route rather than a mounted make_asgi_app(): mounting 307-redirects the bare /metrics path (Starlette mount semantics), and generate_latest over the default registry is exactly what that ASGI app serves anyway — one process, one registry, single uvicorn worker (plan decision #11, no multiprocess registry).

settings.METRICS_ENABLED gates inclusion of this router at app build time (:func:varagity.api.main.create_app), so disabling metrics turns the endpoint into a structured 404.

metrics()

Expose the metric catalog for a Prometheus scrape.

Returns:

Type	Description
Response	The default registry rendered in the Prometheus text format
Response	(text/plain; version=0.0.4).

Source code in varagity/api/routes/metrics.py

@router.get("/metrics")
def metrics() -> Response:
    """Expose the metric catalog for a Prometheus scrape.

    Returns:
        The default registry rendered in the Prometheus text format
        (``text/plain; version=0.0.4``).
    """
    return Response(content=generate_latest(REGISTRY), media_type=CONTENT_TYPE_LATEST)

varagity.api.routes.settings

GET/PATCH /api/settings — the live settings surface (spec_v2 §4.7).

GET returns the full overridable catalog (effective values, groups, override/reingest flags, registry-derived choices) plus both staleness flags — the chunk corpus' and the graph's; PATCH merges new overrides over the active set, validates the whole through the config validators, persists to app_settings, clears the settings cache (via :mod:varagity.api.runtime_settings), and flags the corpus stale when a reingest-affecting knob actually changed on a non-empty corpus. Query-time knobs take effect on the next question; ingest-time knobs surface the v1 "hashes don't change" footgun as the "Re-ingest to apply" affordance instead of a silent no-op.

patch_settings(payload, store, vector_store, gates)

Apply, validate, and persist runtime setting overrides.

A None value clears that override (reverting to the env value). Validation runs on the merged whole through every config validator, so linked constraints (the fusion weight pair, the rerank bounds) hold exactly as they do for .env — an invalid patch changes nothing.

Parameters:

Name	Type	Description	Default
payload	SettingsPatchRequest	The overrides to set or clear.	required
store	SettingsStoreDep	The per-request app-settings store (persistence + flag).	required
vector_store	VectorStoreDep	The vector store (the corpus-emptiness check).	required
gates	GatesDep	The process's per-corpus write gates — during a backup restore the emptiness check is skipped (see below).	required

Returns:

Type	Description
SettingsResponse	The full post-patch catalog; corpus_stale reports whether a
SettingsResponse	reingest is now needed.

Raises:

Type	Description
HTTPException	422 unknown_setting for a name outside the catalog; 422 invalid_settings when the merged settings fail validation; 503 postgres_unreachable if persistence fails (the overrides are rolled back — the process never diverges from the table).

Source code in varagity/api/routes/settings.py

@router.patch("/api/settings")
def patch_settings(
    payload: SettingsPatchRequest,
    store: SettingsStoreDep,
    vector_store: VectorStoreDep,
    gates: GatesDep,
) -> SettingsResponse:
    """Apply, validate, and persist runtime setting overrides.

    A ``None`` value clears that override (reverting to the env value).
    Validation runs on the *merged whole* through every config validator,
    so linked constraints (the fusion weight pair, the rerank bounds) hold
    exactly as they do for ``.env`` — an invalid patch changes nothing.

    Args:
        payload: The overrides to set or clear.
        store: The per-request app-settings store (persistence + flag).
        vector_store: The vector store (the corpus-emptiness check).
        gates: The process's per-corpus write gates — during a backup
            restore the emptiness check is skipped (see below).

    Returns:
        The full post-patch catalog; ``corpus_stale`` reports whether a
        reingest is now needed.

    Raises:
        HTTPException: ``422 unknown_setting`` for a name outside the
            catalog; ``422 invalid_settings`` when the merged settings fail
            validation; ``503 postgres_unreachable`` if persistence fails
            (the overrides are rolled back — the process never diverges
            from the table).
    """
    unknown = sorted(set(payload.overrides) - set(OVERRIDABLE))
    if unknown:
        raise HTTPException(
            status_code=422,
            detail={
                "code": "unknown_setting",
                "message": f"not overridable: {unknown}; overridable: {sorted(OVERRIDABLE)}",
            },
        )

    before = get_settings()
    before_reingest = {name: getattr(before, name) for name in REINGEST_AFFECTING}

    previous = active_overrides()
    candidate = dict(previous)
    for name, value in payload.overrides.items():
        if value is None:
            candidate.pop(name, None)
        else:
            candidate[name] = to_env_value(value)

    try:
        settings = apply_overrides(candidate)
    except ValidationError as error:
        message = "; ".join(
            f"{'.'.join(str(part) for part in item['loc']) or 'settings'}: {item['msg']}"
            for item in error.errors()
        )
        raise HTTPException(
            status_code=422, detail={"code": "invalid_settings", "message": message}
        ) from error

    try:
        for name, value in payload.overrides.items():
            if value is None:
                store.delete_override(name)
            else:
                store.set_override(name, to_env_value(value))
        changed_reingest = sorted(
            name for name in REINGEST_AFFECTING if getattr(settings, name) != before_reingest[name]
        )
        corpus_stale = store.is_corpus_stale()
        graph_stale = store.is_graph_stale()
        # During a backup restore the tables are held by its TRUNCATE until it
        # commits, so counting them would hang this request for the whole
        # load — and the answer is moot: the restore ends by writing the flag
        # its bundle implies, judged against the settings current *then*
        # (ADR-019), so this change is accounted for either way. Flagging now
        # covers a restore that rolls back and leaves the previous rows.
        restoring = gates["rag"].holder == "restore"
        if (
            changed_reingest
            and not corpus_stale
            and (restoring or vector_store.document_count() > 0)
        ):
            store.set_corpus_stale(True)
            corpus_stale = True
            logger.info(
                "reingest-affecting setting(s) changed (%s) — corpus flagged stale",
                ", ".join(changed_reingest),
            )
    except psycopg.Error as error:
        # The table couldn't record what the process now runs with — roll the
        # process back so a restart can't silently lose the change.
        apply_overrides(previous)
        raise HTTPException(
            status_code=503,
            detail={
                "code": "postgres_unreachable",
                "message": f"could not persist overrides — nothing changed ({error})",
            },
        ) from error

    return _catalog(settings, set(candidate), corpus_stale, graph_stale)

read_settings(store)

Report the effective settings and the corpus-stale flag.

Parameters:

Name	Type	Description	Default
store	SettingsStoreDep	The per-request app-settings store.	required

Returns:

Type	Description
SettingsResponse	The overridable catalog with effective (env + override) values.

Source code in varagity/api/routes/settings.py

@router.get("/api/settings")
def read_settings(store: SettingsStoreDep) -> SettingsResponse:
    """Report the effective settings and the corpus-stale flag.

    Args:
        store: The per-request app-settings store.

    Returns:
        The overridable catalog with effective (env + override) values.
    """
    return _catalog(
        get_settings(),
        set(active_overrides()),
        store.is_corpus_stale(),
        store.is_graph_stale(),
    )

varagity.api.routes.documents

Corpus document routes: upload, list, delete (spec_v2 §4.2, spec_v3 §5).

POST /api/documents writes multipart uploads into DOCS_PATH (validated against ALLOWED_EXTENSIONS + UPLOAD_MAX_MB, no auto-ingest — ingestion is its own explicit action). v3 adds optional per-file relative paths (folder uploads): structure is identity, not decoration — doc_id hashes the path relative to DOCS_PATH, so q3/notes.md and q4/notes.md must land at distinct paths or the second silently replaces the first. The optional per-file modified entries (epoch milliseconds, the browser's File.lastModified) restamp each stored file's mtime, so the file_modified_at chunk provenance reflects the document's clock instead of the upload's. GET lists the documents table with each document's extraction mix and its path relative to DOCS_PATH (the GUI's folder-grouping key). DELETE removes a document's chunks from both stores — the v1 gap ("removing a file does not remove its chunks") turned into GUI-driven GC — Elasticsearch first, pgvector last, mirroring the reingest ordering rationale (the documents row is the idempotency marker, so it must go last); with ?remove_file=true the source file is also unlinked when it lives inside DOCS_PATH, so the next ingest can't resurrect the document. POST /api/documents/delete is the same operation over a set (the corpus table's multi-select): the same ordering applied set-wise, in one round trip per store instead of one per document. Both deletes answer 409 corpus_busy while a backup export or restore holds the corpus (ADR-019) — a restore is replacing the very rows a delete would remove — and every route that reads the tables (the list, the previews) answers 503 corpus_restoring during a restore rather than wait out its TRUNCATE for the length of the load.

The …/preview/locate + …/preview/page/{page} pair (ADR-010) backs the evidence panel's page preview: locate scores the source document's pages for a chunk's text and returns highlight rects; the page route serves the rendered PNG. Both degrade per-document (available:false / a 404 with the reason as its code) rather than 500 — the GUI falls back to the full-text view.

delete_document(doc_id, store, bm25, gates, remove_file=False)

Remove a document's chunks from both stores (GUI-driven GC).

Elasticsearch first, pgvector last: the pgvector documents row is the marker — if the ES delete fails, the document still lists and the delete can be retried; a marker deleted first would strand ES chunks invisibly (the exact v1 gap this route closes).

Parameters:

Name	Type	Description	Default
doc_id	str	The document to remove.	required
store	VectorStoreDep	The per-request vector store.	required
bm25	BM25StoreDep	The per-request BM25 store.	required
gates	GatesDep	The process's per-corpus write gates.	required
remove_file	bool	Also unlink the source file — honored only when it resolves inside DOCS_PATH (otherwise the next ingest would simply re-add the document).	False

Returns:

Type	Description
DocumentDeleteResponse	The deletion counts and whether the file was removed.

Raises:

Type	Description
HTTPException	409 corpus_busy while a backup export or restore holds the corpus; 404 document_not_found for an unknown id; 503 es_unreachable when Elasticsearch is down (nothing deleted — retry when it returns).

Source code in varagity/api/routes/documents.py

@router.delete(
    "/api/documents/{doc_id}",
    responses={404: {"model": ErrorResponse}, 409: {"model": ErrorResponse}},
)
def delete_document(
    doc_id: str,
    store: VectorStoreDep,
    bm25: BM25StoreDep,
    gates: GatesDep,
    remove_file: bool = False,
) -> DocumentDeleteResponse:
    """Remove a document's chunks from both stores (GUI-driven GC).

    Elasticsearch first, pgvector last: the pgvector ``documents`` row is
    the marker — if the ES delete fails, the document still lists and the
    delete can be retried; a marker deleted first would strand ES chunks
    invisibly (the exact v1 gap this route closes).

    Args:
        doc_id: The document to remove.
        store: The per-request vector store.
        bm25: The per-request BM25 store.
        gates: The process's per-corpus write gates.
        remove_file: Also unlink the source file — honored only when it
            resolves inside ``DOCS_PATH`` (otherwise the next ingest would
            simply re-add the document).

    Returns:
        The deletion counts and whether the file was removed.

    Raises:
        HTTPException: ``409 corpus_busy`` while a backup export or restore
            holds the corpus; ``404 document_not_found`` for an unknown id;
            ``503 es_unreachable`` when Elasticsearch is down (nothing
            deleted — retry when it returns).
    """
    # Before any store is touched: a restore's TRUNCATE holds the tables, so
    # even the existence lookup below would wait for it.
    refuse_corpus_busy(gates["rag"], _DELETE_BLOCKING)
    documents = {info.doc_id: info for info in store.list_documents()}
    info = documents.get(doc_id)
    if info is None:
        raise HTTPException(
            status_code=404,
            detail={"code": "document_not_found", "message": f"No document with id {doc_id!r}"},
        )

    try:
        bm25.delete_document(doc_id)
    except TransportError as error:
        raise _es_unreachable(error) from error
    store.delete_document(doc_id)

    file_removed = _remove_source_file(info) if remove_file else False
    logger.info(
        "deleted document %s (%d chunk(s)) from both stores%s",
        doc_id,
        info.n_chunks,
        " + source file" if file_removed else "",
    )
    return DocumentDeleteResponse(
        doc_id=doc_id, chunks_deleted=info.n_chunks, file_removed=file_removed
    )

delete_documents(payload, store, bm25, gates, remove_file=False)

Remove several documents' chunks from both stores (bulk GUI-driven GC).

The corpus table's multi-select delete (spec_v2 §4.2). A POST sub-path rather than a body on DELETE — the set to delete is the request's subject, and bodies on DELETE are the kind of thing intermediaries feel free to drop.

Same store ordering as the single-document delete, applied set-wise: one terms delete_by_query against Elasticsearch first, one DELETE … WHERE doc_id = ANY(…) against pgvector last. So the invariant holds for the batch exactly as it does for one document — if Elasticsearch fails, no marker row is gone, every document still lists, and the whole batch is retryable. Unknown ids are reported in not_found rather than raising: a concurrent delete must not fail the rest of the batch.

Parameters:

Name	Type	Description	Default
payload	DocumentBulkDeleteRequest	The requested doc_ids (duplicates collapse; order is preserved).	required
store	VectorStoreDep	The per-request vector store.	required
bm25	BM25StoreDep	The per-request BM25 store.	required
gates	GatesDep	The process's per-corpus write gates.	required
remove_file	bool	Also unlink each source file — honored per document, only inside DOCS_PATH.	False

Returns:

Type	Description
DocumentBulkDeleteResponse	Per-document outcomes plus the ids that had no documents row.

Raises:

Type	Description
HTTPException	409 corpus_busy while a backup export or restore holds the corpus; 503 es_unreachable when Elasticsearch is down (nothing deleted — retry when it returns).

Source code in varagity/api/routes/documents.py

@router.post("/api/documents/delete", responses={409: {"model": ErrorResponse}})
def delete_documents(
    payload: DocumentBulkDeleteRequest,
    store: VectorStoreDep,
    bm25: BM25StoreDep,
    gates: GatesDep,
    remove_file: bool = False,
) -> DocumentBulkDeleteResponse:
    """Remove several documents' chunks from both stores (bulk GUI-driven GC).

    The corpus table's multi-select delete (spec_v2 §4.2). A ``POST``
    sub-path rather than a body on ``DELETE`` — the set to delete is the
    request's subject, and bodies on ``DELETE`` are the kind of thing
    intermediaries feel free to drop.

    Same store ordering as the single-document delete, applied set-wise:
    one ``terms`` ``delete_by_query`` against Elasticsearch first, one
    ``DELETE … WHERE doc_id = ANY(…)`` against pgvector last. So the
    invariant holds for the batch exactly as it does for one document — if
    Elasticsearch fails, no marker row is gone, every document still lists,
    and the whole batch is retryable. Unknown ids are reported in
    ``not_found`` rather than raising: a concurrent delete must not fail
    the rest of the batch.

    Args:
        payload: The requested ``doc_ids`` (duplicates collapse; order is
            preserved).
        store: The per-request vector store.
        bm25: The per-request BM25 store.
        gates: The process's per-corpus write gates.
        remove_file: Also unlink each source file — honored per document,
            only inside ``DOCS_PATH``.

    Returns:
        Per-document outcomes plus the ids that had no ``documents`` row.

    Raises:
        HTTPException: ``409 corpus_busy`` while a backup export or restore
            holds the corpus; ``503 es_unreachable`` when Elasticsearch is
            down (nothing deleted — retry when it returns).
    """
    # Before any store is touched: a restore's TRUNCATE holds the tables, so
    # even the existence lookup below would wait for it.
    refuse_corpus_busy(gates["rag"], _DELETE_BLOCKING)
    documents = {info.doc_id: info for info in store.list_documents()}
    requested = list(dict.fromkeys(payload.doc_ids))  # collapse dupes, keep order
    targets = [documents[doc_id] for doc_id in requested if doc_id in documents]
    not_found = [doc_id for doc_id in requested if doc_id not in documents]

    doc_ids = [info.doc_id for info in targets]
    try:
        bm25.delete_documents(doc_ids)
    except TransportError as error:
        raise _es_unreachable(error) from error
    store.delete_documents(doc_ids)

    deleted = [
        DocumentDeleteResponse(
            doc_id=info.doc_id,
            chunks_deleted=info.n_chunks,
            file_removed=_remove_source_file(info) if remove_file else False,
        )
        for info in targets
    ]
    logger.info(
        "deleted %d document(s) (%d chunk(s)) from both stores%s%s",
        len(deleted),
        sum(entry.chunks_deleted for entry in deleted),
        f" + {sum(entry.file_removed for entry in deleted)} source file(s)" if remove_file else "",
        f"; {len(not_found)} unknown id(s)" if not_found else "",
    )
    return DocumentBulkDeleteResponse(deleted=deleted, not_found=not_found)

list_documents(store)

List every ingested document (the corpus-management table).

Files uploaded but not yet ingested don't appear here — the documents table records ingests; the GUI pairs this list with the upload outcomes it already holds. Each entry carries its relative_path under DOCS_PATH (None for sources living elsewhere) so the GUI can group the table by the folders a folder upload created.

Parameters:

Name	Type	Description	Default
store	VectorStoreDep	The per-request vector store.	required

Returns:

Type	Description
list[DocumentOut]	One entry per document, newest ingest first.

Raises:

Type	Description
HTTPException	503 corpus_restoring while a backup restore replaces the corpus (its TRUNCATE holds the table until it commits).

Source code in varagity/api/routes/documents.py

@router.get(
    "/api/documents",
    dependencies=[Depends(require_rag_readable)],
    responses={503: {"model": ErrorResponse}},
)
def list_documents(store: VectorStoreDep) -> list[DocumentOut]:
    """List every ingested document (the corpus-management table).

    Files uploaded but not yet ingested don't appear here — the ``documents``
    table records ingests; the GUI pairs this list with the upload outcomes
    it already holds. Each entry carries its ``relative_path`` under
    ``DOCS_PATH`` (``None`` for sources living elsewhere) so the GUI can
    group the table by the folders a folder upload created.

    Args:
        store: The per-request vector store.

    Returns:
        One entry per document, newest ingest first.

    Raises:
        HTTPException: ``503 corpus_restoring`` while a backup restore
            replaces the corpus (its ``TRUNCATE`` holds the table until it
            commits).
    """
    docs_root = Path(get_settings().DOCS_PATH).resolve()
    return [
        DocumentOut(
            file_name=Path(info.source).name,
            relative_path=relative_to_root(info.source, docs_root),
            **info.model_dump(),
        )
        for info in store.list_documents()
    ]

locate_preview(doc_id, payload, store)

Find the source page containing a chunk's text, with highlight rects.

The evidence panel's locate step (ADR-010): resolve the document to an openable PDF (containment + content-hash verified; PPTX via the cached conversion), score its pages by word coverage, and return the best page plus normalized highlight rectangles. Every degradable condition — kill switch, unsupported format, missing/edited file, unavailable converter, no textual match — answers 200 available:false with its reason; the GUI falls back to the full-text view.

Parameters:

Name	Type	Description	Default
doc_id	str	The chunk's parent document.	required
payload	PreviewLocateRequest	The chunk text to locate.	required
store	VectorStoreDep	The per-request vector store.	required

Returns:

Type	Description
PreviewLocateResponse	The located page, rects, and coverage — or the degrade reason.

Raises:

Type	Description
HTTPException	404 document_not_found for an unknown id (the one non-degradable condition: there is nothing to fall back to — the GUI shows full text without asking again); 503 corpus_restoring while a backup restore replaces the corpus (the GUI falls back to full text, as for any failed locate).

Source code in varagity/api/routes/documents.py

@router.post(
    "/api/documents/{doc_id}/preview/locate",
    dependencies=[Depends(require_rag_readable)],
    responses={503: {"model": ErrorResponse}},
)
def locate_preview(
    doc_id: str, payload: PreviewLocateRequest, store: VectorStoreDep
) -> PreviewLocateResponse:
    """Find the source page containing a chunk's text, with highlight rects.

    The evidence panel's locate step (ADR-010): resolve the document to an
    openable PDF (containment + content-hash verified; PPTX via the cached
    conversion), score its pages by word coverage, and return the best page
    plus normalized highlight rectangles. Every degradable condition — kill
    switch, unsupported format, missing/edited file, unavailable converter,
    no textual match — answers ``200 available:false`` with its reason; the
    GUI falls back to the full-text view.

    Args:
        doc_id: The chunk's parent document.
        payload: The chunk text to locate.
        store: The per-request vector store.

    Returns:
        The located page, rects, and coverage — or the degrade reason.

    Raises:
        HTTPException: ``404 document_not_found`` for an unknown id (the
            one non-degradable condition: there is nothing to fall back
            *to* — the GUI shows full text without asking again); ``503
            corpus_restoring`` while a backup restore replaces the corpus
            (the GUI falls back to full text, as for any failed locate).
    """
    info = _get_document_or_404(store, doc_id)
    settings = get_settings()
    resolved = resolve_preview_source(info, settings)
    if resolved.pdf_path is None:
        return PreviewLocateResponse(available=False, reason=resolved.reason)
    try:
        result = locate(resolved.pdf_path, payload.text, min_coverage=settings.PREVIEW_MIN_COVERAGE)
    except pdfium.PdfiumError as error:
        # A corrupt/unopenable file must degrade, not 500 (the converted
        # artifact is covered too — hence the conversion-flavored reason).
        logger.warning("preview locate could not open %s (%s): %s", doc_id, info.source, error)
        return PreviewLocateResponse(available=False, reason="conversion_failed")
    if result.page is None:
        return PreviewLocateResponse(
            available=False,
            reason="no_match",
            page_count=result.page_count,
            coverage=result.coverage,
        )
    return PreviewLocateResponse(
        available=True,
        page=result.page,
        page_count=result.page_count,
        rects=[PreviewRect(x0=x0, y0=y0, x1=x1, y1=y1) for x0, y0, x1, y1 in result.rects],
        coverage=result.coverage,
    )

preview_page(doc_id, page, store)

Serve one rendered page of a source document as a PNG.

The image half of the preview pair (ADR-010), immutable-cacheable: doc_id is content-hashed and the resolution step re-verifies the on-disk content_hash before rendering, so a drifted file 404s (file_changed) rather than serving a lying image. Degradable reasons are 404 codes here — an <img> can't read a JSON envelope, and the client only requests pages a successful locate named.

Parameters:

Name	Type	Description	Default
doc_id	str	The document to render.	required
page	int	1-based page number (a locate response's page).	required
store	VectorStoreDep	The per-request vector store.	required

Returns:

Type	Description
Response	The PNG at PREVIEW_RENDER_WIDTH pixels wide, marked
Response	Cache-Control: public, max-age=31536000, immutable.

Raises:

Type	Description
HTTPException	404 document_not_found for an unknown id; 404 with the degrade reason as its code (preview_disabled | unsupported_type | file_missing | file_changed | conversion_unavailable | conversion_failed) when the document can't resolve; 404 page_out_of_range beyond the last page; 503 corpus_restoring while a backup restore replaces the corpus.

Source code in varagity/api/routes/documents.py

@router.get(
    "/api/documents/{doc_id}/preview/page/{page}",
    dependencies=[Depends(require_rag_readable)],
    responses={503: {"model": ErrorResponse}},
)
def preview_page(doc_id: str, page: int, store: VectorStoreDep) -> Response:
    """Serve one rendered page of a source document as a PNG.

    The image half of the preview pair (ADR-010), immutable-cacheable:
    ``doc_id`` is content-hashed and the resolution step re-verifies the
    on-disk ``content_hash`` before rendering, so a drifted file 404s
    (``file_changed``) rather than serving a lying image. Degradable
    reasons are 404 codes here — an ``<img>`` can't read a JSON envelope,
    and the client only requests pages a successful locate named.

    Args:
        doc_id: The document to render.
        page: 1-based page number (a locate response's ``page``).
        store: The per-request vector store.

    Returns:
        The PNG at ``PREVIEW_RENDER_WIDTH`` pixels wide, marked
        ``Cache-Control: public, max-age=31536000, immutable``.

    Raises:
        HTTPException: ``404 document_not_found`` for an unknown id; ``404``
            with the degrade reason as its code (``preview_disabled`` |
            ``unsupported_type`` | ``file_missing`` | ``file_changed`` |
            ``conversion_unavailable`` | ``conversion_failed``) when the
            document can't resolve; ``404 page_out_of_range`` beyond the
            last page; ``503 corpus_restoring`` while a backup restore
            replaces the corpus.
    """
    info = _get_document_or_404(store, doc_id)
    settings = get_settings()
    resolved = resolve_preview_source(info, settings)
    if resolved.pdf_path is None:
        raise HTTPException(
            status_code=404,
            detail={
                "code": str(resolved.reason),
                "message": f"no previewable rendition of {doc_id!r} ({resolved.reason})",
            },
        )
    try:
        png = render_page_png(resolved.pdf_path, page, width=settings.PREVIEW_RENDER_WIDTH)
    except IndexError as error:
        raise HTTPException(
            status_code=404,
            detail={"code": "page_out_of_range", "message": str(error)},
        ) from error
    except pdfium.PdfiumError as error:
        logger.warning("preview render could not open %s (%s): %s", doc_id, info.source, error)
        raise HTTPException(
            status_code=404,
            detail={"code": "conversion_failed", "message": f"could not render {doc_id!r}"},
        ) from error
    return Response(
        content=png,
        media_type="image/png",
        headers={"Cache-Control": "public, max-age=31536000, immutable"},
    )

upload_documents(files, paths=None, modified=None)

Upload file(s) into DOCS_PATH (no auto-ingest).

Each file is validated and stored independently; rejected files are reported per-file so a mixed batch partially succeeds. A same-named existing file is replaced (the re-upload flow) — its changed content hash makes the next ingest re-process it.

With paths (folder uploads, spec_v3 §5.2), each file lands at its sanitized relative path under DOCS_PATH, preserving nested structure — structure is identity: doc_id hashes the relative path, so q3/notes.md and q4/notes.md must not collapse onto one name. Entries pair with files positionally; an empty-string entry keeps that file on the flat path. Without paths the flat contract is untouched.

With modified (same positional contract), each stored file's mtime is restamped to the client's declared last-modified time (epoch milliseconds — the browser's File.lastModified), so the file_modified_at provenance on the next ingest's chunks carries the document's clock rather than the upload instant. An empty-string entry keeps that file's write time; a malformed entry is ignored per file.

Parameters:

Name	Type	Description	Default
files	list[UploadFile]	The multipart file parts.	required
paths	Annotated[list[str] | None, Form()]	Optional relative path per file, positionally aligned with files.	None
modified	Annotated[list[str] | None, Form()]	Optional last-modified epoch-milliseconds per file, positionally aligned with files.	None

Returns:

Type	Description
UploadResponse	Per-file outcomes, in upload order.

Raises:

Type

Description

HTTPException

422 paths_mismatch / 422 modified_mismatch when paths / modified is present with a different length than files (a positional contract must be checked, not trusted); 422 too_many_files / 422 batch_too_large when the batch busts UPLOAD_MAX_FILES / UPLOAD_MAX_TOTAL_MB (checked before anything is written); 422 no_file_stored when every file was rejected on its own merits (extension/size/name/path); 500 docs_path_not_writable when nothing landed because the server couldn't write DOCS_PATH (e.g. the ./docs bind mount is not writable by the api container's user).

Source code in varagity/api/routes/documents.py

@router.post("/api/documents", status_code=201)
def upload_documents(
    files: list[UploadFile],
    paths: Annotated[list[str] | None, Form()] = None,
    modified: Annotated[list[str] | None, Form()] = None,
) -> UploadResponse:
    """Upload file(s) into ``DOCS_PATH`` (no auto-ingest).

    Each file is validated and stored independently; rejected files are
    reported per-file so a mixed batch partially succeeds. A same-named
    existing file is replaced (the re-upload flow) — its changed content
    hash makes the next ingest re-process it.

    With ``paths`` (folder uploads, spec_v3 §5.2), each file lands at its
    sanitized relative path under ``DOCS_PATH``, preserving nested
    structure — structure is *identity*: ``doc_id`` hashes the relative
    path, so ``q3/notes.md`` and ``q4/notes.md`` must not collapse onto one
    name. Entries pair with ``files`` positionally; an empty-string entry
    keeps that file on the flat path. Without ``paths`` the flat contract
    is untouched.

    With ``modified`` (same positional contract), each stored file's mtime
    is restamped to the client's declared last-modified time (epoch
    milliseconds — the browser's ``File.lastModified``), so the
    ``file_modified_at`` provenance on the next ingest's chunks carries the
    document's clock rather than the upload instant. An empty-string entry
    keeps that file's write time; a malformed entry is ignored per file.

    Args:
        files: The multipart file parts.
        paths: Optional relative path per file, positionally aligned with
            ``files``.
        modified: Optional last-modified epoch-milliseconds per file,
            positionally aligned with ``files``.

    Returns:
        Per-file outcomes, in upload order.

    Raises:
        HTTPException: ``422 paths_mismatch`` / ``422 modified_mismatch``
            when ``paths`` / ``modified`` is present with a different
            length than ``files`` (a positional contract must be checked,
            not trusted); ``422 too_many_files`` / ``422
            batch_too_large`` when the batch busts ``UPLOAD_MAX_FILES`` /
            ``UPLOAD_MAX_TOTAL_MB`` (checked before anything is written);
            ``422 no_file_stored`` when every file was rejected on its own
            merits (extension/size/name/path); ``500
            docs_path_not_writable`` when nothing landed because the server
            couldn't write ``DOCS_PATH`` (e.g. the ``./docs`` bind mount is
            not writable by the api container's user).
    """
    settings = get_settings()
    if paths is not None and len(paths) != len(files):
        raise HTTPException(
            status_code=422,
            detail={
                "code": "paths_mismatch",
                "message": f"paths carries {len(paths)} entries for {len(files)} files — "
                "the positional pairing must match exactly",
            },
        )
    if modified is not None and len(modified) != len(files):
        raise HTTPException(
            status_code=422,
            detail={
                "code": "modified_mismatch",
                "message": f"modified carries {len(modified)} entries for {len(files)} files — "
                "the positional pairing must match exactly",
            },
        )
    if len(files) > settings.UPLOAD_MAX_FILES:
        raise HTTPException(
            status_code=422,
            detail={
                "code": "too_many_files",
                "message": f"{len(files)} files exceed UPLOAD_MAX_FILES "
                f"({settings.UPLOAD_MAX_FILES}) — split the upload",
            },
        )
    total_bytes = sum(_upload_size(upload) for upload in files)
    if total_bytes > settings.UPLOAD_MAX_TOTAL_MB * 1024 * 1024:
        raise HTTPException(
            status_code=422,
            detail={
                "code": "batch_too_large",
                "message": f"{total_bytes} bytes exceed UPLOAD_MAX_TOTAL_MB "
                f"({settings.UPLOAD_MAX_TOTAL_MB} MB) — split the upload",
            },
        )
    docs_root = Path(settings.DOCS_PATH)
    not_writable = HTTPException(
        status_code=500,
        detail={
            "code": "docs_path_not_writable",
            "message": (
                f"the API cannot write DOCS_PATH ({docs_root}) — in compose, the "
                "./docs bind mount must be writable by the api container's user "
                "(rebuild the images if you changed uids; see the runbook)"
            ),
        },
    )
    try:
        docs_root.mkdir(parents=True, exist_ok=True)
    except OSError as error:
        logger.error("cannot create DOCS_PATH %s: %s", docs_root, error)
        raise not_writable from error
    max_bytes = settings.UPLOAD_MAX_MB * 1024 * 1024

    matched_paths: list[str | None] = list(paths) if paths is not None else [None] * len(files)
    matched_modified: list[str | None] = (
        list(modified) if modified is not None else [None] * len(files)
    )
    results = [
        _store_upload(
            upload,
            docs_root,
            max_bytes,
            raw_path,
            raw_modified,
            allowed_extensions=settings.allowed_extension_set,
        )
        for upload, raw_path, raw_modified in zip(
            files, matched_paths, matched_modified, strict=True
        )
    ]
    if not any(result.stored for result in results):
        if any(result.reason == "write_failed" for result in results):
            raise not_writable
        raise HTTPException(
            status_code=422,
            detail={
                "code": "no_file_stored",
                "message": "; ".join(f"{r.file_name}: {r.reason}" for r in results)
                or "empty upload",
            },
        )
    return UploadResponse(files=results)

varagity.api.routes.ingest

POST /api/ingest + GET /api/ingest/status (spec_v2 §4.2).

Triggering returns a run handle immediately (the flow runs on a background thread — see :mod:varagity.api.ingest_runner); the status endpoint is an SSE stream of the run's events — a status snapshot, per-stage/per-file progress frames (with per-chunk contextualize ticks), relayed log lines, and the terminal status with the summary counters. The stream replays from the start, so connecting mid-run or after completion renders the same picture the CLI's rich display showed live.

ingest_status(runner) async

Stream the current (or last) ingest run's progress as SSE.

Frames: status (snapshot; also terminal, with the summary), progress (per stage/file/chunk), log (relayed pipeline lines). With no run in this API process the stream is a single idle status frame; after a terminal run it replays that run and closes.

Parameters:

Name	Type	Description	Default
runner	RunnerDep	The process-wide ingest runner.	required

Yields:

Type	Description
AsyncIterator[ServerSentEvent]	The framed events, oldest first, then live until terminal.

Source code in varagity/api/routes/ingest.py

@router.get("/api/ingest/status", response_class=EventSourceResponse)
async def ingest_status(runner: RunnerDep) -> AsyncIterator[ServerSentEvent]:
    """Stream the current (or last) ingest run's progress as SSE.

    Frames: ``status`` (snapshot; also terminal, with the summary),
    ``progress`` (per stage/file/chunk), ``log`` (relayed pipeline lines).
    With no run in this API process the stream is a single idle ``status``
    frame; after a terminal run it replays that run and closes.

    Args:
        runner: The process-wide ingest runner.

    Yields:
        The framed events, oldest first, then live until terminal.
    """
    async for event, payload in runner.subscribe():
        yield ServerSentEvent(event=event, data=payload)

start_ingest(payload, runner, preflight) async

Trigger a background ingest run and return its handle.

Parameters:

Name	Type	Description	Default
payload	IngestStartRequest	reingest=true re-processes unchanged files (the stale-corpus action; setting changes don't change content hashes).	required
runner	RunnerDep	The process-wide ingest runner.	required
preflight	Annotated[Callable[[], Awaitable[None]], Depends(get_ingest_preflight)]	The awaitable reachability check (structured 503).	required

Returns:

Type	Description
IngestRunOut	The new run's snapshot (state running).

Raises:

Type	Description
HTTPException	409 ingest_already_running while a run is in flight; 409 corpus_busy while a backup export or restore holds the corpus (ADR-019 — the message names it); 503 <service>_unreachable when a required backing service is down.

Source code in varagity/api/routes/ingest.py

@router.post(
    "/api/ingest",
    status_code=202,
    responses={409: {"model": ErrorResponse}, 503: {"model": ErrorResponse}},
)
async def start_ingest(
    payload: IngestStartRequest,
    runner: RunnerDep,
    preflight: Annotated[Callable[[], Awaitable[None]], Depends(get_ingest_preflight)],
) -> IngestRunOut:
    """Trigger a background ingest run and return its handle.

    Args:
        payload: ``reingest=true`` re-processes unchanged files (the
            stale-corpus action; setting changes don't change content
            hashes).
        runner: The process-wide ingest runner.
        preflight: The awaitable reachability check (structured 503).

    Returns:
        The new run's snapshot (state ``running``).

    Raises:
        HTTPException: ``409 ingest_already_running`` while a run is in
            flight; ``409 corpus_busy`` while a backup export or restore
            holds the corpus (ADR-019 — the message names it);
            ``503 <service>_unreachable`` when a required backing service is
            down.
    """
    await preflight()
    try:
        return runner.start(reingest=payload.reingest)
    except IngestAlreadyRunning as error:
        raise HTTPException(
            status_code=409,
            detail={"code": "ingest_already_running", "message": str(error)},
        ) from error
    except CorpusBusy as error:
        raise corpus_busy_error(error) from error

varagity.api.routes.graph

Graph routes: corpus upload/list/delete, build, status, export (§5.2, §4.4).

The Graph RAG tab's whole server surface, and the deliberate mirror of the chunk-RAG corpus routes (:mod:varagity.api.routes.documents + :mod:varagity.api.routes.ingest) with the differences the subject matter forces:

• Uploads are sniffed, not allow-listed. ALLOWED_EXTENSIONS governs the chunk-RAG corpus and says nothing here (.db isn't even in it); what decides is whether a registered :class:~varagity.graph.sources.base.MessageSource claims the stored bytes — the imessage probe checks the SQLite magic header and the presence of a message table (behind a cheap .db suffix gate of its own). A file that lands and then fails the sniff is removed again and reported as unsupported_graph_source. The cap is its own setting (GRAPH_UPLOAD_MAX_MB, default 4 GB): a decade of iMessage history is one enormous file, where the corpus cap is document-sized. WAL sidecars are first-class: a live chat.db keeps recent messages in chat.db-wal, so copying all three files (.db, -wal, -shm) is the documented rule — the sidecars are stored beside their database and never sniffed on their own.
• The listing is a directory scan, not a table read. The graph's own record of what it indexed lives in the engine's workdir manifest; what this tab has to show is what is on disk, including files the user dropped in that no source claims. Parse summaries come from the last build's scan (parsing a multi-gigabyte database is a build's job, not a GET's).
• Deleting a source file flags the graph stale. The graph still holds that file's messages — only a reingest rebuild retracts them — so the flag is set and the tab says so (stage-2 decision #16). A delete is refused (409 corpus_busy) while a backup restore holds the graph: the restore ends by writing the flag its bundle recorded, which would silently undo the delete's.
• The reserved varagity-exports/ folder is not corpus (ADR-019): the graph's backup bundles live there, so the listing never shows them, the delete never removes one (it would flag the graph stale for a file that was never a source), and the build's scan never counts them.
• Builds are API-only (stage-2 decision #4): the engine's storages are single-writer per working directory, and this process owns them. There is deliberately no CLI graph build.
• Reading the graph never builds one. The export and entity-detail routes — the graph view's data source (spec_graphrag §4.4) — skip the engine entirely when the working directory does not exist, because opening a session creates the storages and pays the tokenizer's startup. A page load must not turn "nothing indexed yet" into an initialized workdir.

GRAPH_ENABLED=false turns every entry point that acts on the graph — upload, build, and the two read routes the view draws from — into a structured 403 graph_disabled (ADR-017's degrade semantics: never a silent no-op). Listing, deleting, and the status poll keep working, because a disabled graph still has a corpus directory to manage and a tab that must render honestly.

delete_graph_document(name, store, gates)

Remove one file from the graph corpus directory and flag the graph stale.

Deleting the source does not retract what the graph already extracted from it — that costs a rebuild — so this sets the graph-stale flag and the tab surfaces it, rather than letting the graph and its corpus diverge silently (stage-2 decision #16).

Parameters:

Name	Type	Description	Default
name	str	The file's path relative to GRAPH_DOCS_PATH.	required
store	SettingsStoreDep	The per-request app-settings store (the stale flag).	required
gates	GatesDep	The process's per-corpus write gates.	required

Returns:

Type	Description
GraphDocumentDeleteResponse	What was removed and whether the graph is now flagged stale.

Raises:

Type

Description

HTTPException

409 corpus_busy while a backup restore holds the graph (it ends by writing the bundle's own stale flag, which would undo this one — checked before anything is touched); 404 graph_document_not_found when no such file lives inside GRAPH_DOCS_PATH — which covers a path trying to escape it, since an escaping path is not a file this route has, and anything in the reserved varagity-exports/ folder (a backup bundle is not a graph source, whatever path reaches it).

Source code in varagity/api/routes/graph.py

@router.delete(
    "/api/graph/documents/{name:path}",
    responses={404: {"model": ErrorResponse}, 409: {"model": ErrorResponse}},
)
def delete_graph_document(
    name: str, store: SettingsStoreDep, gates: GatesDep
) -> GraphDocumentDeleteResponse:
    """Remove one file from the graph corpus directory and flag the graph stale.

    Deleting the *source* does not retract what the graph already extracted
    from it — that costs a rebuild — so this sets the graph-stale flag and
    the tab surfaces it, rather than letting the graph and its corpus
    diverge silently (stage-2 decision #16).

    Args:
        name: The file's path relative to ``GRAPH_DOCS_PATH``.
        store: The per-request app-settings store (the stale flag).
        gates: The process's per-corpus write gates.

    Returns:
        What was removed and whether the graph is now flagged stale.

    Raises:
        HTTPException: ``409 corpus_busy`` while a backup restore holds the
            graph (it ends by writing the bundle's own stale flag, which
            would undo this one — checked before anything is touched);
            ``404 graph_document_not_found`` when no such file lives inside
            ``GRAPH_DOCS_PATH`` — which covers a path trying to escape it,
            since an escaping path is not a file this route has, and
            anything in the reserved ``varagity-exports/`` folder (a backup
            bundle is not a graph source, whatever path reaches it).
    """
    refuse_corpus_busy(gates["graph"], _DELETE_BLOCKING)
    graph_root = Path(get_settings().GRAPH_DOCS_PATH).resolve()
    target = resolve_contained(graph_root / name, graph_root)
    if target is None or not target.is_file() or is_exports_path(target.relative_to(graph_root)):
        raise HTTPException(
            status_code=404,
            detail={
                "code": "graph_document_not_found",
                "message": f"No graph corpus file at {name!r}",
            },
        )
    relative = target.relative_to(graph_root).as_posix()
    target.unlink()
    _prune_empty_parents(target.parent, graph_root)
    store.set_graph_stale(True)
    logger.info("deleted graph source %s — graph flagged stale until the next rebuild", relative)
    return GraphDocumentDeleteResponse(relative_path=relative, graph_stale=True)

export_graph(service, label=_WHOLE_GRAPH, max_depth=3, max_nodes=MAX_EXPORT_NODES)

Read a drawable slice of the graph (spec_graphrag §4.4).

The graph view's whole data source. The default is the whole graph, degree-ordered so the cap keeps the most connected entities, and truncated says when the cap bit — the view surfaces that rather than implying it drew everything.

Parameters:

Name	Type	Description	Default
service	ServiceDep	The process-wide graph service.	required
label	str	Entity name to centre the slice on; "*" (the default) takes the whole graph.	_WHOLE_GRAPH
max_depth	Annotated[int, Query(ge=1, le=_MAX_EXPORT_DEPTH)]	Hops to walk out from label (ignored for "*").	3
max_nodes	Annotated[int, Query(ge=1, le=MAX_EXPORT_NODES)]	Node cap, defaulting to its own ceiling :data:MAX_EXPORT_NODES (the view draws the fullest slice the contract allows). Above it the request is a 422 rather than a silent clamp: a caller asking for more than the view can render should hear so.	MAX_EXPORT_NODES

Returns:

Type	Description
GraphExportOut	The slice; empty (not an error) when nothing has been indexed yet.

Raises:

Type	Description
HTTPException	403 graph_disabled when the kill switch is off; 503 graph_unavailable when the engine will not open.

Source code in varagity/api/routes/graph.py

@router.get("/api/graph/export", responses={403: {"model": ErrorResponse}})
def export_graph(
    service: ServiceDep,
    label: str = _WHOLE_GRAPH,
    max_depth: Annotated[int, Query(ge=1, le=_MAX_EXPORT_DEPTH)] = 3,
    max_nodes: Annotated[int, Query(ge=1, le=MAX_EXPORT_NODES)] = MAX_EXPORT_NODES,
) -> GraphExportOut:
    """Read a drawable slice of the graph (spec_graphrag §4.4).

    The graph view's whole data source. The default is the *whole* graph,
    degree-ordered so the cap keeps the most connected entities, and
    ``truncated`` says when the cap bit — the view surfaces that rather than
    implying it drew everything.

    Args:
        service: The process-wide graph service.
        label: Entity name to centre the slice on; ``"*"`` (the default)
            takes the whole graph.
        max_depth: Hops to walk out from ``label`` (ignored for ``"*"``).
        max_nodes: Node cap, defaulting to its own ceiling
            :data:`MAX_EXPORT_NODES` (the view draws the fullest slice the
            contract allows). Above it the request is a ``422`` rather than
            a silent clamp: a caller asking for more than the view can
            render should hear so.

    Returns:
        The slice; empty (not an error) when nothing has been indexed yet.

    Raises:
        HTTPException: ``403 graph_disabled`` when the kill switch is off;
            ``503 graph_unavailable`` when the engine will not open.
    """
    _require_enabled()
    export = _read_graph(service, label, max_depth=max_depth, max_nodes=max_nodes)
    return GraphExportOut(
        nodes=[_node_out(node) for node in export.nodes],
        edges=[_edge_out(edge) for edge in export.edges],
        truncated=export.truncated,
    )

graph_build_status(runner) async

Stream the current (or last) graph build's progress as SSE.

Frames: status (snapshot; also terminal, with the summary), progress (scan → parse per file → bound/reset → index → sampled process ticks), log (relayed varagity.graph lines). The stream replays from the start, so a browser opened six hours into a backfill renders the same picture one opened at the start did.

Parameters:

Name	Type	Description	Default
runner	RunnerDep	The process-wide graph build runner.	required

Yields:

Type	Description
AsyncIterator[ServerSentEvent]	The framed events, oldest first, then live until terminal.

Source code in varagity/api/routes/graph.py

@router.get("/api/graph/build/status", response_class=EventSourceResponse)
async def graph_build_status(runner: RunnerDep) -> AsyncIterator[ServerSentEvent]:
    """Stream the current (or last) graph build's progress as SSE.

    Frames: ``status`` (snapshot; also terminal, with the summary),
    ``progress`` (``scan`` → ``parse`` per file → ``bound``/``reset`` →
    ``index`` → sampled ``process`` ticks), ``log`` (relayed
    ``varagity.graph`` lines). The stream replays from the start, so a
    browser opened six hours into a backfill renders the same picture one
    opened at the start did.

    Args:
        runner: The process-wide graph build runner.

    Yields:
        The framed events, oldest first, then live until terminal.
    """
    async for event, payload in runner.subscribe():
        yield ServerSentEvent(event=event, data=payload)

graph_entity(name, service)

Inspect one entity: its summary, its neighbourhood, its source days.

The graph view's click-through (spec_graphrag §4.4): a name in the picture resolves to what the engine merged about it, the relations around it, and — the drill-down that makes the graph answerable — the transcript days it was extracted from.

Parameters:

Name	Type	Description	Default
name	str	The entity's name (path-encoded; extracted names may contain slashes). Canonical spelling resolves directly; any other casing resolves through a whole-graph lookup, because the engine's own label match is exact.	required
service	ServiceDep	The process-wide graph service.	required

Returns:

Type	Description
GraphEntityDetailOut	The entity with its depth-1 relations and its source days.

Raises:

Type	Description
HTTPException	403 graph_disabled when the kill switch is off; 404 entity_not_found when the graph holds no such entity (which includes a graph that has not been built yet); 503 graph_unavailable when the engine will not open.

Source code in varagity/api/routes/graph.py

@router.get(
    "/api/graph/entities/{name:path}",
    responses={403: {"model": ErrorResponse}, 404: {"model": ErrorResponse}},
)
def graph_entity(name: str, service: ServiceDep) -> GraphEntityDetailOut:
    """Inspect one entity: its summary, its neighbourhood, its source days.

    The graph view's click-through (spec_graphrag §4.4): a name in the
    picture resolves to what the engine merged about it, the relations
    around it, and — the drill-down that makes the graph answerable — the
    transcript days it was extracted from.

    Args:
        name: The entity's name (path-encoded; extracted names may contain
            slashes). Canonical spelling resolves directly; any other casing
            resolves through a whole-graph lookup, because the engine's own
            label match is exact.
        service: The process-wide graph service.

    Returns:
        The entity with its depth-1 relations and its source days.

    Raises:
        HTTPException: ``403 graph_disabled`` when the kill switch is off;
            ``404 entity_not_found`` when the graph holds no such entity
            (which includes a graph that has not been built yet);
            ``503 graph_unavailable`` when the engine will not open.
    """
    _require_enabled()
    export = _read_graph(service, name, max_depth=1, max_nodes=MAX_EXPORT_NODES)
    node = _find_node(export, name) if name else None
    if node is None and name:
        # The engine's label lookup is exact, so a differently-cased name
        # comes back as an *empty* slice — the in-slice case fallback never
        # gets a candidate. Find the canonical spelling in the whole graph,
        # then re-slice on it. Honestly bounded: an entity outside the top
        # MAX_EXPORT_NODES by degree is reachable only by exact spelling.
        whole = _read_graph(service, _WHOLE_GRAPH, max_depth=1, max_nodes=MAX_EXPORT_NODES)
        canonical = _find_node(whole, name)
        if canonical is not None:
            export = _read_graph(service, canonical.id, max_depth=1, max_nodes=MAX_EXPORT_NODES)
            node = _find_node(export, canonical.id)
    if node is None:
        raise HTTPException(
            status_code=404,
            detail={"code": "entity_not_found", "message": f"No graph entity named {name!r}"},
        )
    return GraphEntityDetailOut(
        entity=_node_out(node),
        relations=[
            _edge_out(edge) for edge in export.edges if node.id in (edge.source, edge.target)
        ],
        sources=_transcript_refs(node.doc_keys, service.workdir),
    )

graph_status(runner, service, store)

Report the graph's size, staleness, and build state (the tab's poll).

Reads the workdir's summary sidecar rather than the graph itself, so a poll never walks a multi-megabyte graphml (stage-2 decision #18), and skips the engine entirely when nothing has been built here — an unbuilt graph must not make a status poll open a session and pay an engine's startup. workdir_files lists the workdir (a directory read, no file opened) by the graph export's own rule, so the backup panel's "nothing built yet" can never disagree with the export route's nothing_to_export.

Parameters:

Name	Type	Description	Default
runner	RunnerDep	The process-wide graph build runner (build state).	required
service	ServiceDep	The process-wide graph service (document statuses).	required
store	SettingsStoreDep	The per-request app-settings store (the stale flag).	required

Returns:

Type	Description
GraphStatusOut	The status snapshot; every size field is None/empty when
GraphStatusOut	nothing has been indexed, never a zero that would read like an
GraphStatusOut	empty graph.

Source code in varagity/api/routes/graph.py

@router.get("/api/graph/status")
def graph_status(runner: RunnerDep, service: ServiceDep, store: SettingsStoreDep) -> GraphStatusOut:
    """Report the graph's size, staleness, and build state (the tab's poll).

    Reads the workdir's summary sidecar rather than the graph itself, so a
    poll never walks a multi-megabyte graphml (stage-2 decision #18), and
    skips the engine entirely when nothing has been built here — an
    unbuilt graph must not make a status poll open a session and pay an
    engine's startup. ``workdir_files`` lists the workdir (a directory
    read, no file opened) by the graph export's own rule, so the backup
    panel's "nothing built yet" can never disagree with the export route's
    ``nothing_to_export``.

    Args:
        runner: The process-wide graph build runner (build state).
        service: The process-wide graph service (document statuses).
        store: The per-request app-settings store (the stale flag).

    Returns:
        The status snapshot; every size field is ``None``/empty when
        nothing has been indexed, never a zero that would read like an
        empty graph.
    """
    settings = get_settings()
    status = GraphStatusOut(
        enabled=settings.GRAPH_ENABLED,
        stale=store.is_graph_stale(),
        building=runner.building,
        last_build=runner.snapshot(),
    )
    if not service.workdir.is_dir():
        return status
    status.workdir_files = len(workdir_files(service.workdir))
    summary = load_summary(service.workdir)
    if summary is not None:
        status.entities = summary.entities
        status.relations = summary.relations
        status.message_guids = summary.message_guids
    try:
        status.documents = service.document_statuses()
    except GraphUnavailable as error:
        logger.warning("graph status: the session would not open (%s)", error)
    return status

list_graph_documents(runner)

List every file in the graph corpus directory.

A directory scan, not a table read: the graph's own record of what it indexed lives in the engine's workdir, and what this tab must show is what has been dropped in — sidecars and contacts files included. Files the last build parsed also carry their doc_id and a parse summary; files no build has scanned carry None for both, because parsing a multi-gigabyte database is a build's job. The reserved varagity-exports/ folder's backup bundles are never listed.

Parameters:

Name	Type	Description	Default
runner	RunnerDep	The process-wide graph build runner (holds the last scan).	required

Returns:

Type	Description
list[GraphDocumentOut]	One entry per file, in path order.

Source code in varagity/api/routes/graph.py

@router.get("/api/graph/documents")
def list_graph_documents(runner: RunnerDep) -> list[GraphDocumentOut]:
    """List every file in the graph corpus directory.

    A directory scan, not a table read: the graph's own record of what it
    indexed lives in the engine's workdir, and what this tab must show is
    what has been dropped in — sidecars and contacts files included. Files
    the last build parsed also carry their ``doc_id`` and a parse summary;
    files no build has scanned carry ``None`` for both, because parsing a
    multi-gigabyte database is a build's job. The reserved
    ``varagity-exports/`` folder's backup bundles are never listed.

    Args:
        runner: The process-wide graph build runner (holds the last scan).

    Returns:
        One entry per file, in path order.
    """
    graph_root = Path(get_settings().GRAPH_DOCS_PATH)
    scanned = runner.scanned_documents()
    entries: list[GraphDocumentOut] = []
    if not graph_root.is_dir():
        return entries
    for path in sorted(graph_root.rglob("*"), key=lambda item: item.as_posix()):
        if not path.is_file() or is_exports_path(path.relative_to(graph_root)):
            continue
        relative = path.relative_to(graph_root).as_posix()
        stat = path.stat()
        known = scanned.get(relative)
        entries.append(
            GraphDocumentOut(
                name=path.name,
                relative_path=relative,
                size_bytes=stat.st_size,
                modified_at=_as_utc(stat.st_mtime),
                doc_id=known.doc_id if known is not None else None,
                parse=known.parse if known is not None else None,
            )
        )
    return entries

start_graph_build(payload, runner, preflight) async

Trigger a background graph build and return its handle.

This is also how a killed build resumes. The engine keeps document statuses on disk and re-selects every in-flight or failed document, so pressing Build again after a crash finishes the work instead of repeating it — the run's summary reports what it inherited as resumed. That is why reingest is a separate, explicit flag: it is the one path that throws the working directory away.

Parameters:

Name	Type	Description	Default
payload	GraphBuildRequest	reingest (wipe and re-index — the only thing that clears the graph-stale flag), plus the message_limit / since bounds that make a minutes-long spot check possible on an archive whose full build runs for days.	required
runner	RunnerDep	The process-wide graph build runner.	required
preflight	Annotated[Callable[[], Awaitable[None]], Depends(get_graph_build_preflight)]	The awaitable reachability check (structured 503).	required

Returns:

Type	Description
GraphBuildRunOut	The new run's snapshot (state running).

Raises:

Type	Description
HTTPException	403 graph_disabled when the kill switch is off; 409 graph_build_running while a build is in flight; 409 corpus_busy while a graph backup holds the corpus (ADR-019 — the message names it); 503 <service>_unreachable when a required model service is down.

Source code in varagity/api/routes/graph.py

@router.post(
    "/api/graph/build",
    status_code=202,
    responses={
        403: {"model": ErrorResponse},
        409: {"model": ErrorResponse},
        503: {"model": ErrorResponse},
    },
)
async def start_graph_build(
    payload: GraphBuildRequest,
    runner: RunnerDep,
    preflight: Annotated[Callable[[], Awaitable[None]], Depends(get_graph_build_preflight)],
) -> GraphBuildRunOut:
    """Trigger a background graph build and return its handle.

    **This is also how a killed build resumes.** The engine keeps document
    statuses on disk and re-selects every in-flight or failed document, so
    pressing Build again after a crash finishes the work instead of
    repeating it — the run's summary reports what it inherited as
    ``resumed``. That is why ``reingest`` is a separate, explicit flag: it
    is the one path that throws the working directory away.

    Args:
        payload: ``reingest`` (wipe and re-index — the only thing that
            clears the graph-stale flag), plus the ``message_limit`` /
            ``since`` bounds that make a minutes-long spot check possible on
            an archive whose full build runs for days.
        runner: The process-wide graph build runner.
        preflight: The awaitable reachability check (structured 503).

    Returns:
        The new run's snapshot (state ``running``).

    Raises:
        HTTPException: ``403 graph_disabled`` when the kill switch is off;
            ``409 graph_build_running`` while a build is in flight; ``409
            corpus_busy`` while a graph backup holds the corpus (ADR-019 —
            the message names it); ``503 <service>_unreachable`` when a
            required model service is down.
    """
    _require_enabled()
    await preflight()
    try:
        return runner.start(
            reingest=payload.reingest,
            message_limit=payload.message_limit,
            since=payload.since,
        )
    except GraphBuildAlreadyRunning as error:
        raise HTTPException(
            status_code=409,
            detail={"code": "graph_build_running", "message": str(error)},
        ) from error
    except CorpusBusy as error:
        raise corpus_busy_error(error) from error

upload_graph_documents(files)

Upload message-source file(s) into GRAPH_DOCS_PATH (no auto-build).

Each file is written independently and then sniffed: a file no registered message source claims is deleted again and reported as unsupported_graph_source, so a mixed drop partially succeeds and the corpus directory never fills with unreadable files. The chunk-RAG ALLOWED_EXTENSIONS list does not apply — contents decide.

Copy the whole SQLite set — chat.db and its -wal/-shm sidecars: a live database keeps its most recent messages in the write-ahead log, and uploading the .db alone silently loses them. Sidecars are stored beside their database without being sniffed.

Parameters:

Name	Type	Description	Default
files	list[UploadFile]	The multipart file parts.	required

Returns:

Type	Description
GraphUploadResponse	Per-file outcomes, in upload order.

Raises:

Type	Description
HTTPException	403 graph_disabled when the kill switch is off; 422 too_many_files when the batch busts UPLOAD_MAX_FILES; 422 no_file_stored when every file was rejected on its own merits; 500 graph_docs_path_not_writable when nothing landed because the server couldn't write GRAPH_DOCS_PATH (in compose, the ./graph-docs bind mount must be writable by the api container's user).

Source code in varagity/api/routes/graph.py

@router.post("/api/graph/documents", status_code=201, responses={403: {"model": ErrorResponse}})
def upload_graph_documents(files: list[UploadFile]) -> GraphUploadResponse:
    """Upload message-source file(s) into ``GRAPH_DOCS_PATH`` (no auto-build).

    Each file is written independently and then **sniffed**: a file no
    registered message source claims is deleted again and reported as
    ``unsupported_graph_source``, so a mixed drop partially succeeds and the
    corpus directory never fills with unreadable files. The chunk-RAG
    ``ALLOWED_EXTENSIONS`` list does not apply — contents decide.

    Copy the whole SQLite set — ``chat.db`` **and** its ``-wal``/``-shm``
    sidecars: a live database keeps its most recent messages in the
    write-ahead log, and uploading the ``.db`` alone silently loses them.
    Sidecars are stored beside their database without being sniffed.

    Args:
        files: The multipart file parts.

    Returns:
        Per-file outcomes, in upload order.

    Raises:
        HTTPException: ``403 graph_disabled`` when the kill switch is off;
            ``422 too_many_files`` when the batch busts ``UPLOAD_MAX_FILES``;
            ``422 no_file_stored`` when every file was rejected on its own
            merits; ``500 graph_docs_path_not_writable`` when nothing landed
            because the server couldn't write ``GRAPH_DOCS_PATH`` (in
            compose, the ``./graph-docs`` bind mount must be writable by the
            api container's user).
    """
    _require_enabled()
    settings = get_settings()
    if len(files) > settings.UPLOAD_MAX_FILES:
        raise HTTPException(
            status_code=422,
            detail={
                "code": "too_many_files",
                "message": f"{len(files)} files exceed UPLOAD_MAX_FILES "
                f"({settings.UPLOAD_MAX_FILES}) — split the upload",
            },
        )
    graph_root = Path(settings.GRAPH_DOCS_PATH)
    not_writable = HTTPException(
        status_code=500,
        detail={
            "code": "graph_docs_path_not_writable",
            "message": (
                f"the API cannot write GRAPH_DOCS_PATH ({graph_root}) — in compose, the "
                "./graph-docs bind mount must be writable by the api container's user "
                "(see the runbook)"
            ),
        },
    )
    try:
        graph_root.mkdir(parents=True, exist_ok=True)
    except OSError as error:
        logger.error("cannot create GRAPH_DOCS_PATH %s: %s", graph_root, error)
        raise not_writable from error
    # The corpus routes' per-file streaming write, with the graph's own cap
    # and no extension gate: format is decided by the sniff below, not by a
    # name (a chat.db copy is routinely renamed).
    max_bytes = settings.GRAPH_UPLOAD_MAX_MB * 1024 * 1024
    results = [
        _sniff_stored(_store_upload(upload, graph_root, max_bytes), graph_root) for upload in files
    ]
    if not any(result.stored for result in results):
        if any(result.reason == "write_failed" for result in results):
            raise not_writable
        raise HTTPException(
            status_code=422,
            detail={
                "code": "no_file_stored",
                "message": "; ".join(f"{r.file_name}: {r.reason}" for r in results)
                or "empty upload",
            },
        )
    return GraphUploadResponse(files=results)

varagity.api.routes.backups

Backup routes: list, export, restore, status (SSE), download (ADR-019).

The /corpus "Export & restore" panel's whole server surface, mirroring the ingest routes' shape — a 202 start, a status stream that replays the backlog to late joiners, the structured error envelope:

• GET /api/backups/{corpus} lists the bundles in the corpus's reserved varagity-exports/ folder, newest first, each with its bundle.json (read alone through the zip's central directory) and the cheap restore verdict — restorable, the reason when not, and the ingest-setting drift that restoring it would flag stale. A file copied in by hand is listed too; an in-flight .partial never is.
• POST /api/backups/{corpus}/export starts an export run. Everything that can refuse it is checked first, so a refusal is a clean status, never a failed run: postgres unreachable (503), nothing ingested — or, for the graph, nothing ever built (422 nothing_to_export) — the folder unwritable (500), another operation holding the corpus (409 corpus_busy: an ingest or build, another export, a restore).
• POST /api/backups/{corpus}/restore starts a restore run, with the same up-front posture for everything a cheap look can see: the stores it writes unreachable (503), no such bundle (404 bundle_not_found), the corpus busy here (409), the bundle unsound — or already found damaged by an earlier restore — (422 bundle_invalid) or not this stack's to restore (422 bundle_incompatible — another embedding space, or for the graph another engine or engine version), and for RAG an ingest in another process holding the corpus lock (409 corpus_busy). The sha256 of every member is checked by the run itself, as it loads the bundle — one read, with progress, and before anything takes effect.
• GET /api/backups/{corpus}/status streams the run's status / progress / log events.
• GET /api/backups/{corpus}/files/{name} returns a bundle as an attachment — a plain navigation, so the browser downloads it with no CORS involved — resolving name inside the folder or answering 404 bundle_not_found.

Why neither the list nor the route hashes. A sha256 pass reads the whole bundle: the list is loaded on every panel visit, and a restore reads the bundle anyway to load it. So both show the cheap verdict, and the restore run — which verifies every member as it loads it, before its transaction commits or its staged graph is swapped in — remembers a bundle it refused, by name, size and mtime, in this process: from then on the list shows that bundle as damaged and the route refuses it, instead of offering it again, until the file changes.

Why "save to the folder" rather than "open the folder". No browser API can open a folder on the user's machine, and the API runs in a container that can't reach the desktop; the bundle lands where the ticket wants it (beside the sources, inside the bind mount that survives down -v), and the panel shows that path with Copy-path and Download.

corpus is rag or graph (ChatRequest.corpus's vocabulary) on every route. While GRAPH_ENABLED=false, every graph route answers 403 graph_disabled — except the status stream, which (like the graph build's) keeps reporting, so the tab can still render whatever ran.

BackupCorpus = Literal['rag', 'graph'] module-attribute

The corpora with backups — ChatRequest.corpus's vocabulary.

backup_status(corpus, runners) async

Stream the corpus's current (or last) backup run as SSE.

Frames: status (snapshot; also terminal, with the summary — and an export's saved path), progress (a RAG export's snapshot → documents → chunks → finalize; a graph export's snapshot → pack, in bytes → finalize; a RAG restore's validate → load → index → bm25; a graph restore's validate → extract, in bytes → swap → open), log (relayed varagity.backup lines). With no run in this API process the stream is a single idle status frame; after a terminal run it replays that run and closes. Never refused for a disabled graph: a tab must still render what ran.

Parameters:

Name	Type	Description	Default
corpus	BackupCorpus	The corpus whose backup run to follow.	required
runners	RunnersDep	The process-wide backup runners.	required

Yields:

Type	Description
AsyncIterator[ServerSentEvent]	The framed events, oldest first, then live until terminal.

Source code in varagity/api/routes/backups.py

@router.get("/api/backups/{corpus}/status", response_class=EventSourceResponse)
async def backup_status(
    corpus: BackupCorpus, runners: RunnersDep
) -> AsyncIterator[ServerSentEvent]:
    """Stream the corpus's current (or last) backup run as SSE.

    Frames: ``status`` (snapshot; also terminal, with the summary — and an
    export's saved path), ``progress`` (a RAG export's ``snapshot`` →
    ``documents`` → ``chunks`` → ``finalize``; a graph export's ``snapshot``
    → ``pack``, in bytes → ``finalize``; a RAG restore's ``validate`` →
    ``load`` → ``index`` → ``bm25``; a graph restore's ``validate`` →
    ``extract``, in bytes → ``swap`` → ``open``), ``log`` (relayed
    ``varagity.backup`` lines).
    With no run in this API process the stream is a single idle ``status``
    frame; after a terminal run it replays that run and closes. Never
    refused for a disabled graph: a tab must still render what ran.

    Args:
        corpus: The corpus whose backup run to follow.
        runners: The process-wide backup runners.

    Yields:
        The framed events, oldest first, then live until terminal.
    """
    async for event, payload in runners[corpus].subscribe():
        yield ServerSentEvent(event=event, data=payload)

download_backup(corpus, name)

Return one bundle as a file download.

A plain GET answering with Content-Disposition: attachment, so the panel's Download is an ordinary link — the browser saves the file (and offers its own "Show in folder") without a cross-origin fetch.

Parameters:

Name	Type	Description	Default
corpus	EnabledCorpusDep	The corpus the bundle belongs to.	required
name	str	The bundle's file name in the exports folder.	required

Returns:

Type	Description
FileResponse	The bundle's bytes as application/zip.

Raises:

Type	Description
HTTPException	403 graph_disabled for the graph while GRAPH_ENABLED is false; 404 bundle_not_found when no such bundle lives in the folder — which covers a name trying to escape it.

Source code in varagity/api/routes/backups.py

@router.get(
    "/api/backups/{corpus}/files/{name}",
    response_class=FileResponse,
    responses={
        200: {
            "description": "The bundle, as a `Content-Disposition: attachment` download.",
            "content": {"application/zip": {"schema": {"type": "string", "format": "binary"}}},
        },
        403: {"model": ErrorResponse},
        404: {"model": ErrorResponse},
    },
)
def download_backup(corpus: EnabledCorpusDep, name: str) -> FileResponse:
    """Return one bundle as a file download.

    A plain ``GET`` answering with ``Content-Disposition: attachment``, so
    the panel's Download is an ordinary link — the browser saves the file
    (and offers its own "Show in folder") without a cross-origin fetch.

    Args:
        corpus: The corpus the bundle belongs to.
        name: The bundle's file name in the exports folder.

    Returns:
        The bundle's bytes as ``application/zip``.

    Raises:
        HTTPException: ``403 graph_disabled`` for the graph while
            ``GRAPH_ENABLED`` is false; ``404 bundle_not_found`` when no such
            bundle lives in the folder — which covers a name trying to
            escape it.
    """
    path = _bundle_path(corpus, name)
    return FileResponse(path, media_type="application/zip", filename=name)

enabled_corpus(corpus)

Resolve the {corpus} path parameter, refusing the graph while it is switched off.

A dependency rather than a line in each route body, so it runs first — before the routes' store dependencies open a connection — and a disabled graph answers 403 whatever else is down (ADR-017's degrade semantics: a refusal, never a silent no-op).

Parameters:

Name	Type	Description	Default
corpus	BackupCorpus	The path's corpus.	required

Returns:

Type	Description
BackupCorpus	The same corpus.

Raises:

Type	Description
HTTPException	403 graph_disabled for graph while GRAPH_ENABLED is false.

Source code in varagity/api/routes/backups.py

def enabled_corpus(corpus: BackupCorpus) -> BackupCorpus:
    """Resolve the ``{corpus}`` path parameter, refusing the graph while it is switched off.

    A dependency rather than a line in each route body, so it runs *first*
    — before the routes' store dependencies open a connection — and a
    disabled graph answers ``403`` whatever else is down (ADR-017's degrade
    semantics: a refusal, never a silent no-op).

    Args:
        corpus: The path's corpus.

    Returns:
        The same corpus.

    Raises:
        HTTPException: ``403 graph_disabled`` for ``graph`` while
            ``GRAPH_ENABLED`` is false.
    """
    if corpus == "graph" and not get_settings().GRAPH_ENABLED:
        raise HTTPException(
            status_code=403,
            detail={
                "code": "graph_disabled",
                "message": "the graph subsystem is disabled (GRAPH_ENABLED=false) — "
                "turn it on in settings to back up the graph",
            },
        )
    return corpus

list_backups(corpus, damaged)

List the backup bundles in a corpus's varagity-exports/ folder.

A directory read plus one small member per bundle — the manifest is found through each zip's central directory, so a folder of large bundles lists in milliseconds. Each entry carries the cheap restore verdict (no hashing — a restore hashes, and a damage it finds is remembered here).

Parameters:

Name	Type	Description	Default
corpus	EnabledCorpusDep	The corpus whose backups to list.	required
damaged	DamagedDep	The process's remembered restore failures.	required

Returns:

Type	Description
list[BundleOut]	One entry per bundle file, newest first (empty when the folder does
list[BundleOut]	not exist yet).

Raises:

Type	Description
HTTPException	403 graph_disabled for the graph while GRAPH_ENABLED is false.

Source code in varagity/api/routes/backups.py

@router.get("/api/backups/{corpus}", responses={403: {"model": ErrorResponse}})
def list_backups(corpus: EnabledCorpusDep, damaged: DamagedDep) -> list[BundleOut]:
    """List the backup bundles in a corpus's ``varagity-exports/`` folder.

    A directory read plus one small member per bundle — the manifest is
    found through each zip's central directory, so a folder of large
    bundles lists in milliseconds. Each entry carries the cheap restore
    verdict (no hashing — a restore hashes, and a damage it finds is
    remembered here).

    Args:
        corpus: The corpus whose backups to list.
        damaged: The process's remembered restore failures.

    Returns:
        One entry per bundle file, newest first (empty when the folder does
        not exist yet).

    Raises:
        HTTPException: ``403 graph_disabled`` for the graph while
            ``GRAPH_ENABLED`` is false.
    """
    # Drift is a RAG notion (the graph records no ingest settings).
    current = current_ingest_settings() if corpus == "rag" else {}
    return [
        _bundle_out(listing, corpus, current, damaged)
        for listing in list_bundles(exports_dir(corpus), corpus)
    ]

start_export(corpus, runners, preflight, store) async

Start a background export and return its handle.

The bundle is written into <corpus root>/varagity-exports/ as varagity-<corpus>-<UTC yyyymmdd-HHMM>.zip (a same-minute repeat gets a -2 suffix — never a replace): for RAG, from one consistent snapshot of the corpus; for the graph, the engine's workdir copied under the graph's write lock, while graph chat keeps answering.

Parameters:

Name	Type	Description	Default
corpus	EnabledCorpusDep	The corpus to export.	required
runners	RunnersDep	The process-wide backup runners.	required
preflight	PreflightDep	The awaitable reachability check (structured 503).	required
store	VectorStoreDep	The per-request vector store (the RAG nothing-to-export check).	required

Returns:

Type	Description
BackupRunOut	The new run's snapshot (state running).

Raises:

Type

Description

HTTPException

403 graph_disabled for the graph while GRAPH_ENABLED is false; 503 postgres_unreachable when the database is down; 422 nothing_to_export when nothing is ingested (or, for the graph, nothing has been built); 500 docs_path_not_writable / graph_docs_path_not_writable when the exports folder can't be written; 409 corpus_busy while any operation — another export, an ingest or build, a restore — holds the corpus (the message names it).

Source code in varagity/api/routes/backups.py

@router.post(
    "/api/backups/{corpus}/export",
    status_code=202,
    responses={
        403: {"model": ErrorResponse},
        409: {"model": ErrorResponse},
        422: {"model": ErrorResponse},
        500: {"model": ErrorResponse},
        503: {"model": ErrorResponse},
    },
)
async def start_export(
    corpus: EnabledCorpusDep,
    runners: RunnersDep,
    preflight: PreflightDep,
    store: VectorStoreDep,
) -> BackupRunOut:
    """Start a background export and return its handle.

    The bundle is written into ``<corpus root>/varagity-exports/`` as
    ``varagity-<corpus>-<UTC yyyymmdd-HHMM>.zip`` (a same-minute repeat gets
    a ``-2`` suffix — never a replace): for RAG, from one consistent
    snapshot of the corpus; for the graph, the engine's workdir copied under
    the graph's write lock, while graph chat keeps answering.

    Args:
        corpus: The corpus to export.
        runners: The process-wide backup runners.
        preflight: The awaitable reachability check (structured 503).
        store: The per-request vector store (the RAG nothing-to-export
            check).

    Returns:
        The new run's snapshot (state ``running``).

    Raises:
        HTTPException: ``403 graph_disabled`` for the graph while
            ``GRAPH_ENABLED`` is false; ``503 postgres_unreachable`` when
            the database is down; ``422 nothing_to_export`` when nothing is
            ingested (or, for the graph, nothing has been built); ``500
            docs_path_not_writable`` / ``graph_docs_path_not_writable`` when
            the exports folder can't be written; ``409 corpus_busy`` while
            any operation — another export, an ingest or build, a restore —
            holds the corpus (the message names it).
    """
    await preflight(corpus, "export")
    refusal = await run_in_threadpool(_nothing_to_export, corpus, store)
    if refusal is not None:
        raise HTTPException(
            status_code=422,
            detail={"code": "nothing_to_export", "message": refusal},
        )
    await run_in_threadpool(_ensure_writable_exports_dir, corpus)
    try:
        return runners[corpus].start("export")
    except CorpusBusy as error:
        raise corpus_busy_error(error) from error

start_restore(corpus, payload, runners, preflight, store, damaged) async

Start a background restore of one bundle and return its handle.

Restore replaces, with no model call. For RAG, the corpus's rows in both stores become the bundle's, re-rooted onto this stack's DOCS_PATH — nothing parsed, chunked, contextualized or embedded — and the BM25 index is rebuilt from the restored rows. For the graph, the engine's working directory becomes the bundle's — extracted beside the live one and swapped in, the previous graph put back if the restored one won't open — and nothing is re-extracted. Everything a cheap look can see refuses the bundle here, before the 202: the list's verdict, and a damage an earlier restore of this very file found. The sha256 of every member is the run's job — it reads the bundle once, to load it, and nothing takes effect until every member checked out — so a bundle damaged inside an intact zip fails its first run (the corpus untouched) and is refused here from then on.

Parameters:

Name	Type	Description	Default
corpus	EnabledCorpusDep	The corpus to restore.	required
payload	BackupRestoreRequest	The bundle to restore, by name.	required
runners	RunnersDep	The process-wide backup runners.	required
preflight	PreflightDep	The awaitable reachability check (structured 503).	required
store	VectorStoreDep	The per-request vector store (the RAG corpus-lock check).	required
damaged	DamagedDep	The process's remembered restore failures (a bundle an earlier restore found damaged is refused here, before the run).	required

Returns:

Type	Description
BackupRunOut	The new run's snapshot (state running).

Raises:

Type

Description

HTTPException

403 graph_disabled for the graph while GRAPH_ENABLED is false; 503 postgres_unreachable / es_unreachable when a store the restore writes is down; 404 bundle_not_found for a name that is not a bundle in the folder; 409 corpus_busy while another operation holds the corpus, or (RAG) an ingest in another process holds the corpus lock; 422 bundle_invalid for an unsound bundle, or one an earlier restore found damaged; 422 bundle_incompatible for one from another embedding model or dimension, corpus, or format version — or, for the graph, another engine or engine major.minor version.

Source code in varagity/api/routes/backups.py

@router.post(
    "/api/backups/{corpus}/restore",
    status_code=202,
    responses={
        403: {"model": ErrorResponse},
        404: {"model": ErrorResponse},
        409: {"model": ErrorResponse},
        422: {"model": ErrorResponse},
        503: {"model": ErrorResponse},
    },
)
async def start_restore(
    corpus: EnabledCorpusDep,
    payload: BackupRestoreRequest,
    runners: RunnersDep,
    preflight: PreflightDep,
    store: VectorStoreDep,
    damaged: DamagedDep,
) -> BackupRunOut:
    """Start a background restore of one bundle and return its handle.

    **Restore replaces**, with no model call. For RAG, the corpus's rows in
    both stores become the bundle's, re-rooted onto this stack's
    ``DOCS_PATH`` — nothing parsed, chunked, contextualized or embedded —
    and the BM25 index is rebuilt from the restored rows. For the graph, the
    engine's working directory becomes the bundle's — extracted beside the
    live one and swapped in, the previous graph put back if the restored one
    won't open — and nothing is re-extracted. Everything a cheap look can
    see refuses the bundle here, before the ``202``: the list's verdict, and
    a damage an earlier restore of this very file found. The sha256 of every
    member is the run's job — it reads the bundle once, to load it, and
    nothing takes effect until every member checked out — so a bundle
    damaged inside an intact zip fails its first run (the corpus untouched)
    and is refused here from then on.

    Args:
        corpus: The corpus to restore.
        payload: The bundle to restore, by name.
        runners: The process-wide backup runners.
        preflight: The awaitable reachability check (structured 503).
        store: The per-request vector store (the RAG corpus-lock check).
        damaged: The process's remembered restore failures (a bundle an
            earlier restore found damaged is refused here, before the run).

    Returns:
        The new run's snapshot (state ``running``).

    Raises:
        HTTPException: ``403 graph_disabled`` for the graph while
            ``GRAPH_ENABLED`` is false; ``503 postgres_unreachable`` /
            ``es_unreachable`` when a store the restore writes is down;
            ``404 bundle_not_found`` for a name that is not a bundle in the
            folder; ``409 corpus_busy`` while another operation holds the
            corpus, or (RAG) an ingest in another process holds the corpus
            lock; ``422 bundle_invalid`` for an unsound bundle, or one an
            earlier restore found damaged; ``422 bundle_incompatible`` for
            one from another embedding model
            or dimension, corpus, or format version — or, for the graph,
            another engine or engine major.minor version.
    """
    await preflight(corpus, "restore")
    path = _bundle_path(corpus, payload.name)
    runner = runners[corpus]
    try:
        # First: a busy corpus shouldn't cost even the manifest read.
        runner.ensure_free()
    except CorpusBusy as error:
        raise corpus_busy_error(error) from error
    try:
        await run_in_threadpool(_restorable_manifest, path, corpus)
    except BundleIncompatible as error:
        raise HTTPException(
            status_code=422,
            detail={
                "code": "bundle_incompatible",
                "message": f"cannot restore {payload.name}: {error}",
            },
        ) from error
    except BundleInvalid as error:
        raise _bundle_invalid(payload.name, str(error)) from error
    remembered = await run_in_threadpool(damaged.reason_for, corpus, payload.name, path)
    if remembered is not None:
        raise _bundle_invalid(payload.name, remembered)
    # The cross-process half of the gate is RAG's alone: the graph's only
    # writer is this process, so its gate already says everything.
    if corpus == "rag" and not await run_in_threadpool(store.corpus_lock_free):
        raise HTTPException(
            status_code=409,
            detail={
                "code": "corpus_busy",
                "message": (
                    f"the {corpus} corpus is busy — an ingest in another process holds the corpus "
                    "lock (a CLI `main.py ingest` or `chat`); try again when it finishes"
                ),
            },
        )
    try:
        return runner.start("restore", payload.name)
    except CorpusBusy as error:
        raise corpus_busy_error(error) from error

varagity.api.runtime_settings

Runtime settings override layer (spec_v2 §4.7, plan decision #9).

The GUI tunes the pipeline without editing .env: overrides persist in the app_settings table (migration 002, via :class:~varagity.stores.app_settings_store.AppSettingsStore) and are merged over the env-loaded :class:~varagity.config.Settings with the exact mechanism the eval harness proved (pinned_eval_settings): export the values as environment variables — which pydantic-settings gives precedence over .env — then get_settings.cache_clear(). Every module that reads get_settings() (the convention) sees the override on its next call, so query-time knobs take effect on the next question with zero pipeline edits.

This module owns the process-env side: the :data:OVERRIDABLE catalog (which settings the GUI may override, grouped per spec §4.7, flagged when reingest-affecting), atomic apply-with-validation (an invalid merge rolls the environment back and raises, so a bad PATCH can never leave the process half-configured), and the startup replay of persisted rows. Persistence and the corpus-stale flag live in the store; the route (:mod:varagity.api.routes.settings) orchestrates both.

Single-worker caveat (plan decision #11): the environment is process-global state, which is exactly why one uvicorn worker serves the API. A PATCH racing an in-flight request can hand that request the old or new values — acceptable for the single-user posture, guarded by :data:_lock against torn writes.

REINGEST_AFFECTING = frozenset(name for name, spec in (OVERRIDABLE.items()) if spec.reingest_affecting) module-attribute

The settings whose change marks the corpus stale (spec §4.7).

OverridableSetting dataclass

Catalog entry for one GUI-overridable setting.

Attributes:

Name	Type	Description
group	str	The spec §4.7 drawer group (retrieval | generation | ingestion | graph).
reingest_affecting	bool	Whether changing it does not change content hashes (the v1 footgun) — the corpus goes stale until ingest --reingest.
choices	Callable[[], list[str]] | None	Lazy provider of the valid values for enum-like settings (lazy so registries resolve at request time, reflecting any newly registered implementation); None for free-form and numeric settings.

Source code in varagity/api/runtime_settings.py

@dataclass(frozen=True)
class OverridableSetting:
    """Catalog entry for one GUI-overridable setting.

    Attributes:
        group: The spec §4.7 drawer group (``retrieval`` | ``generation`` |
            ``ingestion`` | ``graph``).
        reingest_affecting: Whether changing it does **not** change content
            hashes (the v1 footgun) — the corpus goes stale until
            ``ingest --reingest``.
        choices: Lazy provider of the valid values for enum-like settings
            (lazy so registries resolve at request time, reflecting any
            newly registered implementation); ``None`` for free-form and
            numeric settings.
    """

    group: str
    reingest_affecting: bool = False
    choices: Callable[[], list[str]] | None = None

active_overrides()

Snapshot the overrides currently applied to the process environment.

Returns:

Type	Description
dict[str, str]	Setting name → env-string value (a copy).

Source code in varagity/api/runtime_settings.py

def active_overrides() -> dict[str, str]:
    """Snapshot the overrides currently applied to the process environment.

    Returns:
        Setting name → env-string value (a copy).
    """
    with _lock:
        return dict(_active)

apply_overrides(overrides)

Atomically make overrides the complete active override set.

The merged environment is validated by constructing :class:~varagity.config.Settings (every config validator runs — the weight pair, the rerank constraints, the vocabularies); a failure rolls the environment back to the previous override set and re-raises, so the process never serves half-applied settings.

Parameters:

Name	Type	Description	Default
overrides	Mapping[str, str]	Setting name → env-string value; must be the entire desired set (the route merges patches over :func:active_overrides). Keys must be in :data:OVERRIDABLE.	required

Returns:

Type	Description
Settings	The now-effective settings.

Raises:

Type	Description
KeyError	If a key is not overridable.
ValidationError	If the merged settings are invalid (the previous overrides remain in effect).

Source code in varagity/api/runtime_settings.py

def apply_overrides(overrides: Mapping[str, str]) -> Settings:
    """Atomically make ``overrides`` the complete active override set.

    The merged environment is validated by constructing
    :class:`~varagity.config.Settings` (every config validator runs — the
    weight pair, the rerank constraints, the vocabularies); a failure rolls
    the environment back to the previous override set and re-raises, so the
    process never serves half-applied settings.

    Args:
        overrides: Setting name → env-string value; must be the *entire*
            desired set (the route merges patches over
            :func:`active_overrides`). Keys must be in :data:`OVERRIDABLE`.

    Returns:
        The now-effective settings.

    Raises:
        KeyError: If a key is not overridable.
        pydantic.ValidationError: If the merged settings are invalid (the
            previous overrides remain in effect).
    """
    unknown = sorted(set(overrides) - set(OVERRIDABLE))
    if unknown:
        raise KeyError(f"not overridable: {unknown}; overridable: {sorted(OVERRIDABLE)}")
    global _active
    with _lock:
        previous = dict(_active)
        _swap_env(overrides)
        try:
            settings = get_settings()
        except ValidationError:
            _swap_env(previous)
            raise
        _active = dict(overrides)
        return settings

load_persisted_overrides(load)

Replay persisted overrides into the environment (API startup).

Runs after the migration runner in the lifespan, so overrides survive an api restart. Rows that no longer validate (e.g. a chunker override from a removed registry entry) or are no longer overridable are dropped from the applied set with an error log — the API must boot on env defaults rather than crash on a stale row; the row itself stays put for inspection.

Parameters:

Name	Type	Description	Default
load	Callable[[], dict[str, str]]	Zero-argument provider of the persisted rows (a bound AppSettingsStore.load_overrides; injected so unit tests need no database).	required

Source code in varagity/api/runtime_settings.py

def load_persisted_overrides(load: Callable[[], dict[str, str]]) -> None:
    """Replay persisted overrides into the environment (API startup).

    Runs after the migration runner in the lifespan, so overrides survive an
    ``api`` restart. Rows that no longer validate (e.g. a chunker override
    from a removed registry entry) or are no longer overridable are dropped
    from the *applied* set with an error log — the API must boot on env
    defaults rather than crash on a stale row; the row itself stays put for
    inspection.

    Args:
        load: Zero-argument provider of the persisted rows (a bound
            ``AppSettingsStore.load_overrides``; injected so unit tests
            need no database).
    """
    persisted = load()
    known = {name: value for name, value in persisted.items() if name in OVERRIDABLE}
    dropped = sorted(set(persisted) - set(known))
    if dropped:
        logger.error("ignoring persisted overrides that are no longer overridable: %s", dropped)
    if not known:
        return
    try:
        apply_overrides(known)
    except ValidationError as error:
        logger.error(
            "persisted overrides no longer validate — starting on env defaults "
            "(fix or clear them via PATCH /api/settings): %s",
            error,
        )
        return
    logger.info("applied %d persisted setting override(s): %s", len(known), sorted(known))

reset_for_tests()

Restore the un-overridden environment (test isolation helper).

Clears every active override, restores the baseline env snapshot, and clears the settings cache — the moral equivalent of the settings_env fixture's teardown for this module's state.

Source code in varagity/api/runtime_settings.py

def reset_for_tests() -> None:
    """Restore the un-overridden environment (test isolation helper).

    Clears every active override, restores the baseline env snapshot, and
    clears the settings cache — the moral equivalent of the ``settings_env``
    fixture's teardown for this module's state.
    """
    with _lock:
        _swap_env({})
        _active.clear()

to_env_value(value)

Convert a JSON scalar to the env-string form pydantic-settings parses.

Parameters:

Name	Type	Description	Default
value	bool | int | float | str	The override value as it arrived in the PATCH body.	required

Returns:

Type	Description
str	The environment-variable string (bools lowercase, per the .env
str	convention).

Source code in varagity/api/runtime_settings.py

def to_env_value(value: bool | int | float | str) -> str:
    """Convert a JSON scalar to the env-string form pydantic-settings parses.

    Args:
        value: The override value as it arrived in the PATCH body.

    Returns:
        The environment-variable string (bools lowercase, per the ``.env``
        convention).
    """
    if isinstance(value, bool):
        return "true" if value else "false"
    return str(value)

varagity.api.ingest_runner

Background ingest runs with a replayable live event feed (spec_v2 §4.2).

POST /api/ingest must return a run handle immediately while the corpus ingests (minutes of contextualization on a real corpus), and GET /api/ingest/status must stream per-stage/per-file progress that mirrors the CLI's rich display. This module owns both halves:

• One run at a time. :class:IngestRunner executes :func:~varagity.pipeline.ingest_flow.ingest_flow on a daemon thread — the same tracked Prefect flow the CLI runs (and, since it runs in the API process, the ingest Prometheus counters finally reach the scrape). A second start while one is running raises :class:IngestAlreadyRunning (the route's structured 409).
• One writer per corpus. A run also claims the RAG corpus's :class:~varagity.api.corpus_gate.CorpusGate for its whole life (ADR-019), so an ingest and a backup export or restore never overlap in this process: a start while a backup holds the corpus raises :class:~varagity.api.corpus_gate.CorpusBusy (409 corpus_busy). The run turns terminal and frees the corpus in one critical section, before the terminal frame: a start that sees the run finished — by its frame or by the single-flight check — always finds the corpus free.
• Progress without pipeline edits. The runner wraps the flow's @task-wrapped stages (:data:~varagity.pipeline.ingest_flow.TASK_STAGES) with event emitters — tracking and streaming compose — and passes the loader's on_file observer for exact per-file outcomes. The contextualize stage's per-chunk ticks ride a proxy around the rich progress handle it already advances. varagity.ingest log records (skips, failures, no-text warnings) are relayed as log events, so the browser sees what the terminal would.
• Replay + live fan-out. Events accumulate per run; a subscriber gets the full backlog first (connecting mid-run or after completion both render correctly), then live events until the terminal status frame.

Feed = tuple[str, BaseModel] module-attribute

One feed item: (sse_event_name, payload_model).

IngestAlreadyRunning

Bases: RuntimeError

Raised by :meth:IngestRunner.start while a run is in flight.

Source code in varagity/api/ingest_runner.py

class IngestAlreadyRunning(RuntimeError):
    """Raised by :meth:`IngestRunner.start` while a run is in flight."""

IngestRunner

Owner of the (single) background ingest run and its event feed.

Source code in varagity/api/ingest_runner.py

class IngestRunner:
    """Owner of the (single) background ingest run and its event feed."""

    def __init__(
        self,
        flow: Callable[..., IngestSummary] | None = None,
        *,
        base_stages: IngestStages | None = None,
        on_reingest_complete: Callable[[], None] | None = None,
        gate: CorpusGate | None = None,
    ) -> None:
        """Wire the runner's collaborators (all injectable for tests).

        Args:
            flow: The ingest callable, ``ingest_flow``-compatible
                (injectable so tests run without stores/models/Prefect).
            base_stages: The stage bundle to wrap with emitters; the flow's
                Prefect task stages when omitted.
            on_reingest_complete: Hook fired after a ``reingest=true`` run
                completes; defaults to clearing the corpus-stale flag.
            gate: The RAG corpus's write gate; the process-wide one when
                omitted.
        """
        self._flow = flow if flow is not None else ingest_flow
        self._gate = gate if gate is not None else get_corpus_gates()["rag"]
        self._base_stages = base_stages if base_stages is not None else TASK_STAGES
        self._on_reingest_complete = (
            on_reingest_complete if on_reingest_complete is not None else _clear_corpus_stale
        )
        self._lock = threading.Lock()
        self._run: _RunState | None = None
        self._events: list[Feed] = []
        self._subscribers: list[tuple[asyncio.AbstractEventLoop, asyncio.Queue[Feed | None]]] = []

    # ── run control ────────────────────────────────────────────────────

    def start(self, *, reingest: bool) -> IngestRunOut:
        """Start a background ingest run.

        Args:
            reingest: Delete each discovered document's previous ingest and
                re-process it (the stale-corpus action).

        Returns:
            The new run's handle/state snapshot.

        Raises:
            IngestAlreadyRunning: If a run is already in flight.
            CorpusBusy: If a backup export or restore holds the corpus.
        """
        with self._lock:
            if self._run is not None and self._run.state == "running":
                raise IngestAlreadyRunning(f"run {self._run.run_id} is still running")
            # After the single-flight check, so a second ingest keeps its own
            # ingest_already_running answer; corpus_busy means a backup.
            self._gate.claim("ingest")
            run = _RunState(run_id=uuid4().hex[:12], reingest=reingest)
            self._run = run
            self._events = []
        # Snapshot before the thread starts: the caller's handle (and the
        # feed's first frame) deterministically says "running" even if the
        # run finishes faster than this function returns.
        started = run.to_out()
        self._emit(EVENT_STATUS, IngestStatusEvent(run=started))
        thread = threading.Thread(
            target=self._execute, args=(run,), name=f"ingest-{run.run_id}", daemon=True
        )
        try:
            thread.start()
        except RuntimeError as error:  # no thread to run it: end the run, free the corpus
            self._settle(run, error=f"{type(error).__name__}: {error}")
            self._emit(EVENT_STATUS, IngestStatusEvent(run=run.to_out()))
            self._close_feed()
            raise
        return started

    def snapshot(self) -> IngestRunOut | None:
        """Report the current (or last) run without subscribing.

        Returns:
            The run snapshot, or ``None`` when nothing ever ran.
        """
        with self._lock:
            return self._run.to_out() if self._run is not None else None

    # ── the event feed ─────────────────────────────────────────────────

    async def subscribe(self) -> AsyncIterator[Feed]:
        """Stream this runner's events: full backlog first, then live.

        The stream ends after the terminal ``status`` frame (or immediately
        after an idle/terminal snapshot when nothing is running), so a
        client connecting at any point renders the same picture.

        Yields:
            ``(event_name, payload)`` feed items in emission order.
        """
        loop = asyncio.get_running_loop()
        queue: asyncio.Queue[Feed | None] = asyncio.Queue()
        with self._lock:
            backlog = list(self._events)
            live = self._run is not None and self._run.state == "running"
            if live:
                self._subscribers.append((loop, queue))
        if not backlog:
            # Nothing ever ran in this API process: say so and close.
            yield (EVENT_STATUS, IngestStatusEvent(run=self.snapshot()))
            return
        try:
            for item in backlog:
                yield item
            if not live:
                return
            while (live_item := await queue.get()) is not None:
                yield live_item
        finally:
            with self._lock:
                self._subscribers = [entry for entry in self._subscribers if entry[1] is not queue]

    def _emit(self, event: str, payload: BaseModel) -> None:
        """Append one event and fan it out to live subscribers (any thread).

        Args:
            event: The SSE event name.
            payload: Its typed payload.
        """
        item: Feed = (event, payload)
        with self._lock:
            self._events.append(item)
            subscribers = list(self._subscribers)
        for loop, queue in subscribers:
            loop.call_soon_threadsafe(queue.put_nowait, item)

    def _close_feed(self) -> None:
        """Signal end-of-stream to every live subscriber."""
        with self._lock:
            subscribers, self._subscribers = self._subscribers, []
        for loop, queue in subscribers:
            loop.call_soon_threadsafe(queue.put_nowait, None)

    # ── the worker thread ──────────────────────────────────────────────

    def _execute(self, run: _RunState) -> None:
        """Run the flow with emitting stages; publish the terminal state.

        Args:
            run: The run this thread owns.
        """
        relay = _RelayHandler(self._emit)
        ingest_logger = logging.getLogger(_RELAY_LOGGER)
        # The per-file outcome lines (skips, no-text, failures) log at INFO;
        # the feed must carry them regardless of the process LOG_LEVEL, so
        # the namespace is pinned to INFO for the run (records still pass
        # root handlers' own level filters, so the console stays quiet).
        previous_level = ingest_logger.level
        ingest_logger.setLevel(logging.INFO)
        ingest_logger.addHandler(relay)
        try:
            summary = self._flow(
                reingest=run.reingest,
                verbose=0,
                stages=self._wrap_stages(run),
                on_file=self._make_on_file(run),
            )
        except Exception as error:
            logger.exception("ingest run %s failed", run.run_id)
            self._settle(run, error=f"{type(error).__name__}: {error}")
        else:
            self._settle(run, summary=summary)
        finally:
            ingest_logger.removeHandler(relay)
            ingest_logger.setLevel(previous_level)
            # Whatever got past both arms (a BaseException) still ends the
            # run and frees the corpus; a settled run is left as it is.
            self._settle(run, error="the ingest run was interrupted")
            self._emit(EVENT_STATUS, IngestStatusEvent(run=run.to_out()))
            self._close_feed()
            if run.state == "completed":
                self._record_run_metrics(run)
                if run.reingest:
                    self._on_reingest_complete()

    def _settle(
        self,
        run: _RunState,
        *,
        summary: IngestSummary | None = None,
        error: str | None = None,
    ) -> None:
        """Make the run terminal and free the corpus — together, and once.

        One critical section, like the backup runner's ``_finish``: a start
        that finds this run finished (the single-flight check reads its
        state under the same lock) always finds the corpus free too, rather
        than losing the gate claim to the run that just ended and answering
        ``409 corpus_busy`` for it. A no-op on a run already terminal, so a
        second call can never release a claim a newer run has since made.

        Args:
            run: The run ending.
            summary: Its counters, when it completed.
            error: Its failure, when it did not (wins over ``summary``).
        """
        with self._lock:
            if run.state != "running":
                return
            run.finished_at = datetime.now(UTC)
            if error is None:
                run.state = "completed"
                run.summary = summary
            else:
                run.state = "failed"
                run.error = error
            self._gate.release()

    @staticmethod
    def _record_run_metrics(run: _RunState) -> None:
        """Publish the completed run's headline facts as gauges.

        Args:
            run: The completed run (its ``finished_at``/``summary`` are set
                by :meth:`_execute` before this is called).
        """
        assert run.finished_at is not None and run.summary is not None  # completed run
        metrics.set_last_ingest_run(
            finished_at=run.finished_at.timestamp(),
            duration_seconds=(run.finished_at - run.started_at).total_seconds(),
            documents=run.summary.ingested,
        )

    def _make_on_file(self, run: _RunState) -> Callable[[Path, str, int], None]:
        """Build the loader's per-file observer for this run.

        Args:
            run: The run to update.

        Returns:
            The observer (emits ``file_done`` progress frames).
        """

        def on_file(path: Path, outcome: str, n_chunks: int) -> None:
            with self._lock:
                run.files_done += 1
                done, total = run.files_done, run.files_total
            self._emit(
                EVENT_PROGRESS,
                IngestProgressEvent(
                    stage="file_done",
                    file=path.name,
                    outcome=outcome,
                    total=n_chunks or None,
                    files_done=done,
                    files_total=total,
                ),
            )

        return on_file

    def _wrap_stages(self, run: _RunState) -> IngestStages:
        """Wrap the base stages with progress emitters.

        The wrappers call straight through to the base stages (the Prefect
        tasks — their tracking, retries, and metrics are untouched), adding
        one ``progress`` frame per stage transition and the contextualize
        per-chunk proxy.

        Args:
            run: The run receiving the events.

        Returns:
            The emitting stage bundle.
        """
        base = self._base_stages

        def discover(docs_path: str, verbose: int | None = None) -> Any:
            buckets = base.discover(docs_path, verbose=verbose)
            with self._lock:
                run.files_total = buckets.total
            self._emit(EVENT_PROGRESS, IngestProgressEvent(stage="discover", total=buckets.total))
            return buckets

        def parse(parser: Parser, path: Path, *, verbose: int) -> RawDocument:
            with self._lock:
                run.current_file = path.name
            self._emit(EVENT_PROGRESS, IngestProgressEvent(stage="parse", file=path.name))
            return base.parse(parser, path, verbose=verbose)

        def chunk(raw: RawDocument, *, verbose: int) -> list[Document]:
            chunks = base.chunk(raw, verbose=verbose)
            file_name = raw.source_meta.get("file_name")
            self._emit(
                EVENT_PROGRESS,
                IngestProgressEvent(stage="chunk", file=file_name, total=len(chunks)),
            )
            return chunks

        def contextualize(
            *,
            document_text: str,
            chunk_texts: list[str],
            llm: LLMClient | None,
            file_name: str,
            progress: Progress,
            verbose: int,
        ) -> list[str | None]:
            self._emit(
                EVENT_PROGRESS,
                IngestProgressEvent(
                    stage="contextualize", file=file_name, current=0, total=len(chunk_texts)
                ),
            )
            proxy = _EmittingProgress(progress, self._emit, file_name, len(chunk_texts))
            return base.contextualize(
                document_text=document_text,
                chunk_texts=chunk_texts,
                llm=llm,
                file_name=file_name,
                progress=proxy,
                verbose=verbose,
            )

        def embed(
            texts: list[str], *, embeddings: EmbeddingsClient, verbose: int
        ) -> list[list[float]]:
            with self._lock:
                file_name = run.current_file
            self._emit(
                EVENT_PROGRESS,
                IngestProgressEvent(stage="embed", file=file_name, total=len(texts)),
            )
            return base.embed(texts, embeddings=embeddings, verbose=verbose)

        def store(
            records: list[ChunkRecord],
            vectors: list[list[float]],
            *,
            store: ContextualVectorDB,
            bm25: ElasticsearchBM25,
        ) -> Any:
            result = base.store(records, vectors, store=store, bm25=bm25)
            self._emit(
                EVENT_PROGRESS,
                IngestProgressEvent(stage="store", file=records[0].file_name, total=len(records)),
            )
            return result

        return IngestStages(
            discover=discover,
            parse=parse,
            chunk=chunk,
            contextualize=contextualize,
            embed=embed,
            store=store,
        )

__init__(flow=None, *, base_stages=None, on_reingest_complete=None, gate=None)

Wire the runner's collaborators (all injectable for tests).

Parameters:

Name	Type	Description	Default
flow	Callable[..., IngestSummary] | None	The ingest callable, ingest_flow-compatible (injectable so tests run without stores/models/Prefect).	None
base_stages	IngestStages | None	The stage bundle to wrap with emitters; the flow's Prefect task stages when omitted.	None
on_reingest_complete	Callable[[], None] | None	Hook fired after a reingest=true run completes; defaults to clearing the corpus-stale flag.	None
gate	CorpusGate | None	The RAG corpus's write gate; the process-wide one when omitted.	None

Source code in varagity/api/ingest_runner.py

def __init__(
    self,
    flow: Callable[..., IngestSummary] | None = None,
    *,
    base_stages: IngestStages | None = None,
    on_reingest_complete: Callable[[], None] | None = None,
    gate: CorpusGate | None = None,
) -> None:
    """Wire the runner's collaborators (all injectable for tests).

    Args:
        flow: The ingest callable, ``ingest_flow``-compatible
            (injectable so tests run without stores/models/Prefect).
        base_stages: The stage bundle to wrap with emitters; the flow's
            Prefect task stages when omitted.
        on_reingest_complete: Hook fired after a ``reingest=true`` run
            completes; defaults to clearing the corpus-stale flag.
        gate: The RAG corpus's write gate; the process-wide one when
            omitted.
    """
    self._flow = flow if flow is not None else ingest_flow
    self._gate = gate if gate is not None else get_corpus_gates()["rag"]
    self._base_stages = base_stages if base_stages is not None else TASK_STAGES
    self._on_reingest_complete = (
        on_reingest_complete if on_reingest_complete is not None else _clear_corpus_stale
    )
    self._lock = threading.Lock()
    self._run: _RunState | None = None
    self._events: list[Feed] = []
    self._subscribers: list[tuple[asyncio.AbstractEventLoop, asyncio.Queue[Feed | None]]] = []

snapshot()

Report the current (or last) run without subscribing.

Returns:

Type	Description
IngestRunOut | None	The run snapshot, or None when nothing ever ran.

Source code in varagity/api/ingest_runner.py

def snapshot(self) -> IngestRunOut | None:
    """Report the current (or last) run without subscribing.

    Returns:
        The run snapshot, or ``None`` when nothing ever ran.
    """
    with self._lock:
        return self._run.to_out() if self._run is not None else None

start(*, reingest)

Start a background ingest run.

Parameters:

Name	Type	Description	Default
reingest	bool	Delete each discovered document's previous ingest and re-process it (the stale-corpus action).	required

Returns:

Type	Description
IngestRunOut	The new run's handle/state snapshot.

Raises:

Type	Description
IngestAlreadyRunning	If a run is already in flight.
CorpusBusy	If a backup export or restore holds the corpus.

Source code in varagity/api/ingest_runner.py

def start(self, *, reingest: bool) -> IngestRunOut:
    """Start a background ingest run.

    Args:
        reingest: Delete each discovered document's previous ingest and
            re-process it (the stale-corpus action).

    Returns:
        The new run's handle/state snapshot.

    Raises:
        IngestAlreadyRunning: If a run is already in flight.
        CorpusBusy: If a backup export or restore holds the corpus.
    """
    with self._lock:
        if self._run is not None and self._run.state == "running":
            raise IngestAlreadyRunning(f"run {self._run.run_id} is still running")
        # After the single-flight check, so a second ingest keeps its own
        # ingest_already_running answer; corpus_busy means a backup.
        self._gate.claim("ingest")
        run = _RunState(run_id=uuid4().hex[:12], reingest=reingest)
        self._run = run
        self._events = []
    # Snapshot before the thread starts: the caller's handle (and the
    # feed's first frame) deterministically says "running" even if the
    # run finishes faster than this function returns.
    started = run.to_out()
    self._emit(EVENT_STATUS, IngestStatusEvent(run=started))
    thread = threading.Thread(
        target=self._execute, args=(run,), name=f"ingest-{run.run_id}", daemon=True
    )
    try:
        thread.start()
    except RuntimeError as error:  # no thread to run it: end the run, free the corpus
        self._settle(run, error=f"{type(error).__name__}: {error}")
        self._emit(EVENT_STATUS, IngestStatusEvent(run=run.to_out()))
        self._close_feed()
        raise
    return started

subscribe() async

Stream this runner's events: full backlog first, then live.

The stream ends after the terminal status frame (or immediately after an idle/terminal snapshot when nothing is running), so a client connecting at any point renders the same picture.

Yields:

Type	Description
AsyncIterator[Feed]	(event_name, payload) feed items in emission order.

Source code in varagity/api/ingest_runner.py

async def subscribe(self) -> AsyncIterator[Feed]:
    """Stream this runner's events: full backlog first, then live.

    The stream ends after the terminal ``status`` frame (or immediately
    after an idle/terminal snapshot when nothing is running), so a
    client connecting at any point renders the same picture.

    Yields:
        ``(event_name, payload)`` feed items in emission order.
    """
    loop = asyncio.get_running_loop()
    queue: asyncio.Queue[Feed | None] = asyncio.Queue()
    with self._lock:
        backlog = list(self._events)
        live = self._run is not None and self._run.state == "running"
        if live:
            self._subscribers.append((loop, queue))
    if not backlog:
        # Nothing ever ran in this API process: say so and close.
        yield (EVENT_STATUS, IngestStatusEvent(run=self.snapshot()))
        return
    try:
        for item in backlog:
            yield item
        if not live:
            return
        while (live_item := await queue.get()) is not None:
            yield live_item
    finally:
        with self._lock:
            self._subscribers = [entry for entry in self._subscribers if entry[1] is not queue]

get_ingest_runner()

Provide the process-wide runner (FastAPI dependency, override seam).

Returns:

Type	Description
IngestRunner	The lazily created singleton.

Source code in varagity/api/ingest_runner.py

def get_ingest_runner() -> IngestRunner:
    """Provide the process-wide runner (FastAPI dependency, override seam).

    Returns:
        The lazily created singleton.
    """
    global _default_runner
    if _default_runner is None:
        _default_runner = IngestRunner()
    return _default_runner

varagity.api.graph_runner

Background graph builds with a replayable live event feed (spec_graphrag §5.2).

POST /api/graph/build must return a run handle immediately: a real archive indexes at ADR-017's measured 7.17 s/message, so a full backfill is hours to days of extraction on the single llama.cpp slot. This module is :mod:varagity.api.ingest_runner's shape applied to that job — one run at a time on a daemon thread, an event feed with full backlog replay, log records relayed from the varagity.graph namespace — with the three differences a multi-day job forces:

• Resume is the normal case, not an error path. The engine enqueues documents into a durable status store and re-selects every in-flight or failed one at the top of each batch, so a killed build is finished by starting another (the runner reports what it inherited as resumed). Nothing here needs to checkpoint; the discipline is simply not to wipe unless asked.
• Progress comes from two clocks. Scanning and parsing the corpus directory takes seconds and emits per-file frames; the engine's own extraction takes hours and is sampled — :data:_SAMPLE_INTERVAL_S apart — from its document-status counts, on a second thread, while the build call is still blocked inside the flow.
• Bounds are applied before the engine sees anything. message_limit and since narrow the merged message stream so a spot check costs minutes; a bounded run then tells the session not to prune, because its render is partial by construction and pruning on its say-so would delete the rest of the archive (stage-2 decision #9).

The build itself runs through :func:varagity.pipeline.graph_flow.graph_build_flow, so every attempt is a tracked Prefect flow run, and through :class:~varagity.graph.service.GraphService, so the API stays the single writer of the graph working directory.

One writer per corpus (ADR-019): a run also claims the graph corpus's :class:~varagity.api.corpus_gate.CorpusGate for its whole life, so a build and a graph backup never overlap — a start while a backup holds the corpus raises :class:~varagity.api.corpus_gate.CorpusBusy (409 corpus_busy), and the run turns terminal and frees the corpus in one critical section, before the terminal frame — so a start that sees the run finished, by its frame or by the single-flight check, always finds the corpus free. The reserved varagity-exports/ folder, where the graph's bundles live, is never part of the scanned corpus.

Feed = tuple[str, BaseModel] module-attribute

One feed item: (sse_event_name, payload_model).

GraphBuildAlreadyRunning

Bases: RuntimeError

Raised by :meth:GraphBuildRunner.start while a build is in flight.

Source code in varagity/api/graph_runner.py

class GraphBuildAlreadyRunning(RuntimeError):
    """Raised by :meth:`GraphBuildRunner.start` while a build is in flight."""

GraphBuildRunner

Owner of the (single) background graph build and its event feed.

Source code in varagity/api/graph_runner.py

class GraphBuildRunner:
    """Owner of the (single) background graph build and its event feed."""

    def __init__(
        self,
        flow: Callable[..., BuildReport] | None = None,
        *,
        service: GraphService | None = None,
        on_reingest_complete: Callable[[], None] | None = None,
        gate: CorpusGate | None = None,
    ) -> None:
        """Wire the runner's collaborators (all injectable for tests).

        Args:
            flow: The build callable, ``graph_build_flow``-compatible
                (injectable so tests run without Prefect or an engine).
            service: The graph service to build through; the process
                singleton, resolved on first use, when omitted.
            on_reingest_complete: Hook fired after a ``reingest=true`` build
                completes; defaults to clearing the graph-stale flag.
            gate: The graph corpus's write gate; the process-wide one when
                omitted.
        """
        self._flow = flow if flow is not None else graph_build_flow
        self._gate = gate if gate is not None else get_corpus_gates()["graph"]
        self._service = service
        self._on_reingest_complete = (
            on_reingest_complete if on_reingest_complete is not None else _clear_graph_stale
        )
        self._lock = threading.Lock()
        self._run: _RunState | None = None
        self._events: list[Feed] = []
        self._subscribers: list[tuple[asyncio.AbstractEventLoop, asyncio.Queue[Feed | None]]] = []
        self._scanned: dict[str, ScannedDocument] = {}

    @property
    def service(self) -> GraphService:
        """The graph service this runner builds through.

        Returns:
            The injected service, or the process singleton — resolved
            lazily, so constructing a runner never touches the registry.
        """
        if self._service is None:
            self._service = get_graph_service()
        return self._service

    # ── run control ────────────────────────────────────────────────────

    def start(
        self,
        *,
        reingest: bool = False,
        message_limit: int | None = None,
        since: date | None = None,
    ) -> GraphBuildRunOut:
        """Start a background graph build.

        Args:
            reingest: Wipe the engine's working directory first and index
                from scratch (the only thing that clears the graph-stale
                flag).
            message_limit: Index at most this many messages, newest first.
            since: Ignore messages older than this date.

        Returns:
            The new run's handle/state snapshot.

        Raises:
            GraphBuildAlreadyRunning: If a build is already in flight.
            CorpusBusy: If a graph backup holds the corpus.
        """
        with self._lock:
            if self._run is not None and self._run.state == "running":
                raise GraphBuildAlreadyRunning(f"build {self._run.run_id} is still running")
            # After the single-flight check, so a second build keeps its own
            # graph_build_running answer; corpus_busy means a backup.
            self._gate.claim("build")
            run = _RunState(
                run_id=uuid4().hex[:12],
                reingest=reingest,
                message_limit=message_limit,
                since=since,
            )
            self._run = run
            self._events = []
        # Snapshot before the thread starts: the caller's handle (and the
        # feed's first frame) deterministically says "running" even if the
        # build finishes faster than this function returns.
        started = run.to_out()
        self._emit(EVENT_STATUS, GraphBuildStatusEvent(run=started))
        thread = threading.Thread(
            target=self._execute, args=(run,), name=f"graph-build-{run.run_id}", daemon=True
        )
        try:
            thread.start()
        except RuntimeError as error:  # no thread to run it: end the run, free the corpus
            self._settle(run, summary=None, error=f"{type(error).__name__}: {error}")
            self._emit(EVENT_STATUS, GraphBuildStatusEvent(run=run.to_out()))
            self._close_feed()
            raise
        return started

    def snapshot(self) -> GraphBuildRunOut | None:
        """Report the current (or last) build without subscribing.

        Returns:
            The run snapshot, or ``None`` when nothing ever ran.
        """
        with self._lock:
            return self._run.to_out() if self._run is not None else None

    @property
    def building(self) -> bool:
        """Whether a build is in flight in this process.

        Returns:
            ``True`` while a run has not reached a terminal state.
        """
        with self._lock:
            return self._run is not None and self._run.state == "running"

    def scanned_documents(self) -> dict[str, ScannedDocument]:
        """Report what the last build's scan parsed, keyed by relative path.

        Returns:
            A copy of the per-file parse summaries; empty before any build
            has scanned the corpus (parsing a multi-gigabyte database is a
            build's job, not a list request's).
        """
        with self._lock:
            return dict(self._scanned)

    # ── the event feed ─────────────────────────────────────────────────

    async def subscribe(self) -> AsyncIterator[Feed]:
        """Stream this runner's events: full backlog first, then live.

        The stream ends after the terminal ``status`` frame (or immediately
        after an idle/terminal snapshot when nothing is running), so a
        client connecting at any point — including hours into a backfill —
        renders the same picture.

        Yields:
            ``(event_name, payload)`` feed items in emission order.
        """
        loop = asyncio.get_running_loop()
        queue: asyncio.Queue[Feed | None] = asyncio.Queue()
        with self._lock:
            backlog = list(self._events)
            live = self._run is not None and self._run.state == "running"
            if live:
                self._subscribers.append((loop, queue))
        if not backlog:
            # Nothing ever ran in this API process: say so and close.
            yield (EVENT_STATUS, GraphBuildStatusEvent(run=self.snapshot()))
            return
        try:
            for item in backlog:
                yield item
            if not live:
                return
            while (live_item := await queue.get()) is not None:
                yield live_item
        finally:
            with self._lock:
                self._subscribers = [entry for entry in self._subscribers if entry[1] is not queue]

    def _emit(self, event: str, payload: BaseModel) -> None:
        """Append one event and fan it out to live subscribers (any thread).

        Args:
            event: The SSE event name.
            payload: Its typed payload.
        """
        item: Feed = (event, payload)
        with self._lock:
            self._events.append(item)
            subscribers = list(self._subscribers)
        for loop, queue in subscribers:
            loop.call_soon_threadsafe(queue.put_nowait, item)

    def _close_feed(self) -> None:
        """Signal end-of-stream to every live subscriber."""
        with self._lock:
            subscribers, self._subscribers = self._subscribers, []
        for loop, queue in subscribers:
            loop.call_soon_threadsafe(queue.put_nowait, None)

    # ── the worker thread ──────────────────────────────────────────────

    def _execute(self, run: _RunState) -> None:
        """Scan, bound, optionally reset, build; publish the terminal state.

        Args:
            run: The run this thread owns (this thread is its only writer,
                so its counters are read here without the lock and written
                under it — the feed's readers see whole snapshots).
        """
        relay = _RelayHandler(self._emit)
        graph_logger = logging.getLogger(_RELAY_LOGGER)
        # The per-file parse summaries and the build diff log at INFO; the
        # feed must carry them regardless of the process LOG_LEVEL, so the
        # namespace is pinned to INFO for the run (records still pass root
        # handlers' own level filters, so the console stays quiet).
        previous_level = graph_logger.level
        graph_logger.setLevel(logging.INFO)
        graph_logger.addHandler(relay)
        try:
            report = self._run_build(run)
        except Exception as error:
            logger.exception("graph build %s failed", run.run_id)
            self._settle(
                run, summary=self._summary(run, None), error=f"{type(error).__name__}: {error}"
            )
        else:
            self._settle(run, summary=self._summary(run, report))
        finally:
            graph_logger.removeHandler(relay)
            graph_logger.setLevel(previous_level)
            # Whatever got past both arms (a BaseException) still ends the
            # run and frees the corpus; a settled run is left as it is.
            self._settle(run, summary=None, error="the graph build was interrupted")
            self._emit(EVENT_STATUS, GraphBuildStatusEvent(run=run.to_out()))
            self._close_feed()
            if run.state == "completed" and run.reingest:
                self._on_reingest_complete()

    def _settle(
        self,
        run: _RunState,
        *,
        summary: GraphBuildSummaryOut | None,
        error: str | None = None,
    ) -> None:
        """Make the run terminal and free the corpus — together, and once.

        One critical section, like the backup runner's ``_finish``: a start
        that finds this build finished (the single-flight check reads its
        state under the same lock) always finds the corpus free too, rather
        than losing the gate claim to the build that just ended and
        answering ``409 corpus_busy`` for it. A no-op on a run already
        terminal, so a second call can never release a claim a newer build
        has since made.

        Args:
            run: The run ending.
            summary: Its final counters (also kept for a failed build).
            error: Its failure, when it did not complete.
        """
        with self._lock:
            if run.state != "running":
                return
            run.finished_at = datetime.now(UTC)
            run.summary = summary
            if error is None:
                run.state = "completed"
            else:
                run.state = "failed"
                run.error = error
            self._gate.release()

    def _run_build(self, run: _RunState) -> BuildReport:
        """Do the run's work on the worker thread.

        Args:
            run: The run being executed.

        Returns:
            The engine's build report.
        """
        batches = self._scan(run)
        bounded = apply_bounds(batches, message_limit=run.message_limit, since=run.since)
        indexed = sum(len(batch.messages) for batch in bounded)
        with self._lock:
            run.messages_indexed = indexed
        if run.bounded:
            self._emit(
                EVENT_PROGRESS,
                GraphBuildProgressEvent(stage="bound", current=indexed, total=run.messages_parsed),
            )
        if run.reingest:
            self._reset_workdir()
        resumed = self._inflight_documents()
        with self._lock:
            run.resumed = resumed
        if resumed:
            logger.info("resuming %d document(s) an earlier build left in flight", resumed)
        self._emit(EVENT_PROGRESS, GraphBuildProgressEvent(stage="index", total=indexed))
        stop = threading.Event()
        sampler = threading.Thread(
            target=self._sample_statuses,
            args=(stop,),
            name=f"graph-sample-{run.run_id}",
            daemon=True,
        )
        sampler.start()
        try:
            # Boxed so the archive never serializes into the flow-run record
            # (Prefect's server caps parameters at 512 KB).
            return self._flow(
                self.service, GraphBatches(bounded), prune_removed=not run.bounded, verbose=0
            )
        finally:
            stop.set()
            sampler.join(timeout=_SAMPLE_INTERVAL_S * 2)

    def _scan(self, run: _RunState) -> list[MessageBatch]:
        """Parse every claimable file under ``GRAPH_DOCS_PATH``.

        A file that raises is counted and skipped rather than failing the
        build: one corrupt export must not cost the rest of an archive.

        Args:
            run: The run to update (counters plus the cached parse
                summaries the corpus listing reads).

        Returns:
            One batch per parsed file, in scan order.
        """
        root = Path(get_settings().GRAPH_DOCS_PATH)
        paths = scan_graph_corpus(root)
        with self._lock:
            run.files_scanned = len(paths)
            self._scanned = {}
        self._emit(EVENT_PROGRESS, GraphBuildProgressEvent(stage="scan", total=len(paths)))
        batches: list[MessageBatch] = []
        for index, path in enumerate(paths, start=1):
            if find_message_source(path) is None:
                continue  # sidecars, contacts files — listed, never parsed
            self._emit(
                EVENT_PROGRESS,
                GraphBuildProgressEvent(
                    stage="parse", file=path.name, current=index, total=len(paths)
                ),
            )
            try:
                batch = batch_for_path(path, root)
            except Exception as error:
                logger.warning("could not parse graph source %s: %s", path, error)
                with self._lock:
                    run.files_failed += 1
                continue
            batches.append(batch)
            scanned = ScannedDocument(
                relative_path=batch.relative_path,
                doc_id=batch.doc_id,
                parse=parse_summary(batch),
            )
            with self._lock:
                run.files_parsed += 1
                run.messages_parsed += len(batch.messages)
                self._scanned[batch.relative_path] = scanned
        return batches

    def _reset_workdir(self) -> None:
        """Drop the engine's working directory so the build starts clean.

        The session is closed first (the engine holds open handles on the
        files about to be removed) and reopened straight after, which both
        recreates the directory and keeps the window in which a concurrent
        *read* could see a half-removed workdir as short as an open takes.
        Concurrent writes cannot race this: the run holds the graph corpus's
        gate, so no backup export can be copying the files being removed
        (ADR-019), and the wipe itself runs under the service's write lock,
        like every other change to the workdir.
        """
        workdir = self.service.workdir
        logger.info("reingest: wiping the graph working directory %s", workdir)
        self._emit(EVENT_PROGRESS, GraphBuildProgressEvent(stage="reset"))
        with self.service.writing():
            self.service.close()
            shutil.rmtree(workdir, ignore_errors=True)
            self.service.session()

    def _document_statuses(self) -> dict[str, int]:
        """Read the engine's document counts, tolerating an unopenable graph.

        Returns:
            Status name → document count; empty when the engine would not
            say (which a progress reader treats as "no news", never as
            "zero documents").
        """
        try:
            return self.service.document_statuses()
        except Exception:
            logger.debug("graph document statuses unavailable", exc_info=True)
            return {}

    def _inflight_documents(self) -> int:
        """Count documents a previous build left owing work.

        Returns:
            How many documents this run inherits — the ``resumed`` number
            that makes a kill/restart visible rather than mysterious.
        """
        statuses = self._document_statuses()
        return sum(count for name, count in statuses.items() if name.lower() in _INFLIGHT_STATUSES)

    def _sample_statuses(self, stop: threading.Event) -> None:
        """Emit engine document-status progress until the build finishes.

        Args:
            stop: Set by the build thread when the engine call returns.
        """
        while not stop.wait(_SAMPLE_INTERVAL_S):
            statuses = self._document_statuses()
            if not statuses:
                continue
            self._emit(
                EVENT_PROGRESS,
                GraphBuildProgressEvent(
                    stage="process",
                    docs_done=statuses.get("processed", 0),
                    docs_total=sum(statuses.values()),
                ),
            )

    def _summary(self, run: _RunState, report: BuildReport | None) -> GraphBuildSummaryOut:
        """Fold the run's counters and the engine's report into the wire summary.

        Args:
            run: The finished run.
            report: The engine's build report, or ``None`` when the run
                failed before the engine returned.

        Returns:
            The summary. A failed run still reports what it scanned, which
            is what makes "it never found the file" diagnosable.
        """
        return GraphBuildSummaryOut(
            files_scanned=run.files_scanned,
            files_parsed=run.files_parsed,
            files_failed=run.files_failed,
            messages_parsed=run.messages_parsed,
            messages_indexed=run.messages_indexed,
            messages_seen=report.messages_seen if report is not None else 0,
            wall_clock_s=report.wall_clock_s if report is not None else 0.0,
            resumed=run.resumed,
            documents=self._document_statuses(),
            failures=list(report.failures) if report is not None else [],
        )

building property

Whether a build is in flight in this process.

Returns:

Type	Description
bool	True while a run has not reached a terminal state.

service property

The graph service this runner builds through.

Returns:

Type	Description
GraphService	The injected service, or the process singleton — resolved
GraphService	lazily, so constructing a runner never touches the registry.

__init__(flow=None, *, service=None, on_reingest_complete=None, gate=None)

Wire the runner's collaborators (all injectable for tests).

Parameters:

Name	Type	Description	Default
flow	Callable[..., BuildReport] | None	The build callable, graph_build_flow-compatible (injectable so tests run without Prefect or an engine).	None
service	GraphService | None	The graph service to build through; the process singleton, resolved on first use, when omitted.	None
on_reingest_complete	Callable[[], None] | None	Hook fired after a reingest=true build completes; defaults to clearing the graph-stale flag.	None
gate	CorpusGate | None	The graph corpus's write gate; the process-wide one when omitted.	None

Source code in varagity/api/graph_runner.py

def __init__(
    self,
    flow: Callable[..., BuildReport] | None = None,
    *,
    service: GraphService | None = None,
    on_reingest_complete: Callable[[], None] | None = None,
    gate: CorpusGate | None = None,
) -> None:
    """Wire the runner's collaborators (all injectable for tests).

    Args:
        flow: The build callable, ``graph_build_flow``-compatible
            (injectable so tests run without Prefect or an engine).
        service: The graph service to build through; the process
            singleton, resolved on first use, when omitted.
        on_reingest_complete: Hook fired after a ``reingest=true`` build
            completes; defaults to clearing the graph-stale flag.
        gate: The graph corpus's write gate; the process-wide one when
            omitted.
    """
    self._flow = flow if flow is not None else graph_build_flow
    self._gate = gate if gate is not None else get_corpus_gates()["graph"]
    self._service = service
    self._on_reingest_complete = (
        on_reingest_complete if on_reingest_complete is not None else _clear_graph_stale
    )
    self._lock = threading.Lock()
    self._run: _RunState | None = None
    self._events: list[Feed] = []
    self._subscribers: list[tuple[asyncio.AbstractEventLoop, asyncio.Queue[Feed | None]]] = []
    self._scanned: dict[str, ScannedDocument] = {}

scanned_documents()

Report what the last build's scan parsed, keyed by relative path.

Returns:

Type	Description
dict[str, ScannedDocument]	A copy of the per-file parse summaries; empty before any build
dict[str, ScannedDocument]	has scanned the corpus (parsing a multi-gigabyte database is a
dict[str, ScannedDocument]	build's job, not a list request's).

Source code in varagity/api/graph_runner.py

def scanned_documents(self) -> dict[str, ScannedDocument]:
    """Report what the last build's scan parsed, keyed by relative path.

    Returns:
        A copy of the per-file parse summaries; empty before any build
        has scanned the corpus (parsing a multi-gigabyte database is a
        build's job, not a list request's).
    """
    with self._lock:
        return dict(self._scanned)

snapshot()

Report the current (or last) build without subscribing.

Returns:

Type	Description
GraphBuildRunOut | None	The run snapshot, or None when nothing ever ran.

Source code in varagity/api/graph_runner.py

def snapshot(self) -> GraphBuildRunOut | None:
    """Report the current (or last) build without subscribing.

    Returns:
        The run snapshot, or ``None`` when nothing ever ran.
    """
    with self._lock:
        return self._run.to_out() if self._run is not None else None

start(*, reingest=False, message_limit=None, since=None)

Start a background graph build.

Parameters:

Name	Type	Description	Default
reingest	bool	Wipe the engine's working directory first and index from scratch (the only thing that clears the graph-stale flag).	False
message_limit	int | None	Index at most this many messages, newest first.	None
since	date | None	Ignore messages older than this date.	None

Returns:

Type	Description
GraphBuildRunOut	The new run's handle/state snapshot.

Raises:

Type	Description
GraphBuildAlreadyRunning	If a build is already in flight.
CorpusBusy	If a graph backup holds the corpus.

Source code in varagity/api/graph_runner.py

def start(
    self,
    *,
    reingest: bool = False,
    message_limit: int | None = None,
    since: date | None = None,
) -> GraphBuildRunOut:
    """Start a background graph build.

    Args:
        reingest: Wipe the engine's working directory first and index
            from scratch (the only thing that clears the graph-stale
            flag).
        message_limit: Index at most this many messages, newest first.
        since: Ignore messages older than this date.

    Returns:
        The new run's handle/state snapshot.

    Raises:
        GraphBuildAlreadyRunning: If a build is already in flight.
        CorpusBusy: If a graph backup holds the corpus.
    """
    with self._lock:
        if self._run is not None and self._run.state == "running":
            raise GraphBuildAlreadyRunning(f"build {self._run.run_id} is still running")
        # After the single-flight check, so a second build keeps its own
        # graph_build_running answer; corpus_busy means a backup.
        self._gate.claim("build")
        run = _RunState(
            run_id=uuid4().hex[:12],
            reingest=reingest,
            message_limit=message_limit,
            since=since,
        )
        self._run = run
        self._events = []
    # Snapshot before the thread starts: the caller's handle (and the
    # feed's first frame) deterministically says "running" even if the
    # build finishes faster than this function returns.
    started = run.to_out()
    self._emit(EVENT_STATUS, GraphBuildStatusEvent(run=started))
    thread = threading.Thread(
        target=self._execute, args=(run,), name=f"graph-build-{run.run_id}", daemon=True
    )
    try:
        thread.start()
    except RuntimeError as error:  # no thread to run it: end the run, free the corpus
        self._settle(run, summary=None, error=f"{type(error).__name__}: {error}")
        self._emit(EVENT_STATUS, GraphBuildStatusEvent(run=run.to_out()))
        self._close_feed()
        raise
    return started

subscribe() async

Stream this runner's events: full backlog first, then live.

The stream ends after the terminal status frame (or immediately after an idle/terminal snapshot when nothing is running), so a client connecting at any point — including hours into a backfill — renders the same picture.

Yields:

Type	Description
AsyncIterator[Feed]	(event_name, payload) feed items in emission order.

Source code in varagity/api/graph_runner.py

async def subscribe(self) -> AsyncIterator[Feed]:
    """Stream this runner's events: full backlog first, then live.

    The stream ends after the terminal ``status`` frame (or immediately
    after an idle/terminal snapshot when nothing is running), so a
    client connecting at any point — including hours into a backfill —
    renders the same picture.

    Yields:
        ``(event_name, payload)`` feed items in emission order.
    """
    loop = asyncio.get_running_loop()
    queue: asyncio.Queue[Feed | None] = asyncio.Queue()
    with self._lock:
        backlog = list(self._events)
        live = self._run is not None and self._run.state == "running"
        if live:
            self._subscribers.append((loop, queue))
    if not backlog:
        # Nothing ever ran in this API process: say so and close.
        yield (EVENT_STATUS, GraphBuildStatusEvent(run=self.snapshot()))
        return
    try:
        for item in backlog:
            yield item
        if not live:
            return
        while (live_item := await queue.get()) is not None:
            yield live_item
    finally:
        with self._lock:
            self._subscribers = [entry for entry in self._subscribers if entry[1] is not queue]

ScannedDocument

Bases: BaseModel

One graph source file as the last build's scan found it.

Attributes:

Name	Type	Description
relative_path	str	The file's POSIX path under GRAPH_DOCS_PATH.
doc_id	str	The stable id derived for it (relative path + byte hash).
parse	GraphParseSummary	What the scan parsed out of it.

Source code in varagity/api/graph_runner.py

class ScannedDocument(BaseModel):
    """One graph source file as the last build's scan found it.

    Attributes:
        relative_path: The file's POSIX path under ``GRAPH_DOCS_PATH``.
        doc_id: The stable id derived for it (relative path + byte hash).
        parse: What the scan parsed out of it.
    """

    relative_path: str
    doc_id: str
    parse: GraphParseSummary

apply_bounds(batches, *, message_limit, since)

Narrow parsed batches to the messages a bounded build should index.

since is a date floor and message_limit keeps the newest messages — both bound toward the recent end, which is what a spot check on a decade-long archive wants to look at. The cap applies to the batches' merged timestamp order, so it means "N messages", not "N per file"; batch identity survives, because a build's document ids and the manifest diff are derived from it.

Parameters:

Name	Type	Description	Default
batches	Sequence[MessageBatch]	Parsed source files.	required
message_limit	int | None	Keep at most this many messages; None keeps all.	required
since	date | None	Drop messages older than this date; None keeps all.	required

Returns:

Type	Description
list[MessageBatch]	The narrowed batches, in input order, each keeping only its
list[MessageBatch]	surviving messages. A batch left with none is dropped: handing the
list[MessageBatch]	engine an empty document would claim the conversation is empty.

Source code in varagity/api/graph_runner.py

def apply_bounds(
    batches: Sequence[MessageBatch],
    *,
    message_limit: int | None,
    since: date | None,
) -> list[MessageBatch]:
    """Narrow parsed batches to the messages a bounded build should index.

    ``since`` is a date floor and ``message_limit`` keeps the **newest**
    messages — both bound toward the recent end, which is what a spot check
    on a decade-long archive wants to look at. The cap applies to the
    batches' *merged* timestamp order, so it means "N messages", not "N per
    file"; batch identity survives, because a build's document ids and the
    manifest diff are derived from it.

    Args:
        batches: Parsed source files.
        message_limit: Keep at most this many messages; ``None`` keeps all.
        since: Drop messages older than this date; ``None`` keeps all.

    Returns:
        The narrowed batches, in input order, each keeping only its
        surviving messages. A batch left with none is dropped: handing the
        engine an empty document would claim the conversation is empty.
    """
    if message_limit is None and since is None:
        return list(batches)

    def in_range(message: SourceMessage) -> bool:
        return since is None or message.timestamp.date() >= since

    kept: set[str] | None = None
    if message_limit is not None:
        ordered = sorted(
            (message for batch in batches for message in batch.messages if in_range(message)),
            key=lambda message: (message.timestamp, message.guid),
        )
        kept = {message.guid for message in ordered[-message_limit:]}
    narrowed: list[MessageBatch] = []
    for batch in batches:
        messages = [
            message
            for message in batch.messages
            if in_range(message) and (kept is None or message.guid in kept)
        ]
        if messages:
            narrowed.append(batch.model_copy(update={"messages": messages}))
    return narrowed

get_graph_build_runner()

Provide the process-wide graph build runner (FastAPI dependency, override seam).

Returns:

Type	Description
GraphBuildRunner	The lazily created singleton. Construction resolves nothing — the
GraphBuildRunner	service (and through it the engine registry) is looked up on the
GraphBuildRunner	first build.

Source code in varagity/api/graph_runner.py

def get_graph_build_runner() -> GraphBuildRunner:
    """Provide the process-wide graph build runner (FastAPI dependency, override seam).

    Returns:
        The lazily created singleton. Construction resolves nothing — the
        service (and through it the engine registry) is looked up on the
        first build.
    """
    global _default_runner
    with _default_runner_lock:
        if _default_runner is None:
            _default_runner = GraphBuildRunner()
        return _default_runner

parse_summary(batch)

Summarize one parsed source file for the corpus listing.

Parameters:

Name	Type	Description	Default
batch	MessageBatch	The parsed batch.	required

Returns:

Type	Description
GraphParseSummary	Its message/thread counts and time span (both timestamps None
GraphParseSummary	for a file that parsed to nothing).

Source code in varagity/api/graph_runner.py

def parse_summary(batch: MessageBatch) -> GraphParseSummary:
    """Summarize one parsed source file for the corpus listing.

    Args:
        batch: The parsed batch.

    Returns:
        Its message/thread counts and time span (both timestamps ``None``
        for a file that parsed to nothing).
    """
    timestamps = [message.timestamp for message in batch.messages]
    return GraphParseSummary(
        messages=len(batch.messages),
        threads=len({message.thread_id for message in batch.messages}),
        first=min(timestamps, default=None),
        last=max(timestamps, default=None),
    )

scan_graph_corpus(root)

List every file under the graph corpus root, in stable path order.

Parameters:

Name	Type	Description	Default
root	Path	GRAPH_DOCS_PATH.	required

Returns:

Type	Description
list[Path]	Every regular file under root, sorted by relative path — files
list[Path]	no registered source claims included (a contacts list, a
list[Path]	chat.db's -wal sidecar). The caller decides what to do with
list[Path]	them; the corpus listing shows the user everything they dropped in.
list[Path]	The one exception is the reserved varagity-exports/ folder: its
list[Path]	backup bundles are never corpus, so a build never counts them as
list[Path]	scanned files.

Source code in varagity/api/graph_runner.py

def scan_graph_corpus(root: Path) -> list[Path]:
    """List every file under the graph corpus root, in stable path order.

    Args:
        root: ``GRAPH_DOCS_PATH``.

    Returns:
        Every regular file under ``root``, sorted by relative path — files
        no registered source claims included (a contacts list, a
        ``chat.db``'s ``-wal`` sidecar). The caller decides what to do with
        them; the corpus listing shows the user everything they dropped in.
        The one exception is the reserved ``varagity-exports/`` folder: its
        backup bundles are never corpus, so a build never counts them as
        scanned files.
    """
    if not root.is_dir():
        return []
    return sorted(
        (
            path
            for path in root.rglob("*")
            if path.is_file() and not is_exports_path(path.relative_to(root))
        ),
        key=lambda p: p.as_posix(),
    )

varagity.api.corpus_gate

One writer per corpus, across every runner that writes it (ADR-019).

The ingest runner, the graph build runner, and the backup runner each keep their own single-flight check, which stops a second run of the same kind (ingest_already_running, graph_build_running). A corpus needs more than that once backups exist: an export must not snapshot a corpus a restore is replacing, and a restore must not replace one an ingest is writing. The gate is that shared rule — one :class:CorpusGate per corpus, claimed for the whole of a run by whichever operation got there first, and refused to everyone else with :class:CorpusBusy (the routes' 409 corpus_busy).

It is deliberately small and in-process: the API is the only process that runs these operations through the GUI (the graph's single writer by design), so a threading.Lock is the whole mechanism. Claims never block — a busy corpus is an answer, not a wait.

The cross-process half (RAG only). Some RAG writers run in their own process — uv run main.py ingest, and main.py chat, which ingests on startup — where no in-process gate can see them. The RAG corpus therefore also has a Postgres advisory lock (:meth:~varagity.stores.vector_store.ContextualVectorDB.corpus_lock): every ingest, in any process, holds it shared for its whole run, and a restore holds it exclusive. The gate orders the API's own operations (and names the holder in the 409); the lock is what keeps a CLI ingest and a restore apart. The graph needs no such half — its single writer is this process.

Operation = Literal['ingest', 'build', 'export', 'restore'] module-attribute

Every operation that writes (or snapshots) a corpus.

CorpusBusy

Bases: RuntimeError

Raised by :meth:CorpusGate.claim while another operation holds the corpus.

Attributes:

Name	Type	Description
corpus		The corpus that is busy.
holder	Operation	The operation holding it — named in the message, so the 409 corpus_busy says what to wait for.

Source code in varagity/api/corpus_gate.py

class CorpusBusy(RuntimeError):
    """Raised by :meth:`CorpusGate.claim` while another operation holds the corpus.

    Attributes:
        corpus: The corpus that is busy.
        holder: The operation holding it — named in the message, so the
            ``409 corpus_busy`` says what to wait for.
    """

    def __init__(self, corpus: str, holder: Operation) -> None:
        """Name the busy corpus and its holder.

        Args:
            corpus: The corpus that is busy.
            holder: The operation holding it.
        """
        article = _ARTICLES.get(holder, "a")
        super().__init__(
            f"the {corpus} corpus is busy — {article} {holder} is running; "
            "try again when it finishes"
        )
        self.corpus = corpus
        self.holder: Operation = holder

__init__(corpus, holder)

Name the busy corpus and its holder.

Parameters:

Name	Type	Description	Default
corpus	str	The corpus that is busy.	required
holder	Operation	The operation holding it.	required

Source code in varagity/api/corpus_gate.py

def __init__(self, corpus: str, holder: Operation) -> None:
    """Name the busy corpus and its holder.

    Args:
        corpus: The corpus that is busy.
        holder: The operation holding it.
    """
    article = _ARTICLES.get(holder, "a")
    super().__init__(
        f"the {corpus} corpus is busy — {article} {holder} is running; "
        "try again when it finishes"
    )
    self.corpus = corpus
    self.holder: Operation = holder

CorpusGate

The non-blocking, single-holder write claim on one corpus.

Attributes:

Name	Type	Description
corpus		The corpus this gate guards ("rag" | "graph").

Source code in varagity/api/corpus_gate.py

class CorpusGate:
    """The non-blocking, single-holder write claim on one corpus.

    Attributes:
        corpus: The corpus this gate guards (``"rag"`` | ``"graph"``).
    """

    def __init__(self, corpus: str) -> None:
        """Create an unclaimed gate.

        Args:
            corpus: The corpus this gate guards.
        """
        self.corpus = corpus
        self._lock = threading.Lock()
        self._holder: Operation | None = None

    def claim(self, operation: Operation) -> None:
        """Take the corpus for ``operation``, or refuse at once.

        Args:
            operation: The operation starting.

        Raises:
            CorpusBusy: If any operation (the same kind included) already
                holds the corpus.
        """
        with self._lock:
            if self._holder is not None:
                raise CorpusBusy(self.corpus, self._holder)
            self._holder = operation

    def release(self) -> None:
        """Give the corpus back (idempotent: releasing a free gate is a no-op)."""
        with self._lock:
            self._holder = None

    def ensure_free(self) -> None:
        """Refuse at once if any operation holds the corpus — without claiming it.

        For routes that should answer ``409`` *before* doing expensive work
        (a restore hashing a bundle) or that write outside any run (the
        document deletes). The claim that follows can still lose a race;
        it raises the same error.

        Raises:
            CorpusBusy: If the corpus is held.
        """
        holder = self.holder
        if holder is not None:
            raise CorpusBusy(self.corpus, holder)

    def ensure_not_held_by(self, operations: Collection[Operation]) -> None:
        """Refuse at once if one of ``operations`` holds the corpus — without claiming it.

        :meth:`ensure_free` narrowed to the operations a caller actually
        conflicts with: a document delete is fine beside an ingest but not
        beside a backup, a graph source delete only clashes with a restore.

        Args:
            operations: The holders that refuse the caller.

        Raises:
            CorpusBusy: If one of them holds the corpus.
        """
        holder = self.holder
        if holder is not None and holder in operations:
            raise CorpusBusy(self.corpus, holder)

    @property
    def holder(self) -> Operation | None:
        """The operation holding the corpus right now.

        Returns:
            Its name, or ``None`` when the corpus is free.
        """
        with self._lock:
            return self._holder

holder property

The operation holding the corpus right now.

Returns:

Type	Description
Operation | None	Its name, or None when the corpus is free.

__init__(corpus)

Create an unclaimed gate.

Parameters:

Name	Type	Description	Default
corpus	str	The corpus this gate guards.	required

Source code in varagity/api/corpus_gate.py

def __init__(self, corpus: str) -> None:
    """Create an unclaimed gate.

    Args:
        corpus: The corpus this gate guards.
    """
    self.corpus = corpus
    self._lock = threading.Lock()
    self._holder: Operation | None = None

claim(operation)

Take the corpus for operation, or refuse at once.

Parameters:

Name	Type	Description	Default
operation	Operation	The operation starting.	required

Raises:

Type	Description
CorpusBusy	If any operation (the same kind included) already holds the corpus.

Source code in varagity/api/corpus_gate.py

def claim(self, operation: Operation) -> None:
    """Take the corpus for ``operation``, or refuse at once.

    Args:
        operation: The operation starting.

    Raises:
        CorpusBusy: If any operation (the same kind included) already
            holds the corpus.
    """
    with self._lock:
        if self._holder is not None:
            raise CorpusBusy(self.corpus, self._holder)
        self._holder = operation

ensure_free()

Refuse at once if any operation holds the corpus — without claiming it.

For routes that should answer 409 before doing expensive work (a restore hashing a bundle) or that write outside any run (the document deletes). The claim that follows can still lose a race; it raises the same error.

Raises:

Type	Description
CorpusBusy	If the corpus is held.

Source code in varagity/api/corpus_gate.py

def ensure_free(self) -> None:
    """Refuse at once if any operation holds the corpus — without claiming it.

    For routes that should answer ``409`` *before* doing expensive work
    (a restore hashing a bundle) or that write outside any run (the
    document deletes). The claim that follows can still lose a race;
    it raises the same error.

    Raises:
        CorpusBusy: If the corpus is held.
    """
    holder = self.holder
    if holder is not None:
        raise CorpusBusy(self.corpus, holder)

ensure_not_held_by(operations)

Refuse at once if one of operations holds the corpus — without claiming it.

:meth:ensure_free narrowed to the operations a caller actually conflicts with: a document delete is fine beside an ingest but not beside a backup, a graph source delete only clashes with a restore.

Parameters:

Name	Type	Description	Default
operations	Collection[Operation]	The holders that refuse the caller.	required

Raises:

Type	Description
CorpusBusy	If one of them holds the corpus.

Source code in varagity/api/corpus_gate.py

def ensure_not_held_by(self, operations: Collection[Operation]) -> None:
    """Refuse at once if one of ``operations`` holds the corpus — without claiming it.

    :meth:`ensure_free` narrowed to the operations a caller actually
    conflicts with: a document delete is fine beside an ingest but not
    beside a backup, a graph source delete only clashes with a restore.

    Args:
        operations: The holders that refuse the caller.

    Raises:
        CorpusBusy: If one of them holds the corpus.
    """
    holder = self.holder
    if holder is not None and holder in operations:
        raise CorpusBusy(self.corpus, holder)

release()

Give the corpus back (idempotent: releasing a free gate is a no-op).

Source code in varagity/api/corpus_gate.py

def release(self) -> None:
    """Give the corpus back (idempotent: releasing a free gate is a no-op)."""
    with self._lock:
        self._holder = None

get_corpus_gates()

Provide the process-wide gates, keyed by corpus (FastAPI dependency, override seam).

Returns:

Type	Description
dict[str, CorpusGate]	{"rag": …, "graph": …} — the same two objects for the life of
dict[str, CorpusGate]	the process, so every runner that claims a corpus shares its gate.

Source code in varagity/api/corpus_gate.py

def get_corpus_gates() -> dict[str, CorpusGate]:
    """Provide the process-wide gates, keyed by corpus (FastAPI dependency, override seam).

    Returns:
        ``{"rag": …, "graph": …}`` — the same two objects for the life of
        the process, so every runner that claims a corpus shares its gate.
    """
    return _gates

varagity.api.backup_runner

Background backup runs with a replayable live event feed (ADR-019).

POST /api/backups/{corpus}/export must answer at once while the bundle is written, and GET /api/backups/{corpus}/status must stream where the run is — the same contract ingest and the graph build already keep, so the /corpus panels fold all three with the same kind of pure reducer:

• One writer per corpus. :meth:BackupRunner.start claims the corpus's :class:~varagity.api.corpus_gate.CorpusGate and holds it for the whole run; a second export or restore — or an ingest, which claims the same gate — is refused with :class:~varagity.api.corpus_gate.CorpusBusy, the route's 409 corpus_busy. There is no separate single-flight flag: the gate is the one-run-at-a-time rule.
• A tracked flow. The run executes the rag-export, rag-restore, graph-export or graph-restore Prefect flow on a daemon thread, in-process, exactly as the CLI and the ingest runner run theirs; stage progress rides an on_progress callback and the varagity.backup log records are relayed as log events. Each corpus has its own runner (and gate), so a graph export and a RAG restore run side by side.
• Replay + live fan-out. Events accumulate per run; a subscriber gets the full backlog first, then live events until the terminal status frame — connecting mid-run or after completion renders the same picture.
• Each run relays its own lines. Both corpora's runs log to the one varagity.backup namespace and can run side by side, so a run's relay forwards only the records its own worker logs (a context variable marks them), and the namespace's INFO pin is shared by count — the first run to finish can't silence the other's feed.
• A refused bundle is remembered. A restore verifies every member as it loads it; when that finds the bundle unsound or damaged, the run records it in :class:DamagedBundles, so the list shows it as such and the next restore of it is refused up front.

The terminal transition is one critical section: the run turns terminal, its last frame joins the backlog, live subscribers are detached, and the gate is released together — so a subscriber can never see a finished run without its final frame, and a client reacting to that frame can always start the next run.

BackupOperation = Literal['export', 'restore'] module-attribute

What a backup run can do.

BackupStage = ExportStage | RestoreStage | GraphExportStage | GraphRestoreStage module-attribute

Every stage a backup run's progress frames can name.

ExportResult = RagExport | GraphExportReport module-attribute

What an export flow returns — either corpus's published bundle and its timing.

Feed = tuple[str, BaseModel] module-attribute

One feed item: (sse_event_name, payload_model).

RestoreResult = RagRestore | GraphRestore module-attribute

What a restore flow returns — what either corpus's restore wrote, and its stale flag.

BackupRunner

Owner of one corpus's (single) background backup run and its event feed.

Attributes:

Name	Type	Description
corpus	Corpus	The corpus this runner backs up.

Source code in varagity/api/backup_runner.py

class BackupRunner:
    """Owner of one corpus's (single) background backup run and its event feed.

    Attributes:
        corpus: The corpus this runner backs up.
    """

    def __init__(
        self,
        corpus: Corpus,
        *,
        gate: CorpusGate | None = None,
        export_flow: Callable[..., ExportResult] | None = None,
        restore_flow: Callable[..., RestoreResult] | None = None,
        ingest_settings: Callable[[], dict[str, str]] | None = None,
        damaged: DamagedBundles | None = None,
    ) -> None:
        """Wire the runner's collaborators (all injectable for tests).

        Args:
            corpus: The corpus to back up.
            gate: The corpus's write gate; the process-wide one when omitted.
            export_flow: The export callable — ``rag_export_flow``-compatible
                for ``rag``, ``graph_export_flow``-compatible for ``graph``;
                the corpus's own when omitted (injectable so tests run
                without stores, an engine, or Prefect).
            restore_flow: The restore callable — ``rag_restore_flow``-
                compatible for ``rag``, ``graph_restore_flow``-compatible for
                ``graph``; the corpus's own when omitted.
            ingest_settings: Provider of the reingest-affecting settings —
                what a RAG export's bundle records and what a RAG restore's
                stale flag is judged against; :func:`current_ingest_settings`
                by default (never called for the graph, which has none).
            damaged: Where a restore records a bundle it refused for its
                contents; the process-wide memory the backup routes read
                when omitted.

        Raises:
            KeyError: If no export or restore flow exists for ``corpus`` and
                none was injected.
        """
        self.corpus: Corpus = corpus
        self._gate = gate if gate is not None else get_corpus_gates()[corpus]
        self._export_flow = export_flow if export_flow is not None else _EXPORT_FLOWS[corpus]
        self._restore_flow = restore_flow if restore_flow is not None else _RESTORE_FLOWS[corpus]
        self._ingest_settings = (
            ingest_settings if ingest_settings is not None else current_ingest_settings
        )
        self._damaged = damaged if damaged is not None else get_damaged_bundles()
        self._lock = threading.Lock()
        self._run: _RunState | None = None
        self._events: list[Feed] = []
        self._subscribers: list[tuple[asyncio.AbstractEventLoop, asyncio.Queue[Feed | None]]] = []

    # ── run control ────────────────────────────────────────────────────

    def start(self, operation: BackupOperation, bundle: str | None = None) -> BackupRunOut:
        """Start a background backup run.

        Args:
            operation: What to run (``export`` | ``restore``).
            bundle: The bundle a restore restores — a file name in the
                corpus's exports folder (the route has already resolved and
                verified it). Not taken by an export.

        Returns:
            The new run's handle/state snapshot (state ``running``).

        Raises:
            CorpusBusy: If any operation already holds the corpus — a second
                export or restore included.
            ValueError: If a restore names no bundle, or an export names one.
        """
        if (operation == "restore") != (bundle is not None):
            raise ValueError("a restore takes the bundle to restore; an export takes none")
        ingest_settings = self._ingest_settings() if self.corpus == "rag" else None
        with self._lock:
            self._gate.claim(operation)
            run = _RunState(
                run_id=uuid4().hex[:12],
                corpus=self.corpus,
                operation=operation,
                ingest_settings=ingest_settings,
                bundle=bundle,
            )
            started = run.to_out()
            self._run = run
            # The first frame joins the backlog in the same critical section
            # that makes the run current: a subscriber can never find a live
            # run with an empty feed.
            self._events = [(EVENT_STATUS, BackupStatusEvent(run=started))]
        thread = threading.Thread(
            target=self._execute,
            args=(run,),
            name=f"backup-{self.corpus}-{run.run_id}",
            daemon=True,
        )
        try:
            thread.start()
        except RuntimeError as error:  # no thread to run it: give the corpus back
            self._finish(run, error=f"{type(error).__name__}: {error}")
            raise
        return started

    def snapshot(self) -> BackupRunOut | None:
        """Report the current (or last) run without subscribing.

        Returns:
            The run snapshot, or ``None`` when nothing ever ran.
        """
        with self._lock:
            return self._run.to_out() if self._run is not None else None

    @property
    def running(self) -> bool:
        """Whether a run is in flight.

        Returns:
            ``True`` between a start and its terminal frame.
        """
        with self._lock:
            return self._run is not None and self._run.state == "running"

    def ensure_free(self) -> None:
        """Refuse at once if any operation holds the corpus (without claiming it).

        The restore route's first check: a busy corpus answers ``409``
        before the bundle is even opened.

        Raises:
            CorpusBusy: If the corpus is held.
        """
        self._gate.ensure_free()

    # ── the event feed ─────────────────────────────────────────────────

    async def subscribe(self) -> AsyncIterator[Feed]:
        """Stream this runner's events: full backlog first, then live.

        The stream ends after the terminal ``status`` frame (or immediately
        after an idle/terminal snapshot when nothing is running), so a
        client connecting at any point renders the same picture.

        Yields:
            ``(event_name, payload)`` feed items in emission order.
        """
        loop = asyncio.get_running_loop()
        queue: asyncio.Queue[Feed | None] = asyncio.Queue()
        with self._lock:
            backlog = list(self._events)
            live = self._run is not None and self._run.state == "running"
            if live:
                self._subscribers.append((loop, queue))
        if not backlog:
            # Nothing ever ran for this corpus in this API process: say so.
            yield (EVENT_STATUS, BackupStatusEvent(run=None))
            return
        try:
            for item in backlog:
                yield item
            if not live:
                return
            while (live_item := await queue.get()) is not None:
                yield live_item
        finally:
            with self._lock:
                self._subscribers = [entry for entry in self._subscribers if entry[1] is not queue]

    def _emit(self, event: str, payload: BaseModel) -> None:
        """Append one event and fan it out to live subscribers (any thread).

        Args:
            event: The SSE event name.
            payload: Its typed payload.
        """
        item: Feed = (event, payload)
        with self._lock:
            self._events.append(item)
            subscribers = list(self._subscribers)
        _deliver(subscribers, item)

    def _report_progress(self, stage: BackupStage, current: int | None, total: int | None) -> None:
        """Turn the flow's progress callback into ``progress`` frames.

        Args:
            stage: The export or restore stage.
            current: Rows (a graph export's ``pack``: bytes) done so far in a
                counted stage.
            total: The stage's row (byte) count.
        """
        self._emit(EVENT_PROGRESS, BackupProgressEvent(stage=stage, current=current, total=total))

    # ── the worker thread ──────────────────────────────────────────────

    def _execute(self, run: _RunState) -> None:
        """Run the export or restore flow; publish the terminal state.

        Args:
            run: The run this thread owns.
        """
        relay = _RelayHandler(self._emit, run.run_id)
        backup_logger = logging.getLogger(_RELAY_LOGGER)
        # This worker's records are this run's (the relay matches on it).
        token = _CURRENT_RUN.set(run.run_id)
        # The backup's own lines log at INFO; the feed must carry them
        # whatever the process LOG_LEVEL (root handlers still apply their
        # own level filters, so the console stays as quiet as configured).
        _relay_level.pin()
        backup_logger.addHandler(relay)
        result: ExportResult | RestoreResult | None = None
        failure: str | None = None
        try:
            result = self._restore(run) if run.operation == "restore" else self._export(run)
        except Exception as error:
            logger.exception("backup run %s (%s %s) failed", run.run_id, self.corpus, run.operation)
            failure = f"{type(error).__name__}: {error}"
            if run.operation == "restore" and isinstance(error, BundleInvalid):
                self._remember_refused(run, error)
        finally:
            backup_logger.removeHandler(relay)
            _relay_level.unpin()
            _CURRENT_RUN.reset(token)
            self._finish(run, result=result, error=failure)

    def _remember_refused(self, run: _RunState, error: BundleInvalid) -> None:
        """Record a bundle the restore refused for its contents.

        What the run's verifying read found — a member whose bytes don't
        match their sha256 (or the zip's CRC), a line no row can be made
        from — is a property of the file, so the list shows it from now on
        and the route refuses it before loading it again. Nothing was
        restored: the RAG transaction rolled back, the graph's staged copy
        was never swapped in.

        Args:
            run: The failed restore run.
            error: What the restore found wrong with the bundle.
        """
        assert run.bundle is not None  # start() requires one for a restore
        self._damaged.remember(
            self.corpus, run.bundle, exports_dir(self.corpus) / run.bundle, str(error)
        )

    def _export(self, run: _RunState) -> ExportResult:
        """Run the corpus's export flow into its exports folder.

        A RAG export records the reingest-affecting settings captured at
        start — read here, in the API layer, so the backup package never
        imports it. A graph export needs nothing from this layer: its flow
        reads the graph-stale flag and finds the process's graph service
        itself.

        Args:
            run: The export run this thread owns.

        Returns:
            The flow's published bundle and its timing.
        """
        dest = exports_dir(self.corpus)
        if self.corpus == "rag":
            assert run.ingest_settings is not None  # start() captures them for every RAG run
            return self._export_flow(
                dest, ingest_settings=run.ingest_settings, on_progress=self._report_progress
            )
        return self._export_flow(dest, on_progress=self._report_progress)

    def _restore(self, run: _RunState) -> RestoreResult:
        """Run the corpus's restore flow over the run's bundle.

        A RAG restore gets the *provider* of the reingest-affecting settings,
        not a snapshot: its stale flag is judged against the settings current
        when it is written. A graph restore gets the run's id instead, which
        names the folders it parks beside the workdir while it swaps.

        Args:
            run: The restore run this thread owns.

        Returns:
            What the flow restored, with the stale flag it left.
        """
        assert run.bundle is not None  # start() requires one for a restore
        bundle = exports_dir(self.corpus) / run.bundle
        if self.corpus == "rag":
            return self._restore_flow(
                bundle, ingest_settings=self._ingest_settings, on_progress=self._report_progress
            )
        return self._restore_flow(bundle, run_id=run.run_id, on_progress=self._report_progress)

    def _finish(
        self,
        run: _RunState,
        *,
        result: ExportResult | RestoreResult | None = None,
        error: str | None = None,
    ) -> None:
        """Make the run terminal, publish its last frame, and free the corpus.

        One critical section (see the module docstring): terminal state,
        terminal frame, subscriber detach, and gate release land together.

        Args:
            run: The run ending.
            result: The flow's outcome, when it completed.
            error: The failure, when it did not.
        """
        with self._lock:
            run.finished_at = datetime.now(UTC)
            if isinstance(result, RagExport | GraphExportReport) and error is None:
                run.state = "completed"
                run.bundle = result.bundle.name
                run.saved_to = display_path(self.corpus, result.bundle.name)
                run.summary = BackupSummaryOut(
                    counts=result.counts,
                    size_bytes=result.bundle.size_bytes,
                    wall_clock_s=round(result.wall_clock_s, 3),
                )
            elif isinstance(result, RagRestore | GraphRestore) and error is None:
                run.state = "completed"
                run.summary = BackupSummaryOut(
                    counts=result.counts,
                    size_bytes=result.size_bytes,
                    wall_clock_s=round(result.wall_clock_s, 3),
                    stale=result.stale,
                )
            else:
                run.state = "failed"
                run.error = error or (
                    "the export returned no bundle"
                    if run.operation == "export"
                    else "the restore returned no result"
                )
            item: Feed = (EVENT_STATUS, BackupStatusEvent(run=run.to_out()))
            self._events.append(item)
            subscribers, self._subscribers = self._subscribers, []
            self._gate.release()
        _deliver(subscribers, item)
        _deliver(subscribers, None)

running property

Whether a run is in flight.

Returns:

Type	Description
bool	True between a start and its terminal frame.

__init__(corpus, *, gate=None, export_flow=None, restore_flow=None, ingest_settings=None, damaged=None)

Wire the runner's collaborators (all injectable for tests).

Parameters:

Name	Type	Description	Default
corpus	Corpus	The corpus to back up.	required
gate	CorpusGate | None	The corpus's write gate; the process-wide one when omitted.	None
export_flow	Callable[..., ExportResult] | None	The export callable — rag_export_flow-compatible for rag, graph_export_flow-compatible for graph; the corpus's own when omitted (injectable so tests run without stores, an engine, or Prefect).	None
restore_flow	Callable[..., RestoreResult] | None	The restore callable — rag_restore_flow- compatible for rag, graph_restore_flow-compatible for graph; the corpus's own when omitted.	None
ingest_settings	Callable[[], dict[str, str]] | None	Provider of the reingest-affecting settings — what a RAG export's bundle records and what a RAG restore's stale flag is judged against; :func:current_ingest_settings by default (never called for the graph, which has none).	None
damaged	DamagedBundles | None	Where a restore records a bundle it refused for its contents; the process-wide memory the backup routes read when omitted.	None

Raises:

Type	Description
KeyError	If no export or restore flow exists for corpus and none was injected.

Source code in varagity/api/backup_runner.py

def __init__(
    self,
    corpus: Corpus,
    *,
    gate: CorpusGate | None = None,
    export_flow: Callable[..., ExportResult] | None = None,
    restore_flow: Callable[..., RestoreResult] | None = None,
    ingest_settings: Callable[[], dict[str, str]] | None = None,
    damaged: DamagedBundles | None = None,
) -> None:
    """Wire the runner's collaborators (all injectable for tests).

    Args:
        corpus: The corpus to back up.
        gate: The corpus's write gate; the process-wide one when omitted.
        export_flow: The export callable — ``rag_export_flow``-compatible
            for ``rag``, ``graph_export_flow``-compatible for ``graph``;
            the corpus's own when omitted (injectable so tests run
            without stores, an engine, or Prefect).
        restore_flow: The restore callable — ``rag_restore_flow``-
            compatible for ``rag``, ``graph_restore_flow``-compatible for
            ``graph``; the corpus's own when omitted.
        ingest_settings: Provider of the reingest-affecting settings —
            what a RAG export's bundle records and what a RAG restore's
            stale flag is judged against; :func:`current_ingest_settings`
            by default (never called for the graph, which has none).
        damaged: Where a restore records a bundle it refused for its
            contents; the process-wide memory the backup routes read
            when omitted.

    Raises:
        KeyError: If no export or restore flow exists for ``corpus`` and
            none was injected.
    """
    self.corpus: Corpus = corpus
    self._gate = gate if gate is not None else get_corpus_gates()[corpus]
    self._export_flow = export_flow if export_flow is not None else _EXPORT_FLOWS[corpus]
    self._restore_flow = restore_flow if restore_flow is not None else _RESTORE_FLOWS[corpus]
    self._ingest_settings = (
        ingest_settings if ingest_settings is not None else current_ingest_settings
    )
    self._damaged = damaged if damaged is not None else get_damaged_bundles()
    self._lock = threading.Lock()
    self._run: _RunState | None = None
    self._events: list[Feed] = []
    self._subscribers: list[tuple[asyncio.AbstractEventLoop, asyncio.Queue[Feed | None]]] = []

ensure_free()

Refuse at once if any operation holds the corpus (without claiming it).

The restore route's first check: a busy corpus answers 409 before the bundle is even opened.

Raises:

Type	Description
CorpusBusy	If the corpus is held.

Source code in varagity/api/backup_runner.py

def ensure_free(self) -> None:
    """Refuse at once if any operation holds the corpus (without claiming it).

    The restore route's first check: a busy corpus answers ``409``
    before the bundle is even opened.

    Raises:
        CorpusBusy: If the corpus is held.
    """
    self._gate.ensure_free()

snapshot()

Report the current (or last) run without subscribing.

Returns:

Type	Description
BackupRunOut | None	The run snapshot, or None when nothing ever ran.

Source code in varagity/api/backup_runner.py

def snapshot(self) -> BackupRunOut | None:
    """Report the current (or last) run without subscribing.

    Returns:
        The run snapshot, or ``None`` when nothing ever ran.
    """
    with self._lock:
        return self._run.to_out() if self._run is not None else None

start(operation, bundle=None)

Start a background backup run.

Parameters:

Name	Type	Description	Default
operation	BackupOperation	What to run (export | restore).	required
bundle	str | None	The bundle a restore restores — a file name in the corpus's exports folder (the route has already resolved and verified it). Not taken by an export.	None

Returns:

Type	Description
BackupRunOut	The new run's handle/state snapshot (state running).

Raises:

Type	Description
CorpusBusy	If any operation already holds the corpus — a second export or restore included.
ValueError	If a restore names no bundle, or an export names one.

Source code in varagity/api/backup_runner.py

def start(self, operation: BackupOperation, bundle: str | None = None) -> BackupRunOut:
    """Start a background backup run.

    Args:
        operation: What to run (``export`` | ``restore``).
        bundle: The bundle a restore restores — a file name in the
            corpus's exports folder (the route has already resolved and
            verified it). Not taken by an export.

    Returns:
        The new run's handle/state snapshot (state ``running``).

    Raises:
        CorpusBusy: If any operation already holds the corpus — a second
            export or restore included.
        ValueError: If a restore names no bundle, or an export names one.
    """
    if (operation == "restore") != (bundle is not None):
        raise ValueError("a restore takes the bundle to restore; an export takes none")
    ingest_settings = self._ingest_settings() if self.corpus == "rag" else None
    with self._lock:
        self._gate.claim(operation)
        run = _RunState(
            run_id=uuid4().hex[:12],
            corpus=self.corpus,
            operation=operation,
            ingest_settings=ingest_settings,
            bundle=bundle,
        )
        started = run.to_out()
        self._run = run
        # The first frame joins the backlog in the same critical section
        # that makes the run current: a subscriber can never find a live
        # run with an empty feed.
        self._events = [(EVENT_STATUS, BackupStatusEvent(run=started))]
    thread = threading.Thread(
        target=self._execute,
        args=(run,),
        name=f"backup-{self.corpus}-{run.run_id}",
        daemon=True,
    )
    try:
        thread.start()
    except RuntimeError as error:  # no thread to run it: give the corpus back
        self._finish(run, error=f"{type(error).__name__}: {error}")
        raise
    return started

subscribe() async

Stream this runner's events: full backlog first, then live.

The stream ends after the terminal status frame (or immediately after an idle/terminal snapshot when nothing is running), so a client connecting at any point renders the same picture.

Yields:

Type	Description
AsyncIterator[Feed]	(event_name, payload) feed items in emission order.

Source code in varagity/api/backup_runner.py

async def subscribe(self) -> AsyncIterator[Feed]:
    """Stream this runner's events: full backlog first, then live.

    The stream ends after the terminal ``status`` frame (or immediately
    after an idle/terminal snapshot when nothing is running), so a
    client connecting at any point renders the same picture.

    Yields:
        ``(event_name, payload)`` feed items in emission order.
    """
    loop = asyncio.get_running_loop()
    queue: asyncio.Queue[Feed | None] = asyncio.Queue()
    with self._lock:
        backlog = list(self._events)
        live = self._run is not None and self._run.state == "running"
        if live:
            self._subscribers.append((loop, queue))
    if not backlog:
        # Nothing ever ran for this corpus in this API process: say so.
        yield (EVENT_STATUS, BackupStatusEvent(run=None))
        return
    try:
        for item in backlog:
            yield item
        if not live:
            return
        while (live_item := await queue.get()) is not None:
            yield live_item
    finally:
        with self._lock:
            self._subscribers = [entry for entry in self._subscribers if entry[1] is not queue]

DamagedBundles

Bundles a restore refused for their contents, remembered in-process.

A restore verifies every member as it loads it — the only check that sees a bundle damaged inside an intact zip, or a line no row can be made from — and the list's verdict is deliberately cheap, so what a restore found is remembered here: the list then shows the bundle as not restorable, with that reason, and the restore route refuses it up front instead of loading it again.

Keyed by corpus, name, size and mtime: a bundle whose bytes were replaced (copied over, re-exported under the same name) is a new file and is judged afresh. In-process only — the next API start forgets, and the next restore of that bundle fails (and is remembered) again.

Source code in varagity/api/backup_runner.py

class DamagedBundles:
    """Bundles a restore refused for their contents, remembered in-process.

    A restore verifies every member as it loads it — the only check that
    sees a bundle damaged *inside* an intact zip, or a line no row can be
    made from — and the list's verdict is deliberately cheap, so what a
    restore found is remembered here: the list then shows the bundle as
    not restorable, with that reason, and the restore route refuses it up
    front instead of loading it again.

    Keyed by corpus, name, size and mtime: a bundle whose bytes were
    replaced (copied over, re-exported under the same name) is a new file
    and is judged afresh. In-process only — the next API start forgets, and
    the next restore of that bundle fails (and is remembered) again.
    """

    def __init__(self) -> None:
        """Start with nothing remembered."""
        self._lock = threading.Lock()
        self._reasons: dict[tuple[str, str, int, int], str] = {}

    def remember(self, corpus: str, name: str, path: Path, reason: str) -> None:
        """Remember that a restore refused this bundle.

        Args:
            corpus: The bundle's corpus.
            name: Its file name in the exports folder.
            path: The file (stat'ed for its size and mtime).
            reason: What the restore found.
        """
        try:
            info = path.stat()
        except OSError:
            return  # gone already: nothing left to mislist
        with self._lock:
            self._reasons[(corpus, name, info.st_size, info.st_mtime_ns)] = reason

    def reason(self, corpus: str, listing: BundleListing) -> str | None:
        """Report why a listed bundle was refused, if it was.

        Args:
            corpus: The listing's corpus.
            listing: The bundle as the folder lists it now.

        Returns:
            The remembered reason, or ``None`` when this exact file (name,
            size, mtime) was never refused.
        """
        with self._lock:
            return self._reasons.get((corpus, listing.name, listing.size_bytes, listing.mtime_ns))

    def reason_for(self, corpus: str, name: str, path: Path) -> str | None:
        """Report why a bundle file was refused, if it was (the restore route's check).

        Args:
            corpus: The bundle's corpus.
            name: Its file name in the exports folder.
            path: The file (stat'ed for its size and mtime).

        Returns:
            The remembered reason, or ``None`` when this exact file was never
            refused (or can't be stat'ed).
        """
        try:
            info = path.stat()
        except OSError:
            return None
        with self._lock:
            return self._reasons.get((corpus, name, info.st_size, info.st_mtime_ns))

__init__()

Start with nothing remembered.

Source code in varagity/api/backup_runner.py

def __init__(self) -> None:
    """Start with nothing remembered."""
    self._lock = threading.Lock()
    self._reasons: dict[tuple[str, str, int, int], str] = {}

reason(corpus, listing)

Report why a listed bundle was refused, if it was.

Parameters:

Name	Type	Description	Default
corpus	str	The listing's corpus.	required
listing	BundleListing	The bundle as the folder lists it now.	required

Returns:

Type	Description
str | None	The remembered reason, or None when this exact file (name,
str | None	size, mtime) was never refused.

Source code in varagity/api/backup_runner.py

def reason(self, corpus: str, listing: BundleListing) -> str | None:
    """Report why a listed bundle was refused, if it was.

    Args:
        corpus: The listing's corpus.
        listing: The bundle as the folder lists it now.

    Returns:
        The remembered reason, or ``None`` when this exact file (name,
        size, mtime) was never refused.
    """
    with self._lock:
        return self._reasons.get((corpus, listing.name, listing.size_bytes, listing.mtime_ns))

reason_for(corpus, name, path)

Report why a bundle file was refused, if it was (the restore route's check).

Parameters:

Name	Type	Description	Default
corpus	str	The bundle's corpus.	required
name	str	Its file name in the exports folder.	required
path	Path	The file (stat'ed for its size and mtime).	required

Returns:

Type	Description
str | None	The remembered reason, or None when this exact file was never
str | None	refused (or can't be stat'ed).

Source code in varagity/api/backup_runner.py

def reason_for(self, corpus: str, name: str, path: Path) -> str | None:
    """Report why a bundle file was refused, if it was (the restore route's check).

    Args:
        corpus: The bundle's corpus.
        name: Its file name in the exports folder.
        path: The file (stat'ed for its size and mtime).

    Returns:
        The remembered reason, or ``None`` when this exact file was never
        refused (or can't be stat'ed).
    """
    try:
        info = path.stat()
    except OSError:
        return None
    with self._lock:
        return self._reasons.get((corpus, name, info.st_size, info.st_mtime_ns))

remember(corpus, name, path, reason)

Remember that a restore refused this bundle.

Parameters:

Name	Type	Description	Default
corpus	str	The bundle's corpus.	required
name	str	Its file name in the exports folder.	required
path	Path	The file (stat'ed for its size and mtime).	required
reason	str	What the restore found.	required

Source code in varagity/api/backup_runner.py

def remember(self, corpus: str, name: str, path: Path, reason: str) -> None:
    """Remember that a restore refused this bundle.

    Args:
        corpus: The bundle's corpus.
        name: Its file name in the exports folder.
        path: The file (stat'ed for its size and mtime).
        reason: What the restore found.
    """
    try:
        info = path.stat()
    except OSError:
        return  # gone already: nothing left to mislist
    with self._lock:
        self._reasons[(corpus, name, info.st_size, info.st_mtime_ns)] = reason

current_ingest_settings()

Snapshot the reingest-affecting settings, in env-string form.

What a RAG bundle records as ingest_settings: the values whose change marks the corpus stale (CHUNKING_STRATEGY, CHUNK_SIZE, CHUNK_OVERLAP, CONTEXTUALIZE, OCR_ENGINE), rendered exactly as app_settings stores an override — so a restore compares like with like. Read here, in the API layer, so the backup package never imports it.

Returns:

Type	Description
dict[str, str]	Setting name → env-string value, sorted by name.

Source code in varagity/api/backup_runner.py

def current_ingest_settings() -> dict[str, str]:
    """Snapshot the reingest-affecting settings, in env-string form.

    What a RAG bundle records as ``ingest_settings``: the values whose
    change marks the corpus stale (``CHUNKING_STRATEGY``, ``CHUNK_SIZE``,
    ``CHUNK_OVERLAP``, ``CONTEXTUALIZE``, ``OCR_ENGINE``), rendered exactly
    as ``app_settings`` stores an override — so a restore compares like with
    like. Read here, in the API layer, so the backup package never imports
    it.

    Returns:
        Setting name → env-string value, sorted by name.
    """
    settings = get_settings()
    return {name: to_env_value(getattr(settings, name)) for name in sorted(REINGEST_AFFECTING)}

get_backup_runners()

Provide the process-wide backup runners, keyed by corpus (dependency, override seam).

Returns:

Type	Description
dict[str, BackupRunner]	The lazily created runners, one per corpus (rag, graph) —
dict[str, BackupRunner]	each on its corpus's process-wide gate, so a backup and that
dict[str, BackupRunner]	corpus's ingest or build always exclude each other. Constructing one
dict[str, BackupRunner]	resolves nothing: the graph's flow finds the graph service only when
dict[str, BackupRunner]	an export runs.

Source code in varagity/api/backup_runner.py

def get_backup_runners() -> dict[str, BackupRunner]:
    """Provide the process-wide backup runners, keyed by corpus (dependency, override seam).

    Returns:
        The lazily created runners, one per corpus (``rag``, ``graph``) —
        each on its corpus's process-wide gate, so a backup and that
        corpus's ingest or build always exclude each other. Constructing one
        resolves nothing: the graph's flow finds the graph service only when
        an export runs.
    """
    global _default_runners
    with _default_runners_lock:
        if _default_runners is None:
            _default_runners = {"rag": BackupRunner("rag"), "graph": BackupRunner("graph")}
        return _default_runners

get_damaged_bundles()

Provide the process's refused-bundle memory (FastAPI dependency, override seam).

Returns:

Type	Description
DamagedBundles	The one :class:DamagedBundles for the life of the process — the
DamagedBundles	runners record into it, the backup routes read it.

Source code in varagity/api/backup_runner.py

def get_damaged_bundles() -> DamagedBundles:
    """Provide the process's refused-bundle memory (FastAPI dependency, override seam).

    Returns:
        The one :class:`DamagedBundles` for the life of the process — the
        runners record into it, the backup routes read it.
    """
    return _damaged_bundles