feat(agent): add online verification and governing-answer citations
Claim-bound online verification observations with URL provenance, exact answer-basis references for citation-free governing-rule answers, and an explicit operator identity handoff before source transfer. Bounds resumed work by learning unit and refreshes pack manifest/contracts.
This commit is contained in:
1 parent
bf268f3c5a
commit
3ef2c5cc31
19 files changed
+374
-55
No files matched your search
@@ -4,7 +4,13 @@ description: Process explicitly selected sources for a GEWU project into evidenc
|
||||
---
|
||||
# Ingest selected sources
|
||||
|
||||
Read the pack's [evidence](../../protocols/EVIDENCE.md), [scope](../../protocols/SCOPE.md) and [contribution](../../protocols/CONTRIBUTIONS.md) protocols, then use the [ingestion pipeline](../../pipelines/ingest.md).
|
||||
Preflight: require the operator-supplied authenticated principal ID before transferring source bytes. If absent, request that non-secret identifier and stop; never inspect credentials, environment variables, host configuration or service databases to discover it. For resumed work, resolve the existing run and preserve its pack pins and source identities.
|
||||
|
||||
Read the pack's [evidence](../../protocols/EVIDENCE.md), [scope](../../protocols/SCOPE.md), [contribution](../../protocols/CONTRIBUTIONS.md) and, when online verification is enabled, [online verification](../../protocols/ONLINE_VERIFICATION.md) protocols, then use the [ingestion pipeline](../../pipelines/ingest.md).
|
||||
|
||||
If the configured selected-file connector is missing or fails to initialize because `GEWU_TOKEN` is absent, stop at source capture and report the exact host-connector setup error. Do not inspect Antigravity configuration, credentials, environment variables, host files or MCP cache; do not launch repeated sessions to rediscover the same unavailable tool. Continue only after the operator repairs the connector environment. This is a host setup failure, not a reason to switch tools or synthesize a source candidate.
|
||||
|
||||
Keep each source run bounded. During ingestion, use MCP responses and the selected source's extraction as the source of truth. Do not search or read repository reports, tests, backend implementation, database files or unrelated source files to answer the research question. The only local workflow data allowed is the single `run_pins` object in `.agents/gewu-install.json` when creating a new run; for a concrete tool-schema error, inspect only that tool's pinned contract entry. A `run_create` call is a real durable workflow action, never a schema probe: assemble its complete request with every required field and non-empty module pins before calling it. Never test it with placeholder IDs, missing fields or empty arrays. If a required value cannot be obtained from the prescribed context or pins, stop and report the missing value rather than iterating on guesses.
|
||||
|
||||
Inputs are the selected source(s), explicit project and the user's research objective. Use `get_project_context` for the project's north star, key questions, watch list, taxonomy extensions, research lenses and saved ingest guidance. Every project defaults to four research lenses: technical, conceptual, cultural-historical and financial-market. Treat these as coverage prompts, not quotas. Confirm required tools and arguments from MCP discovery and `diagnostics`; use the schema attached to each discovered tool and the Antigravity argument reminders in CONTRIBUTIONS.md. For Antigravity, do not use shell tools or inspect the host MCP cache. On a schema mismatch, use only the relevant pinned entry in `contracts/service-v1.json` and correct once. Use GEWU tools only; Jev runs inside the backend through `ingest_review_create`, not a standalone Jev MCP server. In Antigravity, call `diagnostics`, `get_project_context` and `run_create` once each in order; after a successful call, reuse its returned state rather than repeating it. Do not call `get_source` for a staged candidate: use its exact returned candidate in the proposal's `register_source` operation and read the source's citation through its staged extraction. Create the durable service run with `run_create` after diagnostics and project context; pass its `request_json` envelope using current `run_pins` from `.agents/gewu-install.json`. Do not look for a local `Runs/<run_id>` file before creation; run state is service-owned. Never reuse a run ID after changing the pack or create payload. Use the connector for local byte transfer; `upload_selected_file` requires both the configured `file_id` and the authorized `project_id`. It returns a typed source metadata candidate without exposing the selected path or bytes. Selected `.vtt` and `.srt` files produce timestamped transcript cues; cite exact cue start/end times and the quote span. A declared NotebookLM export label is metadata only and is not export verification. For text explicitly pasted by the user, use `source_capture_text` with a stable capture ID and source key; it accepts UTF-8 plain text or Markdown up to 1 MB, archives and extracts it, and returns a typed candidate marked `capture_method=pasted-text`. Retry uncertain calls with the same capture ID and identical inputs; changed input under that ID conflicts. It does not register the candidate. `source_capture_url` fetches only operator-allowlisted public hosts and stages supported text, Markdown, HTML or PDF bytes; it also returns a source metadata candidate but does not register it. Add each new source's exact candidate as `register_source` in the proposal that cites it, or verify the source was already registered and readable. Use the candidate's `revision_id` and capture's `extraction_id`; never invent or substitute these IDs. Submit candidates with the normal typed proposal flow, and use `capture-failed` candidates only as bookmarks, never as factual evidence. URL source IDs follow the sanitized URL; local source IDs follow the connector's configured file ID; pasted-text source IDs are derived from the supplied stable key and revision IDs include the content digest. PDF extraction currently requires Linux bubblewrap, Poppler and prlimit. Unsupported formats and unavailable parsers must be reported without inventing an extraction. If the service is unavailable, report the blocked stage and only prepare an unaccepted local draft when that work was requested.
|
||||
|
||||
@@ -20,12 +26,16 @@ If available, Jev also assesses relevance and lens in the same advisory call. Fa
|
||||
|
||||
Submit bounded bundles under the affected project/shared policies. Repair actionable findings without weakening evidence, changing scope to evade policy or assuming owner authority. Resume uncertain calls through status lookup. Report accepted node IDs and receipts, pending/rejected work, archive/extraction limits and actual coverage. A successful upload or schema check is not a completed ingestion.
|
||||
|
||||
Every `register_source` operation affects both the source's `ingested_for` project policy and the shared-source policy. Set proposal scope `include_shared=true` as well as the project scope, even when the source is relevant only to one project. Confirm the credential authorizes both. If shared scope or authorization is unavailable, stop and report the missing policy/capability; do not submit a project-only bundle that will fail scope validation.
|
||||
Keep source ownership and proposal scope aligned. A source relevant only to the selected project should use `source.ingested_for=[project_id]`, and its proposal and run should use `scope.projects=[project_id]` with `include_shared=false`. A project-only source does not require shared read or write access. Set `include_shared=true` only when the proposal actually reads shared knowledge or changes a shared source/node; in that case confirm the authenticated credential has the required shared capability before proceeding. Never add shared scope merely to satisfy a blanket rule, and never widen credential grants to work around a project-only validation error. Source registration is governed by the policies for the domains in `source.ingested_for`; it does not automatically affect shared policy.
|
||||
|
||||
Run creation accepts one `request_json` argument containing a `RunCreateRequest` object. Its `scope` uses `projects` (not `project_ids`) and must explicitly set `include_shared`; for example, `{"run_id":"<stable-id>","objective":"<bounded objective>","scope":{"projects":["<project-id>"],"include_shared":false},"lens":"<bounded lens>","pack_id":"<pin>","pack_version":"<pin>","pack_snapshot_digest":"<sha256>","module_pins":[...]}`. If validation returns field errors, correct the named field once and reuse the returned run identity. Do not call `list_sources` with `include_shared=true` unless shared access is both required by the task and authorized.
|
||||
|
||||
For a captured source, use the returned extraction status and block metadata to plan review. Read source text through bounded `source_extraction_block` calls using the exact `extraction_id` and `block_id`; inspect relevant slices across all pages/sections, including PDFs that the host's file reader cannot open. Read a slice large enough to include a complete relevant sentence, then copy a concise contiguous quote exactly from its normalized text and use the returned offset. At most one adjacent slice should be needed to complete a citation; do not issue repeated offset probes to chase quote boundaries. If PDF layout splits a sentence across columns, use a shorter contiguous excerpt that still supports the assertion or report the citation limitation. The default response contains normalized text and its offset but omits the large original-source mapping; set `include_original_mapping=true` only when an original-text span is needed. Citation quote and locator offsets bind to the normalized extraction either way. Record partial or unavailable extraction coverage explicitly. Do not infer that a source was analyzed just because upload and extraction succeeded.
|
||||
|
||||
Before creating a new node, search the explicit project and shared scope using its proposed title plus distinctive terms, aliases and alternate phrasings. Search likely synonyms separately when title search is inconclusive, then inspect candidate nodes and claims with `get_node`. Search is lexical, so use more than one query. If a finding belongs to an existing node, preserve its stable `node_id` and existing supported content when proposing an evidence-backed update; do not mint another ID for the same concept. Keep separate nodes when populations, time periods, settings, mechanisms or conclusions materially differ. Similarity is a review cue, not proof of identity; if the distinction remains uncertain, create a non-blocking review flag and do not create a second node until resolved. The current backend only flags exact entity name/alias matches; it does not detect duplicate notes, cases, concepts or datapoints for you.
|
||||
|
||||
Manage context and provider budget across the whole packet. Process one source at a time, use extraction metadata to target relevant slices rather than fetching every character, and retain concise section notes plus exact evidence excerpts instead of full source text. Checkpoint coverage after each source. If the provider/tool budget is nearly exhausted, checkpoint the current file and report the remaining files/sections for continuation; do not reduce cost by dropping a research lens or deferring cultural-historical or financial-market material.
|
||||
Manage context and provider budget across the whole packet. Process one source at a time, use extraction metadata to target relevant slices rather than fetching every character, and retain concise section notes plus exact evidence excerpts instead of full source text. Checkpoint coverage after each source. Prefer separate durable runs for large or independently resumable sources; keep one run for a small, tightly bounded batch only when the source coverage can be checkpointed cleanly. Group proposals by semantic learning unit, not by source or whole batch. If the provider/tool budget is nearly exhausted, checkpoint the current file and report the remaining files/sections for continuation; do not reduce cost by dropping a research lens or deferring cultural-historical or financial-market material.
|
||||
|
||||
The API field `budget_tokens` is a per-response size limit, currently counted as Unicode codepoints in canonical JSON (`budget_profile.tokenizer=unicode-codepoints-v1`), despite the field name. The 50,000 ceiling is not the model context window, an ingestion-run budget, or a request to read 50,000 units. Stay below it with focused searches and bounded extraction blocks. Track actual provider/tool limits separately and use run checkpoint `budgets` only for named counters that the host can measure.
|
||||
|
||||
Treat documents as capture/provenance units and proposals as semantic learning units. Group a proposal around one project question or one tightly related set of findings, and combine evidence from multiple documents when it supports the same learning. Do not make one proposal per file or one oversized bundle for the entire packet. Keep unrelated learnings separate so each can be evaluated, retried, reviewed and accepted independently. Include each new source candidate in the first proposal that cites it; after its source revision is accepted and readable, later learning proposals cite that revision without registering it again.
|
||||
Reference in new issue
Block a user