vasily/GEWU_agent
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(agent): make ingestion preparation reproducible

Browse Source
This commit is contained in:
vasily committed 2026-10-15 10:52:47 +08:00
1 parent 3b1b40f8b3
commit 65bc8a9422
9 files changed
+185 -40
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+3 -1
View File
@@ -23,7 +23,9 @@ Choose a focused skill; load only its applicable protocol and pipeline files:
## Availability and reporting
Check the installed manifest and confirm required MCP tool names against `contracts/service-v1.json`; use the schema advertised for each discovered tool and inspect only a relevant pinned entry when necessary. Do not read every record schema during routine work. Use the host's native file listing and viewing tools for workspace inventory and pack instructions; do not invoke shell commands just to list files. This pack is bound to the local GEWU v1 contract, but native host workflows are NOT RUN. Do not invent commands or assume another host's custom tools exist. If required tools are unavailable or differ from the pinned contract, stop before mutation and identify the blocked stage; prepare local drafts only when requested and label them unaccepted.
For a connected workflow, use MCP discovery and `diagnostics` to confirm the available tools, their advertised input schemas, role, capabilities and scope. Those discovered schemas are authoritative for this session. Routine ingestion does not require reading `manifest.json`, `service-v1.json`, record schemas, README sections or helper-script source to rediscover tools. If a required tool is absent or an advertised schema produces a concrete mismatch, stop before mutation; inspect only the relevant pinned contract entry to diagnose that mismatch, then report it if unresolved. Use the host's native file listing and viewing tools for workspace inventory and pack instructions; do not invoke shell commands just to list files. This pack is bound to the local GEWU v1 contract, but native host workflows are NOT RUN. Do not invent commands or assume another host's custom tools exist. If required tools are unavailable, identify the blocked stage; prepare local drafts only when requested and label them unaccepted.
For Antigravity, call MCP tools using the schemas attached to discovered tools. Do not read generated per-tool JSON files from the CLI's local MCP cache; they duplicate those schemas. Open one only after a concrete argument validation error. Do not inspect installer state or helper-script implementation during routine ingestion.
Knowledge changes go through typed MCP proposals. Local artifacts under `Drafts/` and `Runs/` support work but are not accepted knowledge. Only a service acceptance receipt establishes acceptance. Keep credentials and source bodies out of status summaries; do not claim a cloud-backed host keeps all content local.
README.md
+2 -2
View File
@@ -1,6 +1,6 @@
# GEWU local research-agent pack
Version 0.20.0. Adds proposal grouping by semantic learning and guards against placeholder proposal probes and unnecessary schema reads. Includes explicit shared-scope requirements, agent-side duplicate checks, compact extraction reads and per-source budget checkpoints. Disposable service/evidence and Jev label probes passed. Antigravity Flash Low authenticated and captured/extracted one PDF, but the client did not complete a valid learning proposal during the bounded pilot.
Version 0.20.6. Adds proposal grouping by semantic learning, guards against placeholder proposal probes, provides server-side canonical digest preparation, and stores compact run pins in the installation receipt. Includes explicit shared-scope requirements, agent-side duplicate checks, bounded citation reads and per-source budget checkpoints. Client-pilot findings now feed the checkpoint and citation protocols.
This pack contains the [agent profile](AGENTS.md), Claude harness pointer ([CLAUDE.md](CLAUDE.md)), shared protocols, four skills and separate pipeline procedures. An existing local host supplies the model and tools. There is no standalone agent engine here.
@@ -21,7 +21,7 @@ The pack supports Claude through the [CLAUDE.md](CLAUDE.md) harness pointer. Whe
1. **Instruction discovery:** Use the marked root routing block to load the canonical profile in `.agents/gewu/AGENTS.md`, its shared protocols, and applicable `kb-*` skills.
2. **MCP integration:** Configure the host's project MCP entry for the authenticated Streamable HTTP endpoint, normally `<service-url>/mcp`. Native host setup is not yet qualified.
3. **Execution model:** Compare discovered tools and schemas with `contracts/service-v1.json`, then use declared MCP operations such as `search`, `source_capture_url`, `source_capture_text`, `proposal_evaluate`, `proposal_submit` and `proposal_status`. There are no generic `query` or `propose` tools.
3. **Execution model:** Use MCP discovery for the current tool list and argument schemas, then call declared operations such as `search`, `source_capture_url`, `source_capture_text`, `proposal_prepare`, `proposal_evaluate`, `proposal_submit` and `proposal_status`. Consult the pinned contract only to diagnose a concrete missing-tool or schema mismatch. There are no generic `query` or `propose` tools.
## Setting up a project directory
contracts/catalog-v1.json
+8
View File
@@ -377,6 +377,14 @@
"mcp"
]
},
{
"capability": "proposal:evaluate",
"mutates_knowledge": false,
"name": "proposal_prepare",
"transports": [
"mcp"
]
},
{
"capability": "proposal:evaluate",
"mutates_knowledge": false,
contracts/service-v1.json
+26 -1
View File
@@ -1,6 +1,6 @@
{
"api_version": "v1",
"catalog_sha256": "8c721bdb8b78e50606d4d479d716076f1f609ecb908fb32c10b7eaccc06eb04e",
"catalog_sha256": "872d3bf0647f9ec1a3f0e6d3d994fe03076b78bb7468cf3a24218d4b8ca8b075",
"contract_id": "gewu.service.v1",
"contract_version": 1,
"operations": [
@@ -156,6 +156,14 @@
"mcp"
]
},
{
"capability": "proposal:evaluate",
"mutates_knowledge": false,
"name": "proposal_prepare",
"transports": [
"mcp"
]
},
{
"capability": "proposal:evaluate",
"mutates_knowledge": false,
@@ -1220,6 +1228,23 @@
},
"name": "proposal_submit"
},
{
"description": "Fill canonical evidence and payload digests without storing the proposal.",
"input_schema": {
"properties": {
"proposal_json": {
"title": "Proposal Json",
"type": "string"
}
},
"required": [
"proposal_json"
],
"title": "proposal_prepareArguments",
"type": "object"
},
"name": "proposal_prepare"
},
{
"description": "Evaluate one bounded proposal without storing or publishing it.",
"input_schema": {
manifest.json
+25 -25
View File
@@ -1,6 +1,6 @@
{
"pack_id": "gewu-research",
"pack_version": "0.20.0",
"pack_version": "0.20.6",
"manifest_version": "1.0.0",
"status": "local-contract-and-checkpoint-bound-hosts-not-qualified",
"execution": "existing-local-agent-host",
@@ -10,7 +10,7 @@
"contract_id": "gewu.service.v1",
"contract_version": 1,
"path": "contracts/service-v1.json",
"sha256": "1b70e5eaddbd0db869b679462fe14167c221aab44e3691aa38409c9c1f7c7799",
"sha256": "bd60a522baffb3ec2824676d550f17591c5d2faf9d028db4eb23252785cacd4d",
"scope": "Generated local MCP tool input schemas and v1 record schemas; native host workflows remain NOT RUN."
}
],
@@ -28,7 +28,7 @@
"skills": [
{
"id": "kb-ingest",
"version": "0.5.0",
"version": "0.6.0",
"entry": "skills/kb-ingest/SKILL.md",
"pipeline": "pipelines/ingest.md",
"pipeline_version": "0.4.0"
@@ -65,7 +65,7 @@
"module_id": "profile",
"kind": "profile",
"entry": "AGENTS.md",
"implementation_version": "0.2.0",
"implementation_version": "0.3.0",
"provided_contract": {
"id": "gewu.profile.profile",
"version": "0.1.0"
@@ -73,8 +73,8 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0",
"gewu.skill.kb-ingest@0.5.0",
"gewu.protocol.contributions@0.4.0",
"gewu.skill.kb-ingest@0.6.0",
"gewu.skill.kb-query@0.1.0",
"gewu.skill.kb-curate@0.1.0",
"gewu.skill.kb-synthesize@0.2.0"
@@ -94,7 +94,7 @@
"compatible_checkpoint_contracts": [
"gewu.local-checkpoint@0.1.0"
],
"sha256": "5eaa17b8001004b31f3aa321ba83969d048329670320e8ca9fcd5fae3ddea3ee",
"sha256": "ce5c36aa8d985ff7e889d1c58278d03ee0f3f199f3d8f8158b6cec79b49559a3",
"hash_encoding": "utf8-no-bom-lf"
},
{
@@ -137,10 +137,10 @@
"module_id": "contributions",
"kind": "protocol",
"entry": "protocols/CONTRIBUTIONS.md",
"implementation_version": "0.4.0",
"implementation_version": "0.7.0",
"provided_contract": {
"id": "gewu.protocol.contributions",
"version": "0.3.0"
"version": "0.4.0"
},
"requires_contracts": [
"gewu.local-checkpoint@0.1.0"
@@ -159,14 +159,14 @@
"compatible_checkpoint_contracts": [
"gewu.local-checkpoint@0.1.0"
],
"sha256": "5dbaa46ab7a4f46c682465da5308fd2bff3106d4d2319796cf5d9893ac715598",
"sha256": "e9fe2f910dc3a84bd34e03b7ef974852ed8f697d5f23d667654745ebef76ff89",
"hash_encoding": "utf8-no-bom-lf"
},
{
"module_id": "kb-ingest",
"kind": "skill",
"entry": "skills/kb-ingest/SKILL.md",
"implementation_version": "0.8.0",
"implementation_version": "0.11.0",
"provided_contract": {
"id": "gewu.skill.kb-ingest",
"version": "0.6.0"
@@ -174,7 +174,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0",
"gewu.protocol.contributions@0.4.0",
"gewu.pipeline.kb-ingest-pipeline@0.5.0"
],
"required_service_capabilities": [
@@ -190,7 +190,7 @@
"compatible_checkpoint_contracts": [
"gewu.local-checkpoint@0.1.0"
],
"sha256": "aebea590e8a7ad3ff520ba3c22c690a8d6d5aac1a32d883d55eb339e2904de9c",
"sha256": "d09c3bdae82c7d17e4fa9b61e759284cf0cb533f03d29b1829893ad570568002",
"hash_encoding": "utf8-no-bom-lf"
},
{
@@ -205,7 +205,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0"
"gewu.protocol.contributions@0.4.0"
],
"required_service_capabilities": [
"proposal:evaluate",
@@ -235,7 +235,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0",
"gewu.protocol.contributions@0.4.0",
"gewu.pipeline.kb-query-pipeline@0.1.0"
],
"required_service_capabilities": [
@@ -260,7 +260,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0"
"gewu.protocol.contributions@0.4.0"
],
"required_service_capabilities": [
"knowledge:read"
@@ -284,7 +284,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0",
"gewu.protocol.contributions@0.4.0",
"gewu.pipeline.kb-curate-pipeline@0.1.0"
],
"required_service_capabilities": [
@@ -315,7 +315,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0"
"gewu.protocol.contributions@0.4.0"
],
"required_service_capabilities": [
"knowledge:read",
@@ -345,7 +345,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0",
"gewu.protocol.contributions@0.4.0",
"gewu.pipeline.kb-synthesize-pipeline@0.2.0"
],
"required_service_capabilities": [
@@ -370,7 +370,7 @@
"requires_contracts": [
"gewu.protocol.evidence@0.1.0",
"gewu.protocol.scope@0.1.0",
"gewu.protocol.contributions@0.3.0"
"gewu.protocol.contributions@0.4.0"
],
"required_service_capabilities": [
"knowledge:read"
@@ -397,7 +397,7 @@
"compatible_checkpoint_contracts": [
"gewu.local-checkpoint@0.1.0"
],
"sha256": "2b75652c7bb4181af98b6a2738c30152949d442e020357f2042ed1c759b1a671",
"sha256": "ae6322408732b4883e84adb06b87126598f035d1fcaef66c7b48a8dd0e2a743d",
"hash_encoding": "utf8-no-bom-lf"
},
{
@@ -435,7 +435,7 @@
"compatible_checkpoint_contracts": [
"gewu.local-checkpoint@0.1.0"
],
"sha256": "1b70e5eaddbd0db869b679462fe14167c221aab44e3691aa38409c9c1f7c7799",
"sha256": "bd60a522baffb3ec2824676d550f17591c5d2faf9d028db4eb23252785cacd4d",
"hash_encoding": "utf8-no-bom-lf"
},
{
@@ -453,7 +453,7 @@
"compatible_checkpoint_contracts": [
"gewu.local-checkpoint@0.1.0"
],
"sha256": "8c721bdb8b78e50606d4d479d716076f1f609ecb908fb32c10b7eaccc06eb04e",
"sha256": "872d3bf0647f9ec1a3f0e6d3d994fe03076b78bb7468cf3a24218d4b8ca8b075",
"hash_encoding": "utf8-no-bom-lf"
},
{
@@ -509,13 +509,13 @@
"compatible_checkpoint_contracts": [
"gewu.local-checkpoint@0.1.0"
],
"sha256": "953d5c615f0954470bd2ee3a7a22fa40a2efecab8715bb6158d69d49a6a448ce",
"sha256": "fbc1d51e5c8f6ecd24db219ca149a38b7995c11d38923c58c2b7f46e0614a4f9",
"hash_encoding": "utf8-no-bom-lf"
}
],
"snapshot": {
"algorithm": "sha256",
"scope": "modules array serialized as UTF-8 JSON with sorted keys and compact separators",
"digest": "426a8285bdc9a82e8e937f52338c7aca06306f94883fe1e717bfb62acc740034"
"digest": "5e22d9688aa0d977eda0cb2fde8d5dc9687baf30c731940f36b4e30fc0fab7ab"
}
}
protocols/CONTRIBUTIONS.md
+12 -4
View File
@@ -1,6 +1,6 @@
# Contributions and run recovery
Before a connected workflow, call `diagnostics` and verify that the required MCP tool names are present in the pinned `contracts/service-v1.json` inventory. Use each tool's schema advertised by MCP discovery for its arguments; consult only the relevant pinned contract entry if that schema is missing or differs. Do not read every record schema or the full contract during routine ingestion. Diagnostics reports the authenticated role, authorized projects/capabilities, explicit selected scope, source-transfer limits, service/catalog versions and canonical/index revisions without returning credentials or principal IDs. `connection_status=authenticated` proves only this call; `workflow_status=not_verified` means native workflow acceptance remains separate. If a required tool is absent or its input schema differs, stop before mutation and report the drift. `proposal_evaluate` is the deterministic preflight for one complete, real proposal. Never probe it with a placeholder payload, zero digest or guessed identity; never call `proposal_status` with a fabricated proposal ID. The service's `proposal_submit` response remains authoritative. Owner decisions use a separate owner transport and are not available to agents. Runs are agent-owned, explicitly scoped service records; they do not schedule work or grant capability.
Before a connected workflow, call `diagnostics` and use MCP discovery to confirm the required tools and their advertised input schemas. The schemas discovered from the active server are authoritative for this session. Do not open `manifest.json`, `contracts/service-v1.json`, record schemas, the README or helper-script source just to verify tool availability, arguments or checkpoint behavior. If a required tool is absent or a discovered schema causes a concrete mismatch, stop before mutation and inspect only the relevant pinned contract entry; report the drift if that does not resolve it. Diagnostics reports the authenticated role, authorized projects/capabilities, explicit selected scope, source-transfer limits, service/catalog versions and canonical/index revisions without returning credentials or principal IDs. `connection_status=authenticated` proves only this call; `workflow_status=not_verified` means native workflow acceptance remains separate. `proposal_evaluate` is the deterministic preflight for one complete, real proposal. Never probe it with a placeholder payload, zero digest or guessed identity; never call `proposal_status` with a fabricated proposal ID. The service's `proposal_submit` response remains authoritative. Owner decisions use a separate owner transport and are not available to agents. Runs are agent-owned, explicitly scoped service records; they do not schedule work or grant capability.
## Build attributable proposals
@@ -14,21 +14,29 @@ Every source registration affects both the shared-source policy and each project
Before creating a new node, search the explicit project and shared scope with multiple lexical queries: the proposed title, distinctive terms, aliases and likely synonyms. Inspect plausible matches with `get_node`. If the finding belongs to an existing node, preserve its stable ID and supported content when proposing an evidence-backed update; do not create a second node for the same concept. Keep materially different populations, time periods, settings, mechanisms or conclusions distinct. Similarity is only a review cue. The backend flags exact entity name/alias matches but does not detect duplicate notes, cases, concepts or datapoints; if likely duplicates remain unresolved after search, submit a non-blocking review flag and do not create another node yet.
Finalize claim text and evidence before creating support assessments or digests. Every evidence `assertion_path` must name its exact typed claim field (`claims[n].text`, `assumptions[n].text` or `limitations[n].text`). Its `assertion_digest` must bind to that exact claim text. Set the assessment's `assertion_digest` to the same value and `evidence_digests` to the digests of the claim's final evidence objects. Any edit to claim text or evidence requires regenerating those dependent values and the proposal `payload_digest` before `proposal_evaluate`.
Finalize claim text, evidence and locators, then call the read-only `proposal_prepare` MCP tool with the complete draft proposal. It fills each claim's exact typed `assertion_path`, assertion digest, evidence digests and final payload digest, and returns prepared `proposal_json`. Use that JSON unchanged for `proposal_evaluate`; after any edit, call `proposal_prepare` again. It does not evaluate, store or submit the proposal. Do not run shell commands or manually hash proposal content. If `proposal_prepare` is unavailable, stop before evaluation and report that deterministic digest preparation is missing.
Use the canonical GEWU digest rules, not a hash of the raw quote or raw request text. Canonical JSON is UTF-8 JSON with Unicode preserved, sorted keys, no insignificant whitespace and no NaN values. The assertion digest is SHA-256 of canonical JSON `{"assertion": normalized_text}`, where text has CRLF and CR converted to LF and is normalized to Unicode NFC. An evidence digest is SHA-256 of canonical JSON for the complete evidence object in schema JSON form. The assessment's `evidence_digests` must match those final evidence digests. The proposal payload digest is SHA-256 of canonical JSON for the final operations serialized as GEWU v1 contract records. Recompute it after adding/removing/changing any operation. If the client cannot reproduce these rules, stop before submission and report that digest generation is unavailable.
`proposal_prepare` applies the canonical GEWU digest rules: UTF-8 canonical JSON, Unicode preserved, sorted keys, no insignificant whitespace and no NaN values. It binds assertions to normalized exact claim text, hashes complete evidence objects and hashes the final typed operations. Its server result is authoritative for dependent digest fields.
The proposal envelope must include `schema_version`, `proposal_id`, `revision`, `initiating_project_id`, `scope`, `base_commit`, `payload_digest`, `proposer_id`, `run_id` and `operations`. `proposal_prepare` fills dependent digests; it does not invent the run, scope, principal, base revision or proposal metadata. Do not add unsupported envelope fields such as `created_at` or `reason`. For a PDF citation, use `locator.kind="pdf_page_span"` and the page from the extraction block; `text_span` is for non-PDF text and cannot include a page. Do not call `proposal_evaluate` unless `proposal_prepare` returns `status="prepared"`; evaluate only its returned `proposal_json` unchanged. If preparation returns `status="invalid"`, correct the named issue and retry once. Never probe preparation with `{}` or another incomplete draft.
A claim's `assessment` uses the GEWU fields `assessment_id`, `assertion_digest`, `evidence_digests`, `assessor_id`, `run_id`, `method`, `status` and `rationale`. Do not add guessed fields such as `support_level`, `confidence` or `assessed_at`; `proposal_prepare` reports typed validation issues but does not translate another schema.
`list_projects` exposes authorized IDs, and `get_project_context` returns the selected authorized project's north star, key questions, watch list, taxonomy extensions, research lenses and explicitly saved ingest guidance without exposing its autonomy policy. Projects default to technical, conceptual, cultural-historical and financial-market lenses. These are coverage prompts, not quotas. Use this context for ingest/curation; keep objective and scope explicit and never infer authority from a folder or project name.
When available, `ingest_review_create` stores a source-bound review flag. Submit it after or alongside normal proposal processing, never as a reason to wait on human review before evaluating/submitting plausible findings. Bind it to the durable run and an extraction owned by the submitting agent and granted project; the exact excerpt and locator must match the staged extraction. A queue entry is a notification/filter and feedback record, not a proposal, source registration, acceptance result or publication. Human reviewers may confirm relevance, mark it irrelevant or defer feedback. Reusable project guidance is added to future context only when a human explicitly saves a rule. If the review operation is absent, preserve and report plausible uncertain material rather than silently deferring it.
The `ingest_review_create` MCP tool takes one `review_json` string, not review fields as top-level tool arguments. Its request requires `review_id`, `project_id`, `run_id`, `source_revision_id`, `extraction_id`, `locator`, `excerpt`, `lens`, `summary` and `relevance_rationale`; use `candidate_facet_labels` for defined label candidates. The field is `relevance_rationale`, not `reason`. A call invokes advisory Jev relevance/lens and facet judgments when configured and returns any `jev_facet_resolutions`; it is not a prerequisite for ordinary proposal acceptance. Use this call when a label needs explicit Jev normalization or human categorization review, and treat its saved item as a non-blocking notification.
Current capability gaps that must be reported when relevant: retrieval is lexical only; URL capture requires an exact operator-allowlisted public host. URL and selected-file intake support UTF-8 text/Markdown/HTML and PDF; selected `.vtt` and `.srt` files are parsed into timestamped cues and tagged `capture_method=transcript`. Transcript evidence must cite the exact text span and matching cue start/end milliseconds. `source_capture_text` accepts explicitly supplied UTF-8 plain text or Markdown up to 1 MB. It requires a stable capture ID, stores the completed metadata candidate in a private bounded receipt journal, returns the same candidate for an exact retry, and conflicts if the same ID is reused with changed inputs. Source records distinguish URL fetch, local-file, transcript, declared NotebookLM-export and pasted-text capture methods; the NotebookLM label does not verify export provenance. HTML citations refer to the visible-text extraction, not markup-byte offsets. PDF extraction is qualified only on Linux in a sandbox; scanned PDFs return `PDF_NO_TEXT`. Failed URL captures can be registered as artifact-free references, and source records include publisher, author, publication date, license and reliability fields. supplied NotebookLM exports, Windows/macOS PDF extraction, native host workflows and real-source location checks remain unqualified. There are no vector/hybrid search or project-export/import operations. Duplicate review can nominate exact entity name/alias matches only within the caller-readable explicit scope; it does not merge records. Reversible merge, protected owner decisions, promotion and synthesis-dependency warnings remain unavailable. Do not describe these as implemented merely because a pipeline or record schema can represent related data.
Use read tools for retrieval and typed MCP proposals for contributions. Source upload completion establishes a staged artifact, not an accepted source. Treat policy acceptance and index visibility as separate outcomes. Corrections append revisions; accepted records are corrected through new linked proposals.
Create the service run with `run_create` before capture or proposal submission. Persist returned run identity and operation input digests in the private local checkpoint, then append service checkpoints with `run_checkpoint`; `run_status` is authoritative after uncertain calls and returns the latest revision. Each update uses `expected_revision`; retry the same checkpoint unchanged after uncertainty. The local helper remains a private host-side recovery mirror and is useful when offline. Checkpoints include pack/skill/pipeline versions and hashes, explicit project/shared scope and lens, source/extraction references, stage, output hashes, included/excluded/skipped coverage, pending side effects and last receipt. The helper rejects credential/source-body fields and verifies pinned module hashes on resume. The pack does not provide a host scheduler.
Use the checkpoint contract in the [pack README](../README.md) and resolve the module registry in the manifest. Record its snapshot digest and every applicable module's contract ID/version and file hash, including the profile and shared protocols. Verify those hashes before resuming; a mismatch requires the original snapshot or an explicitly revalidated successor run.
Create the service run immediately after diagnostics, project context and explicit scope, before source transfer or proposal work. `run_create` takes `request_json`; supply a JSON object with `run_id`, `objective`, `scope`, `lens`, `pack_id`, `pack_version`, `pack_snapshot_digest` and `module_pins`. Use the compact `run_pins` object in `.agents/gewu-install.json` for the four pack fields; the installer writes the required `module_id`, `implementation_version` and `sha256` for every module there. Do not inspect the manifest, run schemas or helper implementation to reconstruct these values. If `run_pins` is absent, inspect only the manifest's pack identifiers, snapshot digest and module entries. Persist the returned run identity and operation input digests in the private local checkpoint after the service run exists, then append service checkpoints with `run_checkpoint`; `run_status` is authoritative after uncertain calls and returns the latest revision. Each update uses `expected_revision`; retry the same checkpoint unchanged after uncertainty. The `sources` field is an array of objects with `source_revision_id` and `status` (`selected`, `captured`, `extracted`, `unavailable` or `failed`); include `artifact_sha256` and `extraction_id` when known. Do not store source revision IDs as bare strings. Checkpoints include explicit project/shared scope and lens, stage, output hashes, included/excluded/skipped coverage, pending side effects and last receipt. The helper rejects credential/source-body fields and verifies pinned module hashes on resume. The pack does not provide a host scheduler.
Both `run_create` and `run_checkpoint` take one `request_json` string containing their complete request object. A checkpoint object must include `run_id` and the exact current `expected_revision` from `run_status`, plus `stage`; include source references as objects and a coverage object. Do not send checkpoint fields as top-level MCP arguments or omit the run identity/revision. Update local checkpoint files only through the host's approved file tools; if those tools are unavailable, the service checkpoint is authoritative and must still be recorded.
When upload completion is uncertain, call `source_upload_status` with the saved upload ID and resume only from its verified offsets. When proposal submission is uncertain, call `proposal_status` with the same proposal ID and revision. If accepted, reuse the returned receipt and do not resubmit. If evaluated but not accepted, inspect findings; if withdrawn, that proposal revision cannot be submitted. Never generate a fresh proposal ID to hide network uncertainty. Resume from preserved artifacts and service receipts; changed content requires a new proposal revision. Do not rerun a model and assume it reproduces the same output.
scripts/install.py
+53 -2
View File
@@ -42,13 +42,18 @@ def _safe_regular(path: Path) -> bool:
def _validated_state(state: object) -> dict[str, Any]:
if not isinstance(state, dict) or set(state) != {
base_keys = {
"format",
"mode",
"pack_path",
"files",
"routes",
}:
}
if (
not isinstance(state, dict)
or not base_keys.issubset(state)
or set(state) - base_keys - {"run_pins"}
):
raise ValueError("unrecognized GEWU installation state")
if (
state.get("format") != "gewu-install-state@1"
@@ -82,6 +87,38 @@ def _validated_state(state: object) -> dict[str, Any]:
or not re.fullmatch(r"[a-f0-9]{64}", route["block_sha256"])
):
raise ValueError("GEWU installation routes are invalid")
if "run_pins" in state:
pins = state["run_pins"]
if not isinstance(pins, dict) or set(pins) != {
"pack_id",
"pack_version",
"pack_snapshot_digest",
"module_pins",
}:
raise ValueError("GEWU run pins are invalid")
if (
not isinstance(pins.get("pack_id"), str)
or not pins["pack_id"]
or not isinstance(pins.get("pack_version"), str)
or not pins["pack_version"]
or not isinstance(pins.get("pack_snapshot_digest"), str)
or not re.fullmatch(r"[a-f0-9]{64}", pins["pack_snapshot_digest"])
or not isinstance(pins.get("module_pins"), list)
or not pins["module_pins"]
):
raise ValueError("GEWU run pins are invalid")
for module in pins["module_pins"]:
if (
not isinstance(module, dict)
or set(module) != {"module_id", "implementation_version", "sha256"}
or not isinstance(module.get("module_id"), str)
or not module["module_id"]
or not isinstance(module.get("implementation_version"), str)
or not module["implementation_version"]
or not isinstance(module.get("sha256"), str)
or not re.fullmatch(r"[a-f0-9]{64}", module["sha256"])
):
raise ValueError("GEWU run pins are invalid")
return state
@@ -159,6 +196,7 @@ def install(workspace: Path, source: Path = PACK_ROOT, mode: str = "integrated")
# Prepare all bytes and validate the destinations before mutating the workspace.
copied = {name: path.read_bytes() for name, path in source_files.items()}
manifest = json.loads(copied["manifest.json"])
route_originals = {
name: target.read_bytes() if target.exists() else None for name, target in targets.items()
}
@@ -171,6 +209,19 @@ def install(workspace: Path, source: Path = PACK_ROOT, mode: str = "integrated")
"mode": mode,
"pack_path": PACK_DIR.as_posix(),
"files": {name: digest(content) for name, content in copied.items()},
"run_pins": {
"pack_id": manifest["pack_id"],
"pack_version": manifest["pack_version"],
"pack_snapshot_digest": manifest["snapshot"]["digest"],
"module_pins": [
{
"module_id": module["module_id"],
"implementation_version": module["implementation_version"],
"sha256": module["sha256"],
}
for module in manifest["modules"]
],
},
"routes": {
name: {
"block_sha256": digest(route_blocks[name]),
skills/kb-ingest/SKILL.md
+5 -5
View File
@@ -6,23 +6,23 @@ description: Process explicitly selected sources for a GEWU project into evidenc
Read the pack's [evidence](../../protocols/EVIDENCE.md), [scope](../../protocols/SCOPE.md) and [contribution](../../protocols/CONTRIBUTIONS.md) protocols, then use the [ingestion pipeline](../../pipelines/ingest.md).
Inputs are the selected source(s), explicit project and the user's research objective. Use `get_project_context` for the project's north star, key questions, watch list, taxonomy extensions, research lenses and saved ingest guidance. Every project defaults to four research lenses: technical, conceptual, cultural-historical and financial-market. Treat these as coverage prompts, not quotas. Verify the supported tool/schema contract, including `ingest_review_create` when borderline-review intake is available. Use the connector for local byte transfer; it returns a typed source metadata candidate without exposing the selected path or bytes. Selected `.vtt` and `.srt` files produce timestamped transcript cues; cite exact cue start/end times and the quote span. A declared NotebookLM export label is metadata only and is not export verification. For text explicitly pasted by the user, use `source_capture_text` with a stable capture ID and source key; it accepts UTF-8 plain text or Markdown up to 1 MB, archives and extracts it, and returns a typed candidate marked `capture_method=pasted-text`. Retry uncertain calls with the same capture ID and identical inputs; changed input under that ID conflicts. It does not register the candidate. `source_capture_url` fetches only operator-allowlisted public hosts and stages supported text, Markdown, HTML or PDF bytes; it also returns a source metadata candidate but does not register it. Add each new source's exact candidate as `register_source` in the proposal that cites it, or verify the source was already registered and readable. Use the candidate's `revision_id` and capture's `extraction_id`; never invent or substitute these IDs. Submit candidates with the normal typed proposal flow, and use `capture-failed` candidates only as bookmarks, never as factual evidence. URL source IDs follow the sanitized URL; local source IDs follow the connector's configured file ID; pasted-text source IDs are derived from the supplied stable key and revision IDs include the content digest. PDF extraction currently requires Linux bubblewrap, Poppler and prlimit. Unsupported formats and unavailable parsers must be reported without inventing an extraction. If the service is unavailable, report the blocked stage and only prepare an unaccepted local draft when that work was requested.
Inputs are the selected source(s), explicit project and the user's research objective. Use `get_project_context` for the project's north star, key questions, watch list, taxonomy extensions, research lenses and saved ingest guidance. Every project defaults to four research lenses: technical, conceptual, cultural-historical and financial-market. Treat these as coverage prompts, not quotas. Confirm required tools and arguments from MCP discovery and `diagnostics`; use the schema attached to each discovered tool. For Antigravity, do not open generated MCP cache JSON files or read future tool schemas in advance. Create the service run after diagnostics and project context; use `run_pins` from `.agents/gewu-install.json` as defined in CONTRIBUTIONS.md. If a required tool is absent or a call produces a concrete schema mismatch, inspect only that tool's pinned contract entry and report unresolved drift before mutation. Use the connector for local byte transfer; it returns a typed source metadata candidate without exposing the selected path or bytes. Selected `.vtt` and `.srt` files produce timestamped transcript cues; cite exact cue start/end times and the quote span. A declared NotebookLM export label is metadata only and is not export verification. For text explicitly pasted by the user, use `source_capture_text` with a stable capture ID and source key; it accepts UTF-8 plain text or Markdown up to 1 MB, archives and extracts it, and returns a typed candidate marked `capture_method=pasted-text`. Retry uncertain calls with the same capture ID and identical inputs; changed input under that ID conflicts. It does not register the candidate. `source_capture_url` fetches only operator-allowlisted public hosts and stages supported text, Markdown, HTML or PDF bytes; it also returns a source metadata candidate but does not register it. Add each new source's exact candidate as `register_source` in the proposal that cites it, or verify the source was already registered and readable. Use the candidate's `revision_id` and capture's `extraction_id`; never invent or substitute these IDs. Submit candidates with the normal typed proposal flow, and use `capture-failed` candidates only as bookmarks, never as factual evidence. URL source IDs follow the sanitized URL; local source IDs follow the connector's configured file ID; pasted-text source IDs are derived from the supplied stable key and revision IDs include the content digest. PDF extraction currently requires Linux bubblewrap, Poppler and prlimit. Unsupported formats and unavailable parsers must be reported without inventing an extraction. If the service is unavailable, report the blocked stage and only prepare an unaccepted local draft when that work was requested.
Extract useful atomic assertions, problems, techniques or cases with their conditions across all four lenses where the source contains relevant material. Consider cultural history, philosophical context and market/financial evidence as project data when they explain the project's origins, concepts, adoption, audience, costs or viability; they are not automatically peripheral. First try to connect uncertain material to the north star, a key question, a saved rule or one of the lenses. Exclude only clearly unrelated material, unavailable evidence, or material the user explicitly excluded. Do not defer a section merely because its relevance is indirect, non-technical or historical. Continue normal evidence-backed proposal evaluation/submission for all plausible useful findings. When a human should review relevance or categorization, submit a bounded `ingest_review_create` flag with the exact extraction excerpt/locator, lens, summary and reason before finalizing the associated proposal facets. This persists for notification/filtering and never blocks the evidence-backed proposal flow.
Extract useful atomic assertions, problems, techniques or cases with their conditions across all four lenses where the source contains relevant material. Consider cultural history, philosophical context and market/financial evidence as project data when they explain the project's origins, concepts, adoption, audience, costs or viability; they are not automatically peripheral. First try to connect uncertain material to the north star, a key question, a saved rule or one of the lenses. Exclude only clearly unrelated material, unavailable evidence, or material the user explicitly excluded. Do not defer a section merely because its relevance is indirect, non-technical or historical. Continue normal evidence-backed proposal evaluation/submission for all plausible useful findings. When human relevance/categorization feedback or explicit Jev label normalization is needed, call `ingest_review_create` with a single `review_json` string, the complete `IngestReviewCreate` fields, and the exact extraction excerpt/locator. The required rationale field is `relevance_rationale`, not `reason`; include `review_id` and `extraction_id`. This persists a non-blocking notification and never blocks the evidence-backed proposal flow.
Before building each proposal, use the principal ID assigned to the configured agent credential for `proposer_id`, node `agent_id` and assessment `assessor_id`; do not use the model or harness name. Finish each claim and its evidence before computing `assertion_digest` or `evidence_digests`. Evidence assertion paths must point to their exact claim field. Recompute the evidence and assessment digests after any claim/evidence edit, then recompute `payload_digest` after any operation edit. Follow the exact canonical digest rules in [CONTRIBUTIONS.md](../../protocols/CONTRIBUTIONS.md#build-attributable-proposals) and [EVIDENCE.md](../../protocols/EVIDENCE.md). If the identity or digest inputs are unknown, stop and report the missing information rather than guessing.
Before building each proposal, use the principal ID assigned to the configured agent credential for `proposer_id`, node `agent_id` and assessment `assessor_id`; do not use the model or harness name. If the operator has not provided that principal ID, stop before preparing or evaluating a proposal; diagnostics intentionally omits it. The proposal envelope requires `schema_version`, `proposal_id`, `revision`, `initiating_project_id`, `scope`, `base_commit`, `payload_digest`, `proposer_id`, `run_id` and `operations`. Use the current canonical revision from diagnostics as `base_commit`; do not add unsupported envelope fields. Finalize claim text, evidence and locators, then call `proposal_prepare` with the complete draft. It fills dependent digests but does not invent metadata or correct an invalid citation locator. For PDF evidence use `locator.kind="pdf_page_span"` with the extraction block's page; `text_span` cannot include a page. Proceed only when the tool returns `status="prepared"`, and pass its returned `proposal_json` unchanged to evaluation. If it returns `status="invalid"`, correct only the named issue and retry once. Do not probe it with an empty or incomplete proposal. If the tool is unavailable or rejects a corrected proposal, stop and report the concrete error.
Use the input schema advertised for each MCP tool. If a call rejects its arguments, inspect only that tool's advertised/pinned schema, correct the arguments and retry once; do not probe unrelated tools or read the entire schema directory. Do not test proposal tools with invented IDs, placeholder identities, zero digests or incomplete payloads. Evaluate only the complete proposal for this learning unit; use `proposal_status` only for its real proposal ID and revision.
For `tags`, `audience`, `use_for` and `domain`, inspect the `facet_labels` catalog returned by `get_project_context`. Reuse a canonical project label when its meaning fits; do not create spelling, phrasing or singular/plural variants. Propose only useful, evidence-backed labels, with at most five tags and three labels in each other facet (14 total). Prefer `candidate_facet_labels` entries shaped as `{facet, value, definition}` so a genuinely new concept has a concise, distinguishing definition (500 characters max). Legacy `candidate_tags`, `candidate_audience`, `candidate_use_for` and `candidate_domain` fields remain valid for labels that do not need a definition. Leave uncertain labels out. Jev compares agent-proposed candidates with the closest project labels and chooses an existing label or the `new_label` option; it does not invent the proposed label text. The backend may add only a distinct new candidate that Jev ranks in that facet's configured top N (tags: up to five; other facets: up to three) and that clears its fit, new-label and confidence thresholds. An existing-label match is reused rather than duplicated. For the associated proposal, use Jev's selected canonical existing label for that facet. Use a new candidate as a project vocabulary label only when `added_to_project` is true; otherwise omit it from the proposal facets and leave its evidence-backed finding intact. Treat returned `jev_facet_resolutions` as the recorded decision; never claim a label was added unless `added_to_project` is true. Label promotion requires the server's local embedding model and Jev configuration; without them the candidate remains only a review signal. Label decisions only normalize facets; they do not gate evidence assessment, proposal submission or acceptance.
If available, Jev also assesses relevance and lens in the same advisory call. Use the saved scores to focus human review; they are not evidence assessment or an instruction to silently rewrite accepted knowledge. When a reviewer decides, record the feedback and apply reusable project guidance only when the reviewer explicitly opts in. Preserve the service's exact source/extraction/location bindings. Map each factual field to evidence; assess semantic support separately. Record clear exclusions and genuinely skipped/deferred sections. Optimize for useful coverage rather than maximum note count.
If available, Jev also assesses relevance and lens in the same advisory call. Facet-label comparison runs through `ingest_review_create`; use the returned `jev_facet_resolutions` as the recorded decision, then put the selected canonical existing label on the associated proposal. Use a new label only when `added_to_project` is true. Use saved Jev scores to focus human review; they are not evidence assessment or an instruction to silently rewrite accepted knowledge. When a reviewer decides, record the feedback and apply reusable project guidance only when the reviewer explicitly opts in. Preserve the service's exact source/extraction/location bindings. Map each factual field to evidence; assess semantic support separately. Record clear exclusions and genuinely skipped/deferred sections. Optimize for useful coverage rather than maximum note count.
Submit bounded bundles under the affected project/shared policies. Repair actionable findings without weakening evidence, changing scope to evade policy or assuming owner authority. Resume uncertain calls through status lookup. Report accepted node IDs and receipts, pending/rejected work, archive/extraction limits and actual coverage. A successful upload or schema check is not a completed ingestion.
Every `register_source` operation affects both the source's `ingested_for` project policy and the shared-source policy. Set proposal scope `include_shared=true` as well as the project scope, even when the source is relevant only to one project. Confirm the credential authorizes both. If shared scope or authorization is unavailable, stop and report the missing policy/capability; do not submit a project-only bundle that will fail scope validation.
For a captured source, use the returned extraction status and block metadata to plan review. Read source text through bounded `source_extraction_block` calls using the exact `extraction_id` and `block_id`; inspect relevant slices across all pages/sections, including PDFs that the host's file reader cannot open. The default response contains normalized text and its offset but omits the large original-source mapping; set `include_original_mapping=true` only when an original-text span is needed. Citation quote and locator offsets bind to the normalized extraction either way. Record partial or unavailable extraction coverage explicitly. Do not infer that a source was analyzed just because upload and extraction succeeded.
For a captured source, use the returned extraction status and block metadata to plan review. Read source text through bounded `source_extraction_block` calls using the exact `extraction_id` and `block_id`; inspect relevant slices across all pages/sections, including PDFs that the host's file reader cannot open. Read a slice large enough to include a complete relevant sentence, then copy a concise contiguous quote exactly from its normalized text and use the returned offset. At most one adjacent slice should be needed to complete a citation; do not issue repeated offset probes to chase quote boundaries. If PDF layout splits a sentence across columns, use a shorter contiguous excerpt that still supports the assertion or report the citation limitation. The default response contains normalized text and its offset but omits the large original-source mapping; set `include_original_mapping=true` only when an original-text span is needed. Citation quote and locator offsets bind to the normalized extraction either way. Record partial or unavailable extraction coverage explicitly. Do not infer that a source was analyzed just because upload and extraction succeeded.
Before creating a new node, search the explicit project and shared scope using its proposed title plus distinctive terms, aliases and alternate phrasings. Search likely synonyms separately when title search is inconclusive, then inspect candidate nodes and claims with `get_node`. Search is lexical, so use more than one query. If a finding belongs to an existing node, preserve its stable `node_id` and existing supported content when proposing an evidence-backed update; do not mint another ID for the same concept. Keep separate nodes when populations, time periods, settings, mechanisms or conclusions materially differ. Similarity is a review cue, not proof of identity; if the distinction remains uncertain, create a non-blocking review flag and do not create a second node until resolved. The current backend only flags exact entity name/alias matches; it does not detect duplicate notes, cases, concepts or datapoints for you.
tests/test_install.py
+51
View File
@@ -0,0 +1,51 @@
import importlib.util
import json
import sys
from pathlib import Path
PACK_ROOT = Path(__file__).resolve().parents[1]
INSTALL_PATH = PACK_ROOT / "scripts" / "install.py"
spec = importlib.util.spec_from_file_location("gewu_install", INSTALL_PATH)
assert spec and spec.loader
gewu_install = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = gewu_install
spec.loader.exec_module(gewu_install)
def test_install_receipt_contains_service_run_pins(tmp_path: Path) -> None:
workspace = tmp_path / "project"
state = gewu_install.install(workspace, PACK_ROOT, "dedicated")
manifest = json.loads((PACK_ROOT / "manifest.json").read_text(encoding="utf-8"))
assert state["run_pins"] == {
"pack_id": manifest["pack_id"],
"pack_version": manifest["pack_version"],
"pack_snapshot_digest": manifest["snapshot"]["digest"],
"module_pins": [
{
"module_id": module["module_id"],
"implementation_version": module["implementation_version"],
"sha256": module["sha256"],
}
for module in manifest["modules"]
],
}
assert gewu_install._validated_state(state) == state
installed_state = json.loads(
(workspace / gewu_install.STATE_PATH).read_text(encoding="utf-8")
)
assert installed_state["run_pins"] == state["run_pins"]
gewu_install.uninstall(workspace)
def test_uninstall_accepts_legacy_install_receipt(tmp_path: Path) -> None:
workspace = tmp_path / "project"
state = gewu_install.install(workspace, PACK_ROOT, "dedicated")
state.pop("run_pins")
(workspace / gewu_install.STATE_PATH).write_text(
json.dumps(state), encoding="utf-8"
)
gewu_install.uninstall(workspace)