Files
GEWU_agent/skills/kb-ingest/SKILL.md
T

41 lines
17 KiB
Markdown
Raw Normal View History

---
name: kb-ingest
description: Process explicitly selected sources for a GEWU project into evidence-bound MCP proposals, with support assessments, coverage and resumable receipts.
---
# Ingest selected sources
Preflight: require the operator-supplied authenticated principal ID before transferring source bytes. If absent, request that non-secret identifier and stop; never inspect credentials, environment variables, host configuration or service databases to discover it. For resumed work, resolve the existing run and preserve its pack pins and source identities.
Read the pack's [evidence](../../protocols/EVIDENCE.md), [scope](../../protocols/SCOPE.md), [contribution](../../protocols/CONTRIBUTIONS.md) and, when online verification is enabled, [online verification](../../protocols/ONLINE_VERIFICATION.md) protocols, then use the [ingestion pipeline](../../pipelines/ingest.md).
If the configured selected-file connector is missing or fails to initialize because `GEWU_TOKEN` is absent, stop at source capture and report the exact host-connector setup error. Do not inspect Antigravity configuration, credentials, environment variables, host files or MCP cache; do not launch repeated sessions to rediscover the same unavailable tool. Continue only after the operator repairs the connector environment. This is a host setup failure, not a reason to switch tools or synthesize a source candidate.
Keep each source run bounded. During ingestion, use MCP responses and the selected source's extraction as the source of truth. Do not search or read repository reports, tests, backend implementation, database files or unrelated source files to answer the research question. The only local workflow data allowed is the single `run_pins` object in `.agents/gewu-install.json` when creating a new run; for a concrete tool-schema error, inspect only that tool's pinned contract entry. A `run_create` call is a real durable workflow action, never a schema probe: assemble its complete request with every required field and non-empty module pins before calling it. Never test it with placeholder IDs, missing fields or empty arrays. If a required value cannot be obtained from the prescribed context or pins, stop and report the missing value rather than iterating on guesses.
Inputs are the selected source(s), explicit project and the user's research objective. Use `get_project_context` for the project's north star, key questions, watch list, taxonomy extensions, research lenses and saved ingest guidance. Every project defaults to four research lenses: technical, conceptual, cultural-historical and financial-market. Treat these as coverage prompts, not quotas. Confirm required tools and arguments from MCP discovery and `diagnostics`; use the schema attached to each discovered tool and the Antigravity argument reminders in CONTRIBUTIONS.md. For Antigravity, do not use shell tools or inspect the host MCP cache. On a schema mismatch, use only the relevant pinned entry in `contracts/service-v1.json` and correct once. Use GEWU tools only; Jev runs inside the backend through `ingest_review_create`, not a standalone Jev MCP server. In Antigravity, call `diagnostics`, `get_project_context` and `run_create` once each in order; after a successful call, reuse its returned state rather than repeating it. Do not call `get_source` for a staged candidate: use its exact returned candidate in the proposal's `register_source` operation and read the source's citation through its staged extraction. Create the durable service run with `run_create` after diagnostics and project context; pass its `request_json` envelope using current `run_pins` from `.agents/gewu-install.json`. Do not look for a local `Runs/<run_id>` file before creation; run state is service-owned. Never reuse a run ID after changing the pack or create payload. Use the connector for local byte transfer; `upload_selected_file` requires both the configured `file_id` and the authorized `project_id`. It returns a typed source metadata candidate without exposing the selected path or bytes. Selected `.vtt` and `.srt` files produce timestamped transcript cues; cite exact cue start/end times and the quote span. A declared NotebookLM export label is metadata only and is not export verification. For text explicitly pasted by the user, use `source_capture_text` with a stable capture ID and source key; it accepts UTF-8 plain text or Markdown up to 1 MB, archives and extracts it, and returns a typed candidate marked `capture_method=pasted-text`. Retry uncertain calls with the same capture ID and identical inputs; changed input under that ID conflicts. It does not register the candidate. `source_capture_url` fetches only operator-allowlisted public hosts and stages supported text, Markdown, HTML or PDF bytes; it also returns a source metadata candidate but does not register it. Add each new source's exact candidate as `register_source` in the proposal that cites it, or verify the source was already registered and readable. Use the candidate's `revision_id` and capture's `extraction_id`; never invent or substitute these IDs. Submit candidates with the normal typed proposal flow, and use `capture-failed` candidates only as bookmarks, never as factual evidence. URL source IDs follow the sanitized URL; local source IDs follow the connector's configured file ID; pasted-text source IDs are derived from the supplied stable key and revision IDs include the content digest. PDF extraction currently requires Linux bubblewrap, Poppler and prlimit. Unsupported formats and unavailable parsers must be reported without inventing an extraction. If the service is unavailable, report the blocked stage and only prepare an unaccepted local draft when that work was requested.
Extract useful atomic assertions, problems, techniques or cases with their conditions across all four lenses where the source contains relevant material. Consider cultural history, philosophical context and market/financial evidence as project data when they explain the project's origins, concepts, adoption, audience, costs or viability; they are not automatically peripheral. First try to connect uncertain material to the north star, a key question, a saved rule or one of the lenses. Exclude only clearly unrelated material, unavailable evidence, or material the user explicitly excluded. Do not defer a section merely because its relevance is indirect, non-technical or historical. Continue normal evidence-backed proposal evaluation/submission for all plausible useful findings. When human relevance/categorization feedback or explicit Jev label normalization is needed, call `ingest_review_create` with a single `review_json` string, the complete `IngestReviewCreate` fields, and the exact extraction excerpt/locator. The required rationale field is `relevance_rationale`, not `reason`; include `review_id` and `extraction_id`. This persists a non-blocking notification and never blocks the evidence-backed proposal flow.
2026-10-14 21:53:35 +08:00
Before building each proposal, use the principal ID assigned to the configured agent credential for `proposer_id`, node `agent_id` and assessment `assessor_id`; do not use the model or harness name. If the operator has not provided that principal ID, stop before preparing or evaluating a proposal; diagnostics intentionally omits it. The proposal envelope requires `schema_version`, `proposal_id`, `revision`, `initiating_project_id`, `scope`, `base_commit`, `payload_digest`, `proposer_id`, `run_id` and `operations`. Use the current canonical revision from diagnostics as `base_commit`; do not add unsupported envelope fields. Finalize claim text, evidence and locators, then call `proposal_prepare` with the complete draft. It fills dependent digests but does not invent metadata or correct an invalid citation locator. For PDF evidence use `locator.kind="pdf_page_span"` with the extraction block's page; `text_span` cannot include a page. Proceed only when the tool returns `status="prepared"`, and pass its returned `proposal_json` unchanged to evaluation. Do not retype, reconstruct or edit that serialized string between tools; keep proposal content limited to the claims and evidence needed for one learning unit. The service can repair exactly one missing final outer `}` if the result passes the full contract; when this occurs the evaluation or submission response includes `input_repaired: true`. Other malformed JSON stays invalid. If evaluation returns `PROPOSAL_JSON_INVALID`, do not submit; rebuild and prepare the draft once, then stop if it remains invalid. A valid `needs-review` result (including `AUTO_ACCEPT_UNSPECIFIED`) is a policy hold for an owner decision, not a transport failure or automatic-acceptance candidate; do not submit it as eligible. If a tool is unavailable or rejects a corrected proposal, stop and report the concrete error.
Use the input schema advertised for each MCP tool. If a call rejects its arguments, inspect only that tool's advertised/pinned schema, correct the arguments and retry once; do not probe unrelated tools or read the entire schema directory. Do not test proposal tools with invented IDs, placeholder identities, zero digests or incomplete payloads. Evaluate only the complete proposal for this learning unit; use `proposal_status` only for its real proposal ID and revision.
2026-10-14 21:53:35 +08:00
For `tags`, `audience`, `use_for` and `domain`, inspect the `facet_labels` catalog returned by `get_project_context`. Reuse a canonical project label when its meaning fits; do not create spelling, phrasing or singular/plural variants. Propose only useful, evidence-backed labels, with at most five tags and three labels in each other facet (14 total). Prefer `candidate_facet_labels` entries shaped as `{facet, value, definition}` so a genuinely new concept has a concise, distinguishing definition (500 characters max). Legacy `candidate_tags`, `candidate_audience`, `candidate_use_for` and `candidate_domain` fields remain valid for labels that do not need a definition. Leave uncertain labels out. Jev compares agent-proposed candidates with the closest project labels and chooses an existing label or the `new_label` option; it does not invent the proposed label text. The backend may add only a distinct new candidate that Jev ranks in that facet's configured top N (tags: up to five; other facets: up to three) and that clears its fit, new-label and confidence thresholds. An existing-label match is reused rather than duplicated. For the associated proposal, use Jev's selected canonical existing label for that facet. Use a new candidate as a project vocabulary label only when `added_to_project` is true; otherwise omit it from the proposal facets and leave its evidence-backed finding intact. Treat returned `jev_facet_resolutions` as the recorded decision; never claim a label was added unless `added_to_project` is true. Label promotion requires the server's local embedding model and Jev configuration; without them the candidate remains only a review signal. Label decisions only normalize facets; they do not gate evidence assessment, proposal submission or acceptance.
If available, Jev also assesses relevance and lens in the same advisory call. Facet-label comparison runs through `ingest_review_create`; use the returned `jev_facet_resolutions` as the recorded decision, then put the selected canonical existing label on the associated proposal. Use a new label only when `added_to_project` is true. Use saved Jev scores to focus human review; they are not evidence assessment or an instruction to silently rewrite accepted knowledge. When a reviewer decides, record the feedback and apply reusable project guidance only when the reviewer explicitly opts in. Preserve the service's exact source/extraction/location bindings. Map each factual field to evidence; assess semantic support separately. Record clear exclusions and genuinely skipped/deferred sections. Optimize for useful coverage rather than maximum note count.
Submit bounded bundles under the affected project/shared policies. Repair actionable findings without weakening evidence, changing scope to evade policy or assuming owner authority. Resume uncertain calls through status lookup. Report accepted node IDs and receipts, pending/rejected work, archive/extraction limits and actual coverage. A successful upload or schema check is not a completed ingestion.
Keep source ownership and proposal scope aligned. A source relevant only to the selected project should use `source.ingested_for=[project_id]`, and its proposal and run should use `scope.projects=[project_id]` with `include_shared=false`. A project-only source does not require shared read or write access. Set `include_shared=true` only when the proposal actually reads shared knowledge or changes a shared source/node; in that case confirm the authenticated credential has the required shared capability before proceeding. Never add shared scope merely to satisfy a blanket rule, and never widen credential grants to work around a project-only validation error. Source registration is governed by the policies for the domains in `source.ingested_for`; it does not automatically affect shared policy.
Run creation accepts one `request_json` argument containing a `RunCreateRequest` object. Its `scope` uses `projects` (not `project_ids`) and must explicitly set `include_shared`; for example, `{"run_id":"<stable-id>","objective":"<bounded objective>","scope":{"projects":["<project-id>"],"include_shared":false},"lens":"<bounded lens>","pack_id":"<pin>","pack_version":"<pin>","pack_snapshot_digest":"<sha256>","module_pins":[...]}`. If validation returns field errors, correct the named field once and reuse the returned run identity. Do not call `list_sources` with `include_shared=true` unless shared access is both required by the task and authorized.
For a captured source, use the returned extraction status and block metadata to plan review. Read source text through bounded `source_extraction_block` calls using the exact `extraction_id` and `block_id`; inspect relevant slices across all pages/sections, including PDFs that the host's file reader cannot open. Read a slice large enough to include a complete relevant sentence, then copy a concise contiguous quote exactly from its normalized text and use the returned offset. At most one adjacent slice should be needed to complete a citation; do not issue repeated offset probes to chase quote boundaries. If PDF layout splits a sentence across columns, use a shorter contiguous excerpt that still supports the assertion or report the citation limitation. The default response contains normalized text and its offset but omits the large original-source mapping; set `include_original_mapping=true` only when an original-text span is needed. Citation quote and locator offsets bind to the normalized extraction either way. Record partial or unavailable extraction coverage explicitly. Do not infer that a source was analyzed just because upload and extraction succeeded.
Before creating a new node, search the explicit project and shared scope using its proposed title plus distinctive terms, aliases and alternate phrasings. Search likely synonyms separately when title search is inconclusive, then inspect candidate nodes and claims with `get_node`. Search is lexical, so use more than one query. If a finding belongs to an existing node, preserve its stable `node_id` and existing supported content when proposing an evidence-backed update; do not mint another ID for the same concept. Keep separate nodes when populations, time periods, settings, mechanisms or conclusions materially differ. Similarity is a review cue, not proof of identity; if the distinction remains uncertain, create a non-blocking review flag and do not create a second node until resolved. The current backend only flags exact entity name/alias matches; it does not detect duplicate notes, cases, concepts or datapoints for you.
Manage context and provider budget across the whole packet. Process one source at a time, use extraction metadata to target relevant slices rather than fetching every character, and retain concise section notes plus exact evidence excerpts instead of full source text. Checkpoint coverage after each source. Prefer separate durable runs for large or independently resumable sources; keep one run for a small, tightly bounded batch only when the source coverage can be checkpointed cleanly. Group proposals by semantic learning unit, not by source or whole batch. If the provider/tool budget is nearly exhausted, checkpoint the current file and report the remaining files/sections for continuation; do not reduce cost by dropping a research lens or deferring cultural-historical or financial-market material.
The API field `budget_tokens` is a per-response size limit, currently counted as Unicode codepoints in canonical JSON (`budget_profile.tokenizer=unicode-codepoints-v1`), despite the field name. The 50,000 ceiling is not the model context window, an ingestion-run budget, or a request to read 50,000 units. Stay below it with focused searches and bounded extraction blocks. Track actual provider/tool limits separately and use run checkpoint `budgets` only for named counters that the host can measure.
Treat documents as capture/provenance units and proposals as semantic learning units. Group a proposal around one project question or one tightly related set of findings, and combine evidence from multiple documents when it supports the same learning. Do not make one proposal per file or one oversized bundle for the entire packet. Keep unrelated learnings separate so each can be evaluated, retried, reviewed and accepted independently. Include each new source candidate in the first proposal that cites it; after its source revision is accepted and readable, later learning proposals cite that revision without registering it again.