Files
GEWU/docs/IMPLEMENTATION_PLAN.md
2026-10-15 10:53:30 +08:00

56 KiB
Raw Permalink Blame History

GEWU implementation plan

13 October 2026 · Planning baseline: PRD 0.8, GOAL, architecture v0.3.

Build a complete local source-to-citation workflow first, evaluate it with the two pilot projects, then expand it to every P0 requirement before full v1. The pilot serves up to five people researching Dome interactive content and an interactive video platform/service. It may use fewer clients, formats and retrieval capabilities than v1; its results must state that coverage explicitly.

This plan guides implementation; milestone status is recorded below and does not imply a pilot or release gate has passed. The repository also contains an initial agent instruction pack in the agents/gewu submodule. Existing pack files need integration work, not recreation. Architecture choices remain proposed until compatibility probes and the affected decisions below are resolved.

Implementation status (14 October 2026)

M1.2/FR-P1 now has an administrator-only project lifecycle control plane. Project IDs are lowercase immutable slugs; creation supplies a named owner and matching initial supported-assessment policy, and commits projects/<project_id>/project.md plus policies/<sha256(policy_id)>.yaml in one canonical revision. Archive writes a new revision, preserves reads/history and prevents new acceptance. HTTP routes are explicitly allowlisted outside the MCP agent catalog, enforce admin/non-agent authority, browser CSRF/origin checks and a streamed 64 KiB creation-body cap, and share the canonical writer lock with acceptance. Local contract, authorization, size, persistence and race tests pass. Owner authentication and project grants exist, but named-owner account provisioning, owner reset and first-owner bootstrap remain pilot/deployment gates; protected proposal challenges are implemented separately under M1.1. See project lifecycle evidence.

M1.1 now supports proposal-triggered owner review for policy-only holds. Submission binds a deterministic challenge to the proposal revision, complete before/after canonical files, affected project owners and policy versions. A separate /owner-mcp server exposes challenge metadata, the complete bounded diff, and one immutable authorize/reject decision; the owner token remains challenge- and audience-bound. On exact proposal resubmission, current owner grants and decisions are rechecked under the canonical writer lock, every affected domain must be covered, and the acceptance manifest records the decision digests, domains, grant versions, rationale and timestamps. Mandatory validation findings cannot be waived. Unit and MCP integration evidence is in owner review evidence. M1.1 remains partial because first-owner bootstrap, reset/recovery, OS-level owner isolation, actual named owners, and independent backup/restore qualification are still external or unimplemented.

M2.2 is in progress: GEWU pack 0.14.0 includes a copy-based installer with integrated/dedicated routing modes, isolated pack placement, reversible marked pointers for AGENTS.md, CLAUDE.md and GEMINI.md, refusal to overwrite an existing installation, and ownership-checked uninstall that preserves unrelated and later user-added files. Nine isolated filesystem tests pass, including fail-closed rejection of traversal paths, unrecognized routes in edited install state and a symlinked .agents directory. Local run checkpoints pin module hashes and reject resume when a newer verified pack snapshot changes, preserving the checkpoint for explicit revalidation. The installer deliberately leaves MCP host configuration untouched because endpoint and credential provisioning are operator-specific. Native discovery, connection, workflow verification and update/resume behavior in native clients remain NOT RUN; this partial implementation does not close M2.2.

M2.3 has authenticated source_capture_url and source_capture_text MCP operations. URL capture is gated by an exact operator-configured host allowlist; the fetch adapter rejects non-public DNS answers, pins connections to validated addresses, revalidates redirects, limits response types, bytes, duration and concurrency, and removes credentials, query strings and fragments from returned URLs. Pasted-text capture accepts UTF-8 plain text or Markdown up to 1 MB, requires a stable capture ID and source key, checks the authenticated project's submission grant, stores bytes in the private content-addressed artifact store, extracts the exact text and returns a proposal-ready source candidate with capture_method=pasted-text; the candidate is not registered until a typed proposal is submitted. A private disk journal binds the capture ID to the principal, project and full input digest, returns the original candidate on exact retry, conflicts on changed inputs and is bounded to 1,024 receipts. Canonical source records now distinguish URL, local-file, transcript, declared NotebookLM-export and pasted-text capture; selected VTT/SRT files retain cue timestamps and transcript provenance. URL and selected-file intake accept PDF alongside UTF-8 text/Markdown/HTML. The local connector returns a path-free source metadata candidate, derives stable file identity and content-digest revision IDs when the caller omits one, and checkpoints completed metadata so uncertain retries reproduce the same candidate; transfer state is private and bounded to 1,024 entries. Linux PDF extraction uses Poppler inside bubblewrap with network and host-user files isolated, an empty environment, read-only system mounts, and CPU, memory, time, page and output limits; tests verify page locators and inability to read a host temporary file. Mixed text and blank pages return a partial extraction; image-only/scanned PDFs return an explicit unavailable PDF_NO_TEXT result, while malformed, encrypted, oversized and unavailable-parser cases return stable failure codes. URL captures return proposal-ready source metadata candidates with stable sanitized-URL identity and digest-bound revision IDs; failed fetches can be registered as artifact-free capture-failed references. Acceptance verifies artifacts only for archived records, and unarchived records remain ineligible as factual evidence. MCP and connector fixture coverage verifies revision changes, proposal registration, authorization, resumable transfer and artifact bytes, including a local archived candidate when PDF extraction is unavailable. A canonical-registry regression test confirms that a replacement parser extraction is added under its new content-addressed ID while the previous extraction remains resolvable for historical citations (tests/application/test_registry.py). The local parser now accepts timestamped WebVTT and SubRip cues; proposal validation requires the cited cue times to match the extraction block, and the connector tags those files as transcripts. These synthetic cue fixtures and contracts are covered by tests. Still open: supplied NotebookLM export fixtures, parser support on Windows/macOS, real-source location verification and pilot source-path qualification.

M2.4 is in progress. Entity aliases are normalized to NFC with collapsed whitespace and checked for empty, repeated and title-duplicate values. Proposal validation compares exact entity names and aliases within the candidate's project and across only the caller-readable projects in the proposal's explicit scope. Scope is enforced both by the authenticated MCP adapter and again in the validator; evaluation does not disclose matches from unreadable or out-of-scope domains. Matches become generic POSSIBLE_DUPLICATE_ENTITY review findings and cannot auto-accept. Publication rechecks the credential's role and project-grant snapshot under the canonical writer lock. Aliases are indexed for lexical search. Duplicate review nominates candidates without creating accepted links or merges. Policy-only exceptions now use the protected owner review and manifest binding described under M1.1; destructive merge and reversible merge history remain unimplemented, so merge operations stay unavailable. Existing typed links bind evidence to relationship conditions and rationale. A synthetic contradiction bundle is validated and published atomically with two separately cited claims retained (tests/domain/test_validation.py, tests/application/test_acceptance.py; contradiction bundle evidence). The service stores the proposal’s contradiction interpretation; it does not independently infer semantic conflict. Reversible merge history, complete ingest workflow coverage and shared provenance qualification remain open.

M0.1–M0.7 have a backend foundation and a first protocol-level vertical flow. Present: pinned Python and web workspaces; health path; strict v1 records and generated schemas/catalog; two-project/shared fixtures; resumable private text/Markdown intake; immutable text extraction with normalization maps; deterministic claim and relationship evidence checks; per-domain policy eligibility; private Git publication, manifest/receipt journal and crash reconciliation; and a read-only UI. A synthetic fixture now uploads through MCP, extracts a source, submits and automatically accepts an evidence-backed note, then reads the committed note and exact citation. The same transport flow submits a matching quotation at the wrong location and receives an invalid result with EVIDENCE_MISMATCH; exact accepted-proposal retry returns the original receipt without a second commit. Artifact-store contract tests verify identical bytes deduplicate while both project grants remain readable and an ungranted project remains denied. The M0.6 and M0.7 publication, recovery and replacement checks now pass, including subprocess crash boundaries, queued-work gating, SQLite generation rollback and the shared repository contract. The complete upload-to-read restart flow and authenticated human browser reading task remain open; the disposable-slice gate is not closed.

The authenticated canonical-node read is connected to the private Git revision, with project-scoped hashed credentials, revocation, rotation and a one-megabyte response cap. Streamable HTTP MCP exposes the read and contribution workflow, including non-publishing proposal_evaluate, author-scoped proposal_status, writer-locked proposal_withdraw and authenticated secret-free diagnostics over HTTP/MCP. Diagnostics binds the running catalog hash, transport- and role-authorized operations, explicit selected scope, transfer limits and canonical/index revision state; it labels connection as authenticated while leaving workflow verification separate. Agent bearer clients also have run_create, revision-checked/idempotent run_checkpoint and owner-scoped run_status, backed by private atomic files and the canonical writer lock. Run tests cover idempotent create, changed-input conflict, checkpoint retry and stale revisions, owner isolation, credential revocation and closed runs; the live MCP test exercises create/checkpoint/status. Evaluation tests cover valid and invalid proposals and prove evaluation does not advance canonical state or persist status. Withdrawal tests verify accepted work cannot be withdrawn, withdrawal is idempotent, a withdrawn proposal cannot later be submitted, and other agents cannot evaluate, inspect or withdraw it. Recovery tests cover status after ref-before-receipt and prepublication restart; subprocess termination after ref update and before receipt write reconstructs the original receipt after fresh service construction. An unrelated proposal is held until pending reconciliation succeeds; a killed SQLite generation rebuild retains the prior complete index and later serves the new canonical revision. The shared repository contract passes against Git and an in-memory substitute, with the canonical node reader unchanged. Full upload-to-read process restart and target-platform filesystem qualification remain open. Proposal authorship is bound to the authenticated agent; every affected project, including shared policy, requires a grant; the current credential is rechecked while holding the same lock as publication and revocation. Upload sessions, staged extractions and artifact access are principal-scoped, and artifact reads also verify project provenance. A local test-registry seed command is explicit and refuses replacement unless requested. The gewu-connector stdio bridge exposes only explicitly allowlisted local files, streams their bytes internally, checkpoints upload IDs privately, resumes from server-verified offsets and keeps bytes and the bearer token out of tool results. Its restart simulation uses a mock MCP service; an additional test connects it to the real local GEWU MCP endpoint. Authenticated HTTP readers can now download an immutable archived source revision only when the canonical record grants it to the requested project; responses stream verified bytes with a digest ETag, private caching, MIME sniffing protection and sandbox CSP. Native-host installation, complete process-kill qualification, public remote TLS service qualification and client workflows remain open. M1.4 now has a preliminary rebuildable SQLite FTS5 index, authenticated scoped lexical search with query/scope/index-revision-bound continuation cursors, typed-edge neighbor reads, source metadata reads and bounded node history over HTTP/MCP. Reads disclose canonical/index revisions, preserve exact citations, apply freshness warnings and budget complete responses. Search pagination retains unreturned results when response budgeting trims a page and returns a conflict when the indexed snapshot changes. Neighbor source metadata and standalone source reads require the source ingested_for grant to intersect the authorized project; node history reads validated versions from the canonical Git path. Nodes, edges, source metadata and normalized source-grant rows rebuild transactionally under SQLite schema version 4. Neighbor pagination also binds query/scope/page size/index revision, filters unauthorized endpoints in SQL before applying offsets, and expires with 409 when the indexed snapshot changes. The authenticated API returns bounded original and normalized extraction slices with source offset maps, and the web reader opens those cited slices and links the archived revision. Frontend tests cover the interaction; viewport/accessibility qualification remains open. Synthetic SQLite adapter benchmarks now report scoped warm-search p95 below 300 ms at both 10k and 100k total nodes, and index rebuild below ten minutes at 100k; real Git I/O, end-to-end service overhead, target hardware and workload, and retrieval quality remain unqualified. Language evaluation and full performance acceptance remain open. The codepoint counting profile is provisional and is not a model tokenizer. A cross-service writer-lock test now proves a concurrent publication receives BUSY; a comprehensive second-writer and shared-replacement contract suite remains open. The 20 real representative sources and labeled questions have not been supplied. The web reader now connects with an in-memory viewer credential, discovers authorized projects, performs URL-scoped lexical search, opens accepted notes with exact citations, source metadata and node history, and exposes no knowledge-write controls. Human browser sessions now exchange a human-only bearer credential for a short-lived HttpOnly, SameSite=Strict cookie with a double-submit CSRF token; agent credentials cannot create sessions, and browser-session principals cannot submit agent proposals. The policy-exception owner flow is implemented; named pilot ownership and protected merge authorization remain open. The T3 preview loaded the development page, but snapshot automation failed repeatedly; visual protocol screenshots, viewport/zoom checks and keyboard/screen-reader review remain open. Python 3.14 is exercised; Python 3.12 and target operating systems are not qualified. No real ownership or production acceptance policy is configured.

M3.1–M3.2 have a working read-only search and citation reader. The current interface selects an authorized project, performs scoped lexical search, opens canonical notes, displays exact extracted citations, source revisions, history, typed relationship conditions and rationale, and the recorded claim assessment method/status/rationale. It keeps accepted relationships labeled separately from on-demand lexical related suggestions and exposes no knowledge mutation controls. Search includes authorized server-side node-type, past-freshness, confidence, as-of date range and exact tag/audience/use/domain filters; project, type, freshness, confidence, dates, query, facets and selected-node/revision values are reflected in the URL. Direct query/filter URLs restore after authentication, and browser back/forward restores previous searches and reruns them within the selected scope. Editing the search form creates one draft history entry, subsequent edits replace it, and submission does not create duplicate entries. Opening an item records its node ID and owning project; a shared current-node link restores current accepted state after authentication, and a revision selected from history is pinned in the URL and restored by browser back/forward. The facet groups apply before pagination, and continuation cursors bind their values. An empty query is accepted only with a knowledge-type filter, enabling a bounded “Browse problems” view while rejecting untyped browse requests. The typed browse path uses the same authorized scope, budget and cursor contract as lexical search. A paginated source browser now lists indexed records only when their provenance grants intersect the caller's selected project/shared scope, with archive, reliability, publisher, license, revision and authorized download details. Stale knowledge has a direct navigation shortcut into the URL-backed freshness filter; Problems and Topics shortcuts select the corresponding URL-backed node-type filters. Relationship panels state that excluded-project names and counts are not disclosed. Problem detail now labels accepted links as approaches and related knowledge, distinguishes the absence of visible accepted approaches from suggestions, and displays recorded tag/audience/use/domain facets. Knowledge nodes now support explicit assumption and limitation claim groups; each statement uses the same exact evidence, assessment and citation requirements as an ordinary claim, with a distinct assertion path and unique claim ID. Search, canonical reads, history and the reader carry those citations through, and the UI labels the groups separately. Revisions exposed by the bounded history response can be selected through an authorized revision-scoped read; pinned views omit current-index relationships and explain this limit. M3.3 remains partial: cross-project relationships are safely omitted rather than explained individually, and visual/accessibility acceptance is open. Frontend tests cover shared query and selected-node/revision URL restoration, search and node back/forward, and synthetic source-list, facets, problem detail, typed assumption/limitation rendering and citation paths plus distinct initial, empty-result and request-failure states. At 320×800, T3 preview evaluation of the disconnected screen confirms no horizontal document overflow; the navigation strip scrolls horizontally within its own row. T3 snapshot automation fails, so the authenticated view has no screenshot, 200% zoom, screen-reader, axe or real two-minute reading-task evidence.

M0.1 foundation verification now includes a two-test Playwright Chromium suite and a GitHub Actions workflow for push/pull-request quality gates. The suite exercises authenticated read-only UI behavior against a routed synthetic API and checks the narrow-screen citation reader; it is not live-backend or accessibility acceptance. M0.1's local acceptance criteria pass. The hosted workflow and its Python 3.12/Node 22 matrix have not yet been observed. M0.2 now exports the composed HTTP API's OpenAPI schema, includes canonical record schemas, generates TypeScript API types, and checks actual HTTP route IDs against the explicit HTTP operation allowlist; the reader uses generated request/response types. The full PRD contract audit and final budget-counting profile remain open. M0.3's two-project/shared fixture now checks that every affected project's auto-accept policy permits a source shared across project grants; a restrictive policy blocks automatic acceptance. Wrong-location, unsupported-assessment, stale-assessment, missing-policy and scope-escape cases have explicit assertions. Real pilot sources and labeled expected-answer questions remain missing. M0.4's local upload/extraction checks now explicitly reject bad chunk and artifact digests, prove interrupted data cannot be extracted, and retain byte deduplication grants and location maps; the authenticated MCP slice proves the same protocol boundary. Cross-platform and real-source format qualification remain open. M0.5 now rejects policies that attempt to waive required support assessment; tests also prove stale assessments cannot be reused, invalid submissions remain staged, and routine fixture work publishes under policy. Its local acceptance checks pass; semantic assessor quality remains a separate pilot gate.

M1.1 now has actual MCP denial coverage for viewer proposal/upload calls and a publication-time SQLite revocation test that stops a key revoked during validation before any canonical commit. Owner account provisioning, Argon2id password storage, encrypted TOTP seeds, replay protection, account-level failure throttling, explicit grants, revocation, challenge-bound five-minute sessions and the HTTPS/loopback-only authentication endpoint are implemented. Policy-only proposal holds now create exact-diff owner challenges; the isolated owner MCP can review and decide, and authorized resubmission records decision evidence in the acceptance manifest. Reset/recovery, OS-level owner isolation, first-owner bootstrap and named pilot owners remain open. No owner accounts are provisioned, so M1.1 stays partial.

M2.1 — local GEWU pack binding (partial)

Pack version 0.14.0 now pins an MCP contract snapshot generated from MCPServer.list_tools(), plus the generated operation catalog and record schemas. It includes bounded authenticated pasted-text capture, transcript cue locators and source capture-method metadata. Optional include_projects is exposed in HTTP and MCP search scope; end-to-end tests prove omitted includes resolve project defaults, explicit empty suppresses them, authorization covers expanded scope, and direct node/source reads plus neighbor and source-list results obey scope. The pack contract snapshot is regenerated for these additive request and record fields. A contract test rebuilds a temporary service and compares discovered tool input schemas against the bundled snapshot. The pack includes a hash/snapshot verifier and local atomic checkpoint tool recording run identity, scope, lens, coverage reasons, pending operation digests, proposal revisions, receipts and output hashes; resume rejects module drift. The service now exposes private agent-owned run create/checkpoint/status operations, and the contribution protocol describes their revision/idempotency recovery contract. Proposal validation rejects node or assessment attribution that differs from the authenticated proposer. These checks cover local contract consistency and synthetic behavior only. Codex/Claude/Antigravity/OpenCode installations, real pilot workflows, native host resume and OS matrix evidence remain NOT RUN; MCP now exposes the project lens to principals with project read permission, without exposing the autonomy policy.

Delivery boundaries

Delivery Included Completion gate
Disposable local slice Two fixture projects plus shared; one client; text/Markdown; test policy; source upload, note proposal, exact evidence/support checks, automatic acceptance, canonical read and browser citation One complete valid flow, an invalid-location rejection, shared-source reuse, exact retry and restart recovery. No production-data or policy claim.
Evaluated pilot Selected real sources/client/format; actual roles and separate agent identities; owner-approved policy; installed GEWU pack; scoped search and problem/source reading; quality sampling Useful cited answers with agents offline, recorded semantic quality, enforced contribution boundaries and all pilot gates passed.
Full v1 Every PRD P0 feature, hybrid retrieval, required inputs, four harnesses, declared Windows/macOS/Linux coverage, offline operation, protected decisions, recovery and local-to-VPS handover Complete P0 and non-functional evidence, independent restore, measured capacity/retrieval and native-client workflows on the target deployment.
Follow-up Selected M4/M5 P1 features Separate feature gates; these do not delay P0 scope, source retention, authority or recovery.

Use the proposed Python/FastAPI/Pydantic service, Git canonical records, private artifacts, derived SQLite index, stdio MCP connector and React/TypeScript reading app. Resolve and lock dependencies during scaffolding. Evaluate the embedding model and vector adapter before treating their candidates as committed choices. No new stack research or package-version claims are part of this plan.

Decisions and readiness

Record outcomes in task notes or an ADR when they change a contract, authority boundary, persistence or deployment. A pending production decision does not prevent isolated fixture work; it does prevent claiming the affected pilot/release gate passed.

Decision Proposed starting point / required input Needed before
Pilot client and platform Use Codex on the available Linux development host for disposable tests. Owner selects the real pilot pair; Windows/Codex remains an architecture recommendation. Native pilot acceptance
Pilot content Text/Markdown for the disposable slice; collect 20 representative sources across both domains and labeled questions. Finalize project IDs; dome-performance and interactive-video are suggestions. Include the deep-research report when supplied, labeled as an AI-synthesized secondary source. Real ingestion and quality evaluation
Ownership and bootstrap Name the first admin, project owners and shared-library owner. Use separate operator bootstrap and human owner MCP authorization. Pilot account/policy provisioning
Automatic acceptance Fixture policy initially permits additive shared sources and project factual notes with valid evidence/support. Owner must enumerate production rules for sources, notes, datapoints, entities, edges and interpretations in each affected domain. Missing rules hold changes. Pilot automatic acceptance
Bundle and read behavior Adopt the acceptance-contract recommendation: atomic bundles under all affected policies, conservative base-commit conflicts, and canonical accepted-revision reads during index lag. M0 persistence contract
Quality sampling Name reviewer, sample size/cadence, semantic-error limits and correction/escalation response. Same-agent extraction/assessment is already permitted for routine notes. Pilot evaluation
Retrieval profile Evaluate local embeddings on the labeled languages/corpus; fix tokenizer, vector packaging, graph/candidate caps and index generation metadata. M1 hybrid acceptance
Operating envelope Specify concurrent users/agents, upload/archive limits, retention/rights, VPS resources, backup owner, recovery-point and recovery-time objectives, benchmark hardware and percentile. Hosted pilot and full-v1 measurements
Intake and release coverage Define the initial NotebookLM export path and supported representations, exact OS/harness pairs, and any P1 launch requirements. Start with manual supplied exports unless another path is selected. Format/client qualification and release scope

Execution order

Keep the PRD's M0–M5 names. Deliver the disposable flow and pilot through a narrow pass across M0–M3, then broaden M1–M3 for v1. Each row below is a bounded work package; split it into reviewable changes around observable behaviors rather than implementing an entire layer at once.

M0 — Contracts, fixtures and acceptance foundation

Task Owning module and deliverable Depends on Acceptance evidence
M0.1 Scaffold a runnable foundation Bootstrap/development: Python package and web workspace, lockfiles, isolated data-root configuration, CI scripts and one health path. Follow the architecture's proposed layout; instantiate only needed modules. None Recorded runtime versions; backend lint/type/test and frontend lint/type/unit/browser scripts run; import-boundary checks exclude transport/provider imports from domain code.
M0.2 Publish shared contracts Domain/catalog: strict versioned records, node/edge schemas, project registry/lenses, scope, proposal lifecycle, errors, upload/run limits and whole-response budgets. Generate JSON schemas, read API types and explicit HTTP/MCP allowlists. M0.1; bundle/read decision Valid/invalid golden fixtures; unknown fields/unsafe IDs rejected; no mutation or owner operation generated for the read API.
M0.3 Create acceptance fixtures Domain/evidence: two projects plus shared, sources and expected claims/relationships, valid and adversarial locators, labeled unsupported assertions and stale dependencies. Keep illustrative PRD examples separate from executable fixtures. M0.2 Wrong-location quote, correct quote with unsupported assertion, stale assessment, missing policy, cross-policy bundle and scope-escape cases have explicit expected outcomes. Semantic cases evaluate assessor quality separately from deterministic checks.
M0.4 Implement immutable intake Artifact/extraction stores: resumable begin/chunk/finalize, source manifests, hash dedupe without lost provenance, text/Markdown location maps, safe serialization and immutable extraction artifacts. M0.2–M0.3 Matching chunk retries succeed; changed chunk/incorrect hash fail; interrupted uploads are uncitable; two projects reuse one artifact; unavailable extraction remains explicit.
M0.5 Implement proposal validation Validator/policy: schema, scope, links, quote-at-location, assertion-bound assessments, findings, policy-domain evaluation and revision invalidation. M0.3–M0.4 Mandatory checks cannot be waived by policy; changed assertions cannot reuse assessments; failing factual claims remain staged; routine fixture proposals become eligible without human approval.
M0.6 Implement durable publication Repository/workflow: staged revisions, journal, exclusive writer, final authority/base/policy rechecks, private Git index, expected-head publication, committed acceptance manifest and immutable receipt reconciliation. M0.5; identity port One commit per accepted revision; changed-payload retry conflicts; concurrent writes serialize; unrelated files are excluded; committed-tree reads work before indexing.
M0.7 Prove recovery and replacement boundaries Recovery/module contracts: inject crashes/write failures around proposal, artifact, ref, receipt and index boundaries; block new publication while reconciliation is incomplete. M0.6 Ref-before-receipt retry reconstructs the original receipt without another commit; live retries and unrelated queued work respect the recovery gate; a second writer is excluded. Subprocess termination immediately after ref update and before receipt write survives fresh repository/workflow adapter construction; an empty derived index rebuilds from the recovered canonical commit and returns its citations. The shared repository contract passes against Git and an independent in-memory adapter, and the node reader works against the substitute unchanged. A bad pending-manifest digest blocks an unrelated proposal before it is staged; after reconciliation is repaired, the original receipt is recovered and the queued proposal succeeds. The index transaction is killed during rebuild over an existing generation; SQLite retains that generation, then a normal search rebuilds and serves the complete current canonical set.

M0 exits with executable contracts and a disposable accepted source/note that can be read at its committed revision. Real-source semantic evaluation and all 20-source M0 fixtures remain visible until supplied and checked; synthetic fixtures alone cannot close that PRD requirement.

M1 — Authenticated service, MCP and retrieval

Task Owning module and deliverable Depends on Acceptance evidence
M1.1 Establish identities and authority Identity/control: admin/editor/viewer, stable agents and hashed revocable keys, owner grants, browser sessions, restricted bootstrap and separate owner client/challenges. M0.2; ownership decisions for pilot Viewer proposals denied; browser admin still cannot author; rotation retains attribution; revoked key fails at publication; agent cannot impersonate assessor/owner. Protected authorization is digest-bound and expires; replay/change/grant-revocation and recovery cases pass.
M1.2 Expose one application core MCP/read HTTP: catalog-backed reads, upload/proposal/status/revision/withdraw/evaluate and run operations, bounded errors/diagnostics. M0.4–M0.7, M1.1 Actual transports enforce capabilities and scope; no generic HTTP/CLI knowledge writes or raw queries; accepted reads disclose canonical/index revisions; diagnostics contain no secrets.
M1.3 Bridge selected local files Connector: stdio MCP to service, explicit-file byte transfer, bounded upload resume, service identity checks and protocol-only stdout. M1.2 Selected file reaches service by bytes/hash, including a remote service; arbitrary VPS-local path interpretation fails; byte payloads and credentials stay outside model-visible responses.
M1.4 Deliver scoped lexical reading Index/query: FTS, typed edges, project/shared filters and default_includes resolution, canonical revision reads, neighbors/history, read-time freshness and synthesis dependency warnings, complete citation groups and tokenizer accounting. M0.7, M1.2 IDs/redirects/graph/citations obey scope; omitted project includes resolve defaults plus shared for one project context, explicit empty includes suppress defaults, and every expanded project is authorized; stale warnings survive budgets; too-small budget returns a defined error; index deletion/rebuild preserves authority and citations; index lag is disclosed.
M1.5 Qualify hybrid retrieval Embedding/vector/ranking adapters: install the pinned intfloat/multilingual-e5-small@614241f622f53c4eeff9890bdc4f31cfecc418b3 candidate and retrieval extra on the GEWU host; activate with GEWU_EMBEDDING_MODEL_PATH and verified GEWU_EMBEDDING_MODEL_SHA256; scoped keyword/vector fusion; capped graph expansion, language fixtures and isolated index generations. M1.4; retrieval/benchmark decisions The MSI/local GEWU deployment can load the model from a verified local path using its artifact digest and environment settings, then run offline. Source vectors record model/profile metadata and can be rebuilt or replaced without changing canonical records. Top-five source recall meets the PRD target on labeled questions; old citations survive model/parser changes; keyword degradation is labeled. Real provider swap and generation rollback pass consumer contracts without search-consumer changes.

M1.1–M1.4 support the pilot. M1.5 and all four native client flows are required to close full M1/v1. Authentication and enforced data isolation precede any non-disposable shared use.

M2 — Local agent integration and input coverage

Task Owning module and deliverable Depends on Acceptance evidence
M2.1 Bind the existing GEWU pack Agent profile/skills/pipelines: use real tool/schema contracts, pin pack/module hashes, record scope/lens, source coverage, skips, proposal IDs and receipts in durable runs. Update the separate pack submodule deliberately. M1.2–M1.4 Installed pack completes ingest/query, attributes same-agent assessments, reports unavailable P1 capabilities, and resumes an uncertain outcome through service status without regeneration or duplicate acceptance.
M2.2 Build additive installation Harness adapters: preserve existing personas, skills and MCP entries; dedicated/integrated workspace routing; thin discovery pointers including needed GEMINI routing; copy fallback and owned-entry rollback. M2.1, M1.3 Install/uninstall leaves unrelated configuration intact; discovery, connection and workflow verification recorded separately; newer pack installation cannot silently alter a pinned active run.
M2.3 Expand source intake Fetch/parser adapters: URL/HTML, PDF, supported local files, pasted text and supplied NotebookLM exports; transcript locators where applicable; archived downloads, source identity/revision changes, reference-only failure records. M0.4, M1.3; format limits Exact locations verified on real fixtures; changed URL creates a revision; failed capture preserves prior bytes; scanned/encrypted/unsupported material returns honest status. Parser replacement retains old extraction resolution. Required input paths are qualified; unsupported document subtypes are listed explicitly.
M2.4 Complete linking and consistency Entity/relationship application services plus ingest skills: canonical aliases, duplicate candidates, reversible merge history, evidence/rationale-bound typed links, contradictions and supersession. M0.5–M0.6, M2.1 Shared dedupe preserves provenance; substantive links include conditions/support; similarity does not create accepted edges; destructive merges require authority; contradictions retain both claims.
M2.5 Qualify native hosts and quality Acceptance/client fixtures: Codex, Claude, Antigravity and OpenCode on declared supported OS pairs; five common documents through two agents; sampling and labeled coverage/error evaluation. M2.1–M2.4; quality/coverage decisions Each supported pair transfers, rejects invalid evidence, auto-accepts eligible work, retries/resumes and retrieves citations. Convergence compares findings/support errors; no pass based on matching prose or tool listing.
M2.6 Control project facet vocabularies Ingest and taxonomy: project-scoped canonical labels for tags, audience, use and domain; bounded agent proposals; exact/alias, lexical and embedding-based shortlist retrieval; persist label vectors in the derived SQLite index keyed to canonical label ID and embedding profile; Jev fit and reuse-versus-new decisions; review feedback and alias/merge history. M1.5, M2.1, M2.4 Server-enforced proposal limits apply per facet. For each proposal, candidate retrieval stays within authorized project/shared scope and supplies the closest existing labels to Jev alongside a new-label option. Label vectors remain linked to their canonical project labels and can be rebuilt when labels or model profiles change. Decisions distinguish label fit from reuse, and uncertain cases remain reviewable. Existing matches resolve to canonical labels; approved, sufficiently distinct new labels enter the project vocabulary and are reused on later records. Search through canonical names or aliases reaches the same records. Thresholds are tuned on labeled duplicate/near-duplicate/distinct-label examples; adding or merging a label preserves provenance and is reversible.

For the pilot, complete M2.1–M2.2, M2.6, and the selected-format/client portions of M2.3–M2.5. Remaining matrix cells and source paths stay NOT RUN or BLOCKED until verified; they remain full-v1 obligations.

M2.6 implementation sequence: controlled project labels

The goal is to keep facet vocabulary reusable without preventing a project from adding a genuinely useful concept. Labels are scoped by project and facet (tags, audience, use_for, domain). Similarity retrieval creates a shortlist; Jev judges fit and reuse; GEWU applies the decision policy. A similarity score alone never merges labels or authorizes a new one.

Step Work Acceptance evidence
1. Install and activate local embeddings On the GEWU host, install the retrieval extra and the pinned intfloat/multilingual-e5-small@614241f622f53c4eeff9890bdc4f31cfecc418b3 model once. Configure GEWU_EMBEDDING_MODEL_PATH and GEWU_EMBEDDING_MODEL_SHA256 in the service environment; keep the artifact digest check, CPU/local-only loading and offline operation. Do not put model installation on the Antigravity/client agent. Service startup verifies the model digest and loads it; an embedding probe verifies its dimensions. With the local model installed, label and source embedding generation works offline. Unset settings retain lexical mode; incomplete or invalid settings fail clearly. The service never downloads weights at runtime.
2. Define the project label catalog and persistence Add stable label IDs and project-scoped catalog entries with facet, canonical name, short definition, aliases, lifecycle status (active, merged, retired), provenance and timestamps. Keep the canonical catalog in GEWU's authoritative project data. Mirror label embeddings into a derived SQLite table keyed by label ID and embedding profile (model ID, artifact digest, dimensions and text digest); the catalog itself is stored with canonical project records, not duplicated as mutable SQLite state. Rebuild vectors from canonical data; never treat a vector as the label's source of truth. Labels and vectors survive restart. Index rebuilding recreates vectors from canonical project data. A model/profile change cannot mix vectors from different profiles. Reads, proposals and shortlist queries enforce project/shared authorization.
3. Inventory and migrate current free-text facets Scan existing knowledge for tags, audiences, uses and domains. Normalize exact duplicates conservatively (Unicode NFC, case and whitespace); preserve original strings as aliases and retain usage counts/provenance. Produce likely semantic duplicate groups for review, but do not automatically merge merely related terms. Every current facet value maps to a catalog entry without losing node provenance or searchability. Exact duplicate normalization is auditable. Semantic near-duplicates are presented as candidates for owner review before merge.
4. Retrieve closest existing labels For each agent proposal, search only the authorized project/facet catalog. Resolve exact names and aliases first, then combine lexical retrieval with cosine similarity over label name + definition + aliases. Embed the proposal's label + short meaning/context and return a bounded top-K shortlist to Jev. Start with a small K (for example, five); tune candidate recall on human-labeled synonym, near-neighbor and distinct-label pairs. Treat vector similarity as candidate retrieval, not equivalence probability. If embeddings are unavailable, lexical matches may be shown, but new-label automation stays off. The correct existing synonym appears in the shortlist for the labeled evaluation set, including English/Chinese fixtures. Similar-but-distinct labels remain available as separate options. Shortlists never disclose labels outside the caller's allowed scope.
5. Update ingest skills and contracts In kb-ingest, fetch the project's active vocabulary before classifying. Prefer canonical labels; propose a new label only when the source supports a reusable distinction, with a short definition. Enforce at most five tags and three labels in each other facet (14 total) in both skill and MCP contract. Send candidate labels with the existing source-bound review flag. Agent pack/schema tells the agent how to read the catalog and submit bounded, definition-bearing proposals. Agent prose cannot mutate the catalog directly.
6. Extend the Jev call In the same Decisions API request, ask a fit noul for each candidate and a choice among that facet's retrieved canonical labels plus a new label option. Include short definitions so terms are compared by meaning, not string resemblance. Parse and persist per-option probabilities, selected option and choice confidence alongside existing relevance/lens/facet-fit scores. Jev selects the proposed text or an existing option; it does not generate label text. One request returns typed, bounded, validated judgments. Invalid/missing answers leave the review flag stored without Jev data. Persisted judgments identify the candidate/options and model so later policy/threshold changes can be audited.
7. Apply conservative decisions and reviewer feedback Current policy auto-adds only when Jev selects the new-label option, the proposal ranks in the configured top N for its facet, fit and new-label probabilities each meet their threshold, and choice confidence clears its threshold. Defaults: tags top 5; audience/use_for/domain top 3 each; fit >= 0.90, new >= 0.90 and confidence >= 0.80; local embeddings and Jev are both required. Jev selects from agent-proposed labels and retrieved project labels; new_label means the proposal text is accepted, not that Jev creates new wording. Persist the decision on the ingest-review row and the new label, Jev scores and review provenance in the canonical project catalog. Existing-label matches are recorded as reuse decisions. Human feedback can guide future ingest, not model weights. Decisions and label additions are attributable; the catalog appears in later project context. Tune thresholds against labeled false-new and false-merge outcomes before lowering them. Ambiguous/unsupported cases remain in the review queue.
8. Make search resolve canonical labels Store canonical label IDs on accepted nodes (or a migration-safe representation that resolves to IDs). Exact searches for canonical names and aliases resolve through the catalog to the same ID set. Keep label vectors and search aliases synchronized with canonical revision and active embedding profile. Searching/filtering by a canonical label or any approved alias returns the same authorized records; changing a label definition or model profile triggers a safe index rebuild without changing citations or canonical knowledge.
9. Pilot and tune automation Run the configured policy over the Dome documents. Review match/new-label decisions, duplicate proposals, unique-label count, one-record labels, and reviewer corrections by facet. Tune caps, retrieval K and thresholds on these judgments; threshold env vars can tighten or relax promotion. Pilot report includes per-facet false-new and false-merge rates, shortlist recall, one-off label rate, review burden, latency and offline behavior. Current defaults are conservative initial settings, not measured acceptance claims.

Rollout boundary: existing node facets continue to read during migration. Catalog resolution is introduced compatibly; bulk canonical rewrites and merges occur only after a reviewed migration map exists. Canonical project data remains authoritative, while SQLite catalog/vector tables are derived and safely rebuildable.

M3 — Read-only application and pilot evaluation

Task Owning module and deliverable Depends on Acceptance evidence
M3.1 Build reading primitives Design system/app shell: editorial tokens, typography, scope selector, result/evidence/relationship rows, freshness and system states; generated read-client types. M0.2, M1.2; UI fixtures may start earlier Real rendered review under the visual protocol; keyboard navigation and error/empty/stale/index-lag states; semantic tokens replace feature-local styling.
M3.2 Connect search and citations Search/node/source/history features: URL-owned scope/filters/query/selected node/revision, archived revision download/location preview, typed neighbors and citing notes. M1.4, M3.1 Scope → search → note → quote → exact archived revision works with agents stopped; direct query/filter URLs restore after authentication, selected node and history revision deep links and browser back/forward restore the reader; typing query/facets does not submit until Search; obsolete responses do not replace current scope; logout clears private caches; safe Markdown/preview rendering. History is bounded to the latest 20 revisions; real two-minute tasks and visual/accessibility qualification remain open.
M3.3 Deliver problem/topic reading Problem/topic/staleness features: approaches, assumptions, cases, source references, facets and separately labeled related suggestions. M2.4, M3.2 A viewer explores multiple approaches and their conditions; excluded projects remain explicit; no capture/edit/accept/generate controls or hidden mutation client. Search supports URL-owned exact tag/audience/use/domain facets, applied in SQLite before pagination and bound into cursors. “Browse problems” lists scoped problem records without a text query, with type-bound pagination; cross-project explanations and visual/accessibility acceptance remain open. Evidence panel replacement preserves public API/navigation.
M3.4 Evaluate the pilot Acceptance/quality: real two-project corpus, shared-source reuse, viewer/editor/admin and two agent identities, retry/restart, quality sampling and timed reading tasks. Pilot portions of M0–M3; production pilot decisions Person finds a cited answer within two minutes; location validity and assessment coverage meet PRD requirements; semantic errors and reviewer effort recorded against agreed limits. All missing full-v1 features remain listed.

Inspect the primary path at the visual protocol's desktop/mobile sizes, 200% zoom and 320-pixel reflow. Combine automated accessibility checks with keyboard and screen-reader evidence. The pilot requires enough real UI to test research usefulness; a service-only demonstration cannot close M3.4.

Full-v1 hardening and local-to-VPS release

This gate spans M0–M3; it is not a new product milestone. Finish all P0 coverage before labeling a release v1.

Task Deliverable Required evidence
R1 Enforce deployment boundaries Restricted service identity, private data/credential mounts, parser sandbox/resource limits, bounded queues/uploads, authenticated artifacts, URL/DNS/redirect restrictions and MCP admission controls. Traversal/symlink escape, malicious source instructions, secret-read, private-network fetch, unsafe preview and denied-capability cases fail through actual boundaries. Separate-process existence alone does not prove sandboxing.
R2 Exercise offline and portable operation Native Windows/macOS/Linux paths, safe storage keys, non-symlink installation, local model/tokenizer/font assets and parser/vector packaging. Existing-source validation, ingestion, index/rebuild and reading work offline; configured local-model workflow demonstrated where claimed; new external fetching reports unavailable. Declared host/OS matrix passes.
R3 Measure product and capacity targets Fixed workload/corpus/hardware, latency percentiles, retrieval quality, queue saturation and cold/warm rebuild profiles. PRD search target under 300 ms at 10,000 project nodes; 100,000-node scale and under-ten-minute rebuild evaluated with stated cache assumptions; ≥80% top-five source recall on 20–30 questions per project; acceptance rate tracked toward ≥70% after tuning. Exceptions remain visible, not redefined as passes.
R4 Prove backup, restore and migrations Consistent canonical/artifact/workflow/control backups; protected owner-key recovery; schema/index migrations and rollback procedures. Independent restore verifies hashes, grants, manifests, runs, receipts and citation reads, then disposable acceptance. Index deletion loses no authority. Revoked access is not revived; owner-key recovery and missing-key reset paths tested; recovery objectives measured.
R5 Transfer authority and qualify VPS Locked release images, Compose/Caddy, private volumes, freeze/transfer/verify local state, rebuild, client endpoint switch and controlled rollback/handback. One writer only; authenticated HTTPS/browser reading and native remote upload/accept/retry/resume/retrieval evidence; restored test copy cannot become a second authority; local and deployed revisions recorded separately.
R6 Close release coverage P0 traceability and acceptance ledger, runbooks, limitations and first-month reuse follow-up. No required gate is FAIL, BLOCKED or NOT RUN. Record shared-node reuse during the first month rather than claiming it passed on release day.

M4 and M5 — Selected follow-up

M4 adds P1 analysis/task briefs, synthesis compilation/refresh, deterministic evidence bundles or saved briefs, curator reports, audience lenses, matrices, comparisons, timelines and graph exploration. Decide brief semantics and release priority before implementation. Existing P0 synthesis dependency warnings and source/relationship reading are already delivered in M1/M3.

M5 adds P1 shared promotion and project export/import with redirects, provenance, policy-domain authority and durable receipts. Imports enter through MCP proposals. Exercise three projects for this later milestone; the confirmed pilot has two. P0 cross-project scope and dedupe are already enforced in M0–M2. Keep project confidentiality, automatic hooks, clustering and slide export under their existing P2 priorities.

P0 traceability

This maps implementation ownership; the release ledger must still record each requirement separately, including the PRD's non-tabular evidence, lifecycle and source-retention rules.

PRD requirement Work packages
FR-P1, FR-P2, FR-P5 M0.2–M0.3, M1.2, M1.4, M2.1, M3.2
FR-P3 M0.4, M2.4
FR-P8 M1.1–M1.2, M0.6–M0.7, R1/R4
FR-I1, FR-I2, FR-I9 M0.4, M1.3, M2.3
FR-I3, FR-I4, FR-I7 M0.3–M0.5, M2.1, M2.5
FR-I6 M2.4
FR-I8, FR-I11 M0.6–M0.7, M1.2, M2.1–M2.2
FR-V1, FR-V2, FR-V3 M0.5, M1.1–M1.2, M3.4
FR-V4 M2.4, M1.4, M3.2
FR-V5 M0.6–M0.7, M1.1–M1.2
FR-A1, FR-A9, FR-A10 M0.2, M1.2–M1.4, M2.2, M2.5
FR-A2 M1.5
FR-A3 M0.2, M1.4–M1.5
FR-A5 M0.6, M1.1–M1.2, R1/R5
FR-A6, FR-A11 M2.1–M2.2, M2.5
FR-C3; displayed synthesis warnings (§6.3.2) M1.4, M3.2–M3.3
FR-U1, FR-U2, FR-U3, FR-U6, FR-U9 M3.1–M3.4
Non-functional requirements and success metrics M0.1–M0.7, M2.5, M3.4, R1–R6

Task completion and evidence

Each implementation task records: task ID, observable outcome, requirement IDs, owning module/public contract, dependencies, failure cases, state migration/rollback where applicable, checks and evidence. Apply the development protocol and modularity protocol; do not duplicate their contracts in every task.

The acceptance ledger tracks each M0–M3 and R1–R6 gate, its current status, evidence scope and missing proof. Detailed fixture and command results remain in dated acceptance records. Each gate must distinguish structural pack validation, installed discovery, connection, native workflow and semantic quality; no fixture-only evidence closes a pilot or release gate.

Dependencies determine sequence, not fixed calendar dates. Staffing and source/decision availability are unknown; estimate effort after M0 contracts and the first vertical flow expose parser, client and recovery costs. Review progress at the disposable-flow, pilot-evaluation and full-v1 gates, with unresolved decisions and failed evidence visible.

First implementation batch

  1. Complete M0.1 with isolated temporary data roots, actual verification scripts and initial import-boundary checks.
  2. Implement M0.2–M0.3 for a shared text source and one project factual note, plus wrong-location, stale-assessment and scope fixtures for both projects.
  3. Implement M0.4–M0.7 for that bundle under an explicitly named test policy; prove exact retry and ref-before-receipt recovery.
  4. Add the minimal M1 identity/MCP/connector/read path, bind kb-ingest/kb-query, and connect a minimal M3 citation view.
  5. Demonstrate the disposable source-to-citation flow, then finalize real pilot inputs/policy and expand the same contracts. Do not scaffold P1 workflows before this path works.