174 lines
47 KiB
Markdown
174 lines
47 KiB
Markdown
# GEWU implementation plan
|
||
|
||
13 October 2026 · Planning baseline: [PRD 0.8](PRD.md), [GOAL](GOAL.md), [architecture v0.3](architecture/ARCHITECTURE.md).
|
||
|
||
Build a complete local source-to-citation workflow first, evaluate it with the two pilot projects, then expand it to every P0 requirement before full v1. The pilot serves up to five people researching Dome interactive content and an interactive video platform/service. It may use fewer clients, formats and retrieval capabilities than v1; its results must state that coverage explicitly.
|
||
|
||
This plan guides implementation; milestone status is recorded below and does not imply a pilot or release gate has passed. The repository also contains an initial agent instruction pack in the `agents/gewu` submodule. Existing pack files need integration work, not recreation. Architecture choices remain proposed until compatibility probes and the affected decisions below are resolved.
|
||
|
||
## Implementation status (14 October 2026)
|
||
|
||
M1.2/FR-P1 now has an administrator-only project lifecycle control plane. Project IDs are lowercase immutable slugs; creation supplies a named owner and matching initial supported-assessment policy, and commits `projects/<project_id>/project.md` plus `policies/<sha256(policy_id)>.yaml` in one canonical revision. Archive writes a new revision, preserves reads/history and prevents new acceptance. HTTP routes are explicitly allowlisted outside the MCP agent catalog, enforce admin/non-agent authority, browser CSRF/origin checks and a streamed 64 KiB creation-body cap, and share the canonical writer lock with acceptance. Local contract, authorization, size, persistence and race tests pass. Owner authentication and project grants exist, but named-owner account provisioning, owner reset and first-owner bootstrap remain pilot/deployment gates; protected proposal challenges are implemented separately under M1.1. See [project lifecycle evidence](acceptance/2026-10-14-project-lifecycle.md).
|
||
|
||
M1.1 now supports proposal-triggered owner review for policy-only holds. Submission binds a deterministic challenge to the proposal revision, complete before/after canonical files, affected project owners and policy versions. A separate `/owner-mcp` server exposes challenge metadata, the complete bounded diff, and one immutable authorize/reject decision; the owner token remains challenge- and audience-bound. On exact proposal resubmission, current owner grants and decisions are rechecked under the canonical writer lock, every affected domain must be covered, and the acceptance manifest records the decision digests, domains, grant versions, rationale and timestamps. Mandatory validation findings cannot be waived. Unit and MCP integration evidence is in [owner review evidence](acceptance/2026-10-14-owner-review.md). M1.1 remains partial because first-owner bootstrap, reset/recovery, OS-level owner isolation, actual named owners, and independent backup/restore qualification are still external or unimplemented.
|
||
|
||
M2.2 is in progress: GEWU pack 0.14.0 includes a copy-based installer with integrated/dedicated routing modes, isolated pack placement, reversible marked pointers for AGENTS.md, CLAUDE.md and GEMINI.md, refusal to overwrite an existing installation, and ownership-checked uninstall that preserves unrelated and later user-added files. Nine isolated filesystem tests pass, including fail-closed rejection of traversal paths, unrecognized routes in edited install state and a symlinked .agents directory. Local run checkpoints pin module hashes and reject resume when a newer verified pack snapshot changes, preserving the checkpoint for explicit revalidation. The installer deliberately leaves MCP host configuration untouched because endpoint and credential provisioning are operator-specific. Native discovery, connection, workflow verification and update/resume behavior in native clients remain NOT RUN; this partial implementation does not close M2.2.
|
||
|
||
M2.3 has authenticated `source_capture_url` and `source_capture_text` MCP operations. URL capture is gated by an exact operator-configured host allowlist; the fetch adapter rejects non-public DNS answers, pins connections to validated addresses, revalidates redirects, limits response types, bytes, duration and concurrency, and removes credentials, query strings and fragments from returned URLs. Pasted-text capture accepts UTF-8 plain text or Markdown up to 1 MB, requires a stable capture ID and source key, checks the authenticated project's submission grant, stores bytes in the private content-addressed artifact store, extracts the exact text and returns a proposal-ready source candidate with `capture_method=pasted-text`; the candidate is not registered until a typed proposal is submitted. A private disk journal binds the capture ID to the principal, project and full input digest, returns the original candidate on exact retry, conflicts on changed inputs and is bounded to 1,024 receipts. Canonical source records now distinguish URL, local-file, transcript, declared NotebookLM-export and pasted-text capture; selected VTT/SRT files retain cue timestamps and transcript provenance. URL and selected-file intake accept PDF alongside UTF-8 text/Markdown/HTML. The local connector returns a path-free source metadata candidate, derives stable file identity and content-digest revision IDs when the caller omits one, and checkpoints completed metadata so uncertain retries reproduce the same candidate; transfer state is private and bounded to 1,024 entries. Linux PDF extraction uses Poppler inside bubblewrap with network and host-user files isolated, an empty environment, read-only system mounts, and CPU, memory, time, page and output limits; tests verify page locators and inability to read a host temporary file. Mixed text and blank pages return a partial extraction; image-only/scanned PDFs return an explicit unavailable `PDF_NO_TEXT` result, while malformed, encrypted, oversized and unavailable-parser cases return stable failure codes. URL captures return proposal-ready source metadata candidates with stable sanitized-URL identity and digest-bound revision IDs; failed fetches can be registered as artifact-free `capture-failed` references. Acceptance verifies artifacts only for archived records, and unarchived records remain ineligible as factual evidence. MCP and connector fixture coverage verifies revision changes, proposal registration, authorization, resumable transfer and artifact bytes, including a local archived candidate when PDF extraction is unavailable. A canonical-registry regression test confirms that a replacement parser extraction is added under its new content-addressed ID while the previous extraction remains resolvable for historical citations (`tests/application/test_registry.py`). The local parser now accepts timestamped WebVTT and SubRip cues; proposal validation requires the cited cue times to match the extraction block, and the connector tags those files as transcripts. These synthetic cue fixtures and contracts are covered by tests. Still open: supplied NotebookLM export fixtures, parser support on Windows/macOS, real-source location verification and pilot source-path qualification.
|
||
|
||
M2.4 is in progress. Entity aliases are normalized to NFC with collapsed whitespace and checked for empty, repeated and title-duplicate values. Proposal validation compares exact entity names and aliases within the candidate's project and across only the caller-readable projects in the proposal's explicit scope. Scope is enforced both by the authenticated MCP adapter and again in the validator; evaluation does not disclose matches from unreadable or out-of-scope domains. Matches become generic `POSSIBLE_DUPLICATE_ENTITY` review findings and cannot auto-accept. Publication rechecks the credential's role and project-grant snapshot under the canonical writer lock. Aliases are indexed for lexical search. Duplicate review nominates candidates without creating accepted links or merges. Policy-only exceptions now use the protected owner review and manifest binding described under M1.1; destructive merge and reversible merge history remain unimplemented, so merge operations stay unavailable. Existing typed links bind evidence to relationship conditions and rationale. A synthetic contradiction bundle is validated and published atomically with two separately cited claims retained (`tests/domain/test_validation.py`, `tests/application/test_acceptance.py`; [contradiction bundle evidence](acceptance/2026-10-14-contradiction-bundle.md)). The service stores the proposal’s contradiction interpretation; it does not independently infer semantic conflict. Reversible merge history, complete ingest workflow coverage and shared provenance qualification remain open.
|
||
|
||
M0.1–M0.7 have a backend foundation and a first protocol-level vertical flow. Present: pinned Python and web workspaces; health path; strict v1 records and generated schemas/catalog; two-project/shared fixtures; resumable private text/Markdown intake; immutable text extraction with normalization maps; deterministic claim and relationship evidence checks; per-domain policy eligibility; private Git publication, manifest/receipt journal and crash reconciliation; and a read-only UI. A synthetic fixture now uploads through MCP, extracts a source, submits and automatically accepts an evidence-backed note, then reads the committed note and exact citation. The same transport flow submits a matching quotation at the wrong location and receives an `invalid` result with `EVIDENCE_MISMATCH`; exact accepted-proposal retry returns the original receipt without a second commit. Artifact-store contract tests verify identical bytes deduplicate while both project grants remain readable and an ungranted project remains denied. The M0.6 and M0.7 publication, recovery and replacement checks now pass, including subprocess crash boundaries, queued-work gating, SQLite generation rollback and the shared repository contract. The complete upload-to-read restart flow and authenticated human browser reading task remain open; the disposable-slice gate is not closed.
|
||
|
||
The authenticated canonical-node read is connected to the private Git revision, with project-scoped hashed credentials, revocation, rotation and a one-megabyte response cap. Streamable HTTP MCP exposes the read and contribution workflow, including non-publishing `proposal_evaluate`, author-scoped `proposal_status`, writer-locked `proposal_withdraw` and authenticated secret-free `diagnostics` over HTTP/MCP. Diagnostics binds the running catalog hash, transport- and role-authorized operations, explicit selected scope, transfer limits and canonical/index revision state; it labels connection as authenticated while leaving workflow verification separate. Agent bearer clients also have `run_create`, revision-checked/idempotent `run_checkpoint` and owner-scoped `run_status`, backed by private atomic files and the canonical writer lock. Run tests cover idempotent create, changed-input conflict, checkpoint retry and stale revisions, owner isolation, credential revocation and closed runs; the live MCP test exercises create/checkpoint/status. Evaluation tests cover valid and invalid proposals and prove evaluation does not advance canonical state or persist status. Withdrawal tests verify accepted work cannot be withdrawn, withdrawal is idempotent, a withdrawn proposal cannot later be submitted, and other agents cannot evaluate, inspect or withdraw it. Recovery tests cover status after ref-before-receipt and prepublication restart; subprocess termination after ref update and before receipt write reconstructs the original receipt after fresh service construction. An unrelated proposal is held until pending reconciliation succeeds; a killed SQLite generation rebuild retains the prior complete index and later serves the new canonical revision. The shared repository contract passes against Git and an in-memory substitute, with the canonical node reader unchanged. Full upload-to-read process restart and target-platform filesystem qualification remain open. Proposal authorship is bound to the authenticated agent; every affected project, including shared policy, requires a grant; the current credential is rechecked while holding the same lock as publication and revocation. Upload sessions, staged extractions and artifact access are principal-scoped, and artifact reads also verify project provenance. A local test-registry seed command is explicit and refuses replacement unless requested. The `gewu-connector` stdio bridge exposes only explicitly allowlisted local files, streams their bytes internally, checkpoints upload IDs privately, resumes from server-verified offsets and keeps bytes and the bearer token out of tool results. Its restart simulation uses a mock MCP service; an additional test connects it to the real local GEWU MCP endpoint. Authenticated HTTP readers can now download an immutable archived source revision only when the canonical record grants it to the requested project; responses stream verified bytes with a digest ETag, private caching, MIME sniffing protection and sandbox CSP. Native-host installation, complete process-kill qualification, public remote TLS service qualification and client workflows remain open. M1.4 now has a preliminary rebuildable SQLite FTS5 index, authenticated scoped lexical search with query/scope/index-revision-bound continuation cursors, typed-edge neighbor reads, source metadata reads and bounded node history over HTTP/MCP. Reads disclose canonical/index revisions, preserve exact citations, apply freshness warnings and budget complete responses. Search pagination retains unreturned results when response budgeting trims a page and returns a conflict when the indexed snapshot changes. Neighbor source metadata and standalone source reads require the source ingested_for grant to intersect the authorized project; node history reads validated versions from the canonical Git path. Nodes, edges, source metadata and normalized source-grant rows rebuild transactionally under SQLite schema version 4. Neighbor pagination also binds query/scope/page size/index revision, filters unauthorized endpoints in SQL before applying offsets, and expires with 409 when the indexed snapshot changes. The authenticated API returns bounded original and normalized extraction slices with source offset maps, and the web reader opens those cited slices and links the archived revision. Frontend tests cover the interaction; viewport/accessibility qualification remains open. Synthetic SQLite adapter benchmarks now report scoped warm-search p95 below 300 ms at both 10k and 100k total nodes, and index rebuild below ten minutes at 100k; real Git I/O, end-to-end service overhead, target hardware and workload, and retrieval quality remain unqualified. Language evaluation and full performance acceptance remain open. The codepoint counting profile is provisional and is not a model tokenizer. A cross-service writer-lock test now proves a concurrent publication receives `BUSY`; a comprehensive second-writer and shared-replacement contract suite remains open. The 20 real representative sources and labeled questions have not been supplied. The web reader now connects with an in-memory viewer credential, discovers authorized projects, performs URL-scoped lexical search, opens accepted notes with exact citations, source metadata and node history, and exposes no knowledge-write controls. Human browser sessions now exchange a human-only bearer credential for a short-lived HttpOnly, SameSite=Strict cookie with a double-submit CSRF token; agent credentials cannot create sessions, and browser-session principals cannot submit agent proposals. The policy-exception owner flow is implemented; named pilot ownership and protected merge authorization remain open. The T3 preview loaded the development page, but snapshot automation failed repeatedly; visual protocol screenshots, viewport/zoom checks and keyboard/screen-reader review remain open. Python 3.14 is exercised; Python 3.12 and target operating systems are not qualified. No real ownership or production acceptance policy is configured.
|
||
|
||
M3.1–M3.2 have a working read-only search and citation reader. The current interface selects an authorized project, performs scoped lexical search, opens canonical notes, displays exact extracted citations, source revisions, history, typed relationship conditions and rationale, and the recorded claim assessment method/status/rationale. It keeps accepted relationships labeled separately from on-demand lexical related suggestions and exposes no knowledge mutation controls. Search includes authorized server-side node-type, past-freshness, confidence, as-of date range and exact tag/audience/use/domain filters; project, type, freshness, confidence, dates, query, facets and selected-node/revision values are reflected in the URL. Direct query/filter URLs restore after authentication, and browser back/forward restores previous searches and reruns them within the selected scope. Editing the search form creates one draft history entry, subsequent edits replace it, and submission does not create duplicate entries. Opening an item records its node ID and owning project; a shared current-node link restores current accepted state after authentication, and a revision selected from history is pinned in the URL and restored by browser back/forward. The facet groups apply before pagination, and continuation cursors bind their values. An empty query is accepted only with a knowledge-type filter, enabling a bounded “Browse problems” view while rejecting untyped browse requests. The typed browse path uses the same authorized scope, budget and cursor contract as lexical search. A paginated source browser now lists indexed records only when their provenance grants intersect the caller's selected project/shared scope, with archive, reliability, publisher, license, revision and authorized download details. Stale knowledge has a direct navigation shortcut into the URL-backed freshness filter; Problems and Topics shortcuts select the corresponding URL-backed node-type filters. Relationship panels state that excluded-project names and counts are not disclosed. Problem detail now labels accepted links as approaches and related knowledge, distinguishes the absence of visible accepted approaches from suggestions, and displays recorded tag/audience/use/domain facets. Knowledge nodes now support explicit assumption and limitation claim groups; each statement uses the same exact evidence, assessment and citation requirements as an ordinary claim, with a distinct assertion path and unique claim ID. Search, canonical reads, history and the reader carry those citations through, and the UI labels the groups separately. Revisions exposed by the bounded history response can be selected through an authorized revision-scoped read; pinned views omit current-index relationships and explain this limit. M3.3 remains partial: cross-project relationships are safely omitted rather than explained individually, and visual/accessibility acceptance is open. Frontend tests cover shared query and selected-node/revision URL restoration, search and node back/forward, and synthetic source-list, facets, problem detail, typed assumption/limitation rendering and citation paths plus distinct initial, empty-result and request-failure states. At 320×800, T3 preview evaluation of the disconnected screen confirms no horizontal document overflow; the navigation strip scrolls horizontally within its own row. T3 snapshot automation fails, so the authenticated view has no screenshot, 200% zoom, screen-reader, axe or real two-minute reading-task evidence.
|
||
|
||
M0.1 foundation verification now includes a two-test Playwright Chromium suite and a GitHub Actions workflow for push/pull-request quality gates. The suite exercises authenticated read-only UI behavior against a routed synthetic API and checks the narrow-screen citation reader; it is not live-backend or accessibility acceptance. M0.1's local acceptance criteria pass. The hosted workflow and its Python 3.12/Node 22 matrix have not yet been observed. M0.2 now exports the composed HTTP API's OpenAPI schema, includes canonical record schemas, generates TypeScript API types, and checks actual HTTP route IDs against the explicit HTTP operation allowlist; the reader uses generated request/response types. The full PRD contract audit and final budget-counting profile remain open. M0.3's two-project/shared fixture now checks that every affected project's auto-accept policy permits a source shared across project grants; a restrictive policy blocks automatic acceptance. Wrong-location, unsupported-assessment, stale-assessment, missing-policy and scope-escape cases have explicit assertions. Real pilot sources and labeled expected-answer questions remain missing. M0.4's local upload/extraction checks now explicitly reject bad chunk and artifact digests, prove interrupted data cannot be extracted, and retain byte deduplication grants and location maps; the authenticated MCP slice proves the same protocol boundary. Cross-platform and real-source format qualification remain open. M0.5 now rejects policies that attempt to waive required support assessment; tests also prove stale assessments cannot be reused, invalid submissions remain staged, and routine fixture work publishes under policy. Its local acceptance checks pass; semantic assessor quality remains a separate pilot gate.
|
||
|
||
M1.1 now has actual MCP denial coverage for viewer proposal/upload calls and a publication-time SQLite revocation test that stops a key revoked during validation before any canonical commit. Owner account provisioning, Argon2id password storage, encrypted TOTP seeds, replay protection, account-level failure throttling, explicit grants, revocation, challenge-bound five-minute sessions and the HTTPS/loopback-only authentication endpoint are implemented. Policy-only proposal holds now create exact-diff owner challenges; the isolated owner MCP can review and decide, and authorized resubmission records decision evidence in the acceptance manifest. Reset/recovery, OS-level owner isolation, first-owner bootstrap and named pilot owners remain open. No owner accounts are provisioned, so M1.1 stays partial.
|
||
|
||
### M2.1 — local GEWU pack binding (partial)
|
||
|
||
Pack version 0.14.0 now pins an MCP contract snapshot generated from `MCPServer.list_tools()`, plus the generated operation catalog and record schemas. It includes bounded authenticated pasted-text capture, transcript cue locators and source capture-method metadata. Optional `include_projects` is exposed in HTTP and MCP search scope; end-to-end tests prove omitted includes resolve project defaults, explicit empty suppresses them, authorization covers expanded scope, and direct node/source reads plus neighbor and source-list results obey scope. The pack contract snapshot is regenerated for these additive request and record fields. A contract test rebuilds a temporary service and compares discovered tool input schemas against the bundled snapshot. The pack includes a hash/snapshot verifier and local atomic checkpoint tool recording run identity, scope, lens, coverage reasons, pending operation digests, proposal revisions, receipts and output hashes; resume rejects module drift. The service now exposes private agent-owned run create/checkpoint/status operations, and the contribution protocol describes their revision/idempotency recovery contract. Proposal validation rejects node or assessment attribution that differs from the authenticated proposer. These checks cover local contract consistency and synthetic behavior only. Codex/Claude/Antigravity/OpenCode installations, real pilot workflows, native host resume and OS matrix evidence remain NOT RUN; MCP now exposes the project lens to principals with project read permission, without exposing the autonomy policy.
|
||
|
||
## Delivery boundaries
|
||
|
||
| Delivery | Included | Completion gate |
|
||
| --- | --- | --- |
|
||
| Disposable local slice | Two fixture projects plus shared; one client; text/Markdown; test policy; source upload, note proposal, exact evidence/support checks, automatic acceptance, canonical read and browser citation | One complete valid flow, an invalid-location rejection, shared-source reuse, exact retry and restart recovery. No production-data or policy claim. |
|
||
| Evaluated pilot | Selected real sources/client/format; actual roles and separate agent identities; owner-approved policy; installed GEWU pack; scoped search and problem/source reading; quality sampling | Useful cited answers with agents offline, recorded semantic quality, enforced contribution boundaries and all pilot gates passed. |
|
||
| Full v1 | Every PRD P0 feature, hybrid retrieval, required inputs, four harnesses, declared Windows/macOS/Linux coverage, offline operation, protected decisions, recovery and local-to-VPS handover | Complete P0 and non-functional evidence, independent restore, measured capacity/retrieval and native-client workflows on the target deployment. |
|
||
| Follow-up | Selected M4/M5 P1 features | Separate feature gates; these do not delay P0 scope, source retention, authority or recovery. |
|
||
|
||
Use the proposed Python/FastAPI/Pydantic service, Git canonical records, private artifacts, derived SQLite index, stdio MCP connector and React/TypeScript reading app. Resolve and lock dependencies during scaffolding. Evaluate the embedding model and vector adapter before treating their candidates as committed choices. No new stack research or package-version claims are part of this plan.
|
||
|
||
## Decisions and readiness
|
||
|
||
Record outcomes in task notes or an ADR when they change a contract, authority boundary, persistence or deployment. A pending production decision does not prevent isolated fixture work; it does prevent claiming the affected pilot/release gate passed.
|
||
|
||
| Decision | Proposed starting point / required input | Needed before |
|
||
| --- | --- | --- |
|
||
| Pilot client and platform | Use Codex on the available Linux development host for disposable tests. Owner selects the real pilot pair; Windows/Codex remains an architecture recommendation. | Native pilot acceptance |
|
||
| Pilot content | Text/Markdown for the disposable slice; collect 20 representative sources across both domains and labeled questions. Finalize project IDs; `dome-performance` and `interactive-video` are suggestions. Include the deep-research report when supplied, labeled as an AI-synthesized secondary source. | Real ingestion and quality evaluation |
|
||
| Ownership and bootstrap | Name the first admin, project owners and shared-library owner. Use separate operator bootstrap and human owner MCP authorization. | Pilot account/policy provisioning |
|
||
| Automatic acceptance | Fixture policy initially permits additive shared sources and project factual notes with valid evidence/support. Owner must enumerate production rules for sources, notes, datapoints, entities, edges and interpretations in each affected domain. Missing rules hold changes. | Pilot automatic acceptance |
|
||
| Bundle and read behavior | Adopt the acceptance-contract recommendation: atomic bundles under all affected policies, conservative base-commit conflicts, and canonical accepted-revision reads during index lag. | M0 persistence contract |
|
||
| Quality sampling | Name reviewer, sample size/cadence, semantic-error limits and correction/escalation response. Same-agent extraction/assessment is already permitted for routine notes. | Pilot evaluation |
|
||
| Retrieval profile | Evaluate local embeddings on the labeled languages/corpus; fix tokenizer, vector packaging, graph/candidate caps and index generation metadata. | M1 hybrid acceptance |
|
||
| Operating envelope | Specify concurrent users/agents, upload/archive limits, retention/rights, VPS resources, backup owner, recovery-point and recovery-time objectives, benchmark hardware and percentile. | Hosted pilot and full-v1 measurements |
|
||
| Intake and release coverage | Define the initial NotebookLM export path and supported representations, exact OS/harness pairs, and any P1 launch requirements. Start with manual supplied exports unless another path is selected. | Format/client qualification and release scope |
|
||
|
||
## Execution order
|
||
|
||
Keep the PRD's M0–M5 names. Deliver the disposable flow and pilot through a narrow pass across M0–M3, then broaden M1–M3 for v1. Each row below is a bounded work package; split it into reviewable changes around observable behaviors rather than implementing an entire layer at once.
|
||
|
||
### M0 — Contracts, fixtures and acceptance foundation
|
||
|
||
| Task | Owning module and deliverable | Depends on | Acceptance evidence |
|
||
| --- | --- | --- | --- |
|
||
| M0.1 Scaffold a runnable foundation | Bootstrap/development: Python package and web workspace, lockfiles, isolated data-root configuration, CI scripts and one health path. Follow the architecture's proposed layout; instantiate only needed modules. | None | Recorded runtime versions; backend lint/type/test and frontend lint/type/unit/browser scripts run; import-boundary checks exclude transport/provider imports from domain code. |
|
||
| M0.2 Publish shared contracts | Domain/catalog: strict versioned records, node/edge schemas, project registry/lenses, scope, proposal lifecycle, errors, upload/run limits and whole-response budgets. Generate JSON schemas, read API types and explicit HTTP/MCP allowlists. | M0.1; bundle/read decision | Valid/invalid golden fixtures; unknown fields/unsafe IDs rejected; no mutation or owner operation generated for the read API. |
|
||
| M0.3 Create acceptance fixtures | Domain/evidence: two projects plus shared, sources and expected claims/relationships, valid and adversarial locators, labeled unsupported assertions and stale dependencies. Keep illustrative PRD examples separate from executable fixtures. | M0.2 | Wrong-location quote, correct quote with unsupported assertion, stale assessment, missing policy, cross-policy bundle and scope-escape cases have explicit expected outcomes. Semantic cases evaluate assessor quality separately from deterministic checks. |
|
||
| M0.4 Implement immutable intake | Artifact/extraction stores: resumable begin/chunk/finalize, source manifests, hash dedupe without lost provenance, text/Markdown location maps, safe serialization and immutable extraction artifacts. | M0.2–M0.3 | Matching chunk retries succeed; changed chunk/incorrect hash fail; interrupted uploads are uncitable; two projects reuse one artifact; unavailable extraction remains explicit. |
|
||
| M0.5 Implement proposal validation | Validator/policy: schema, scope, links, quote-at-location, assertion-bound assessments, findings, policy-domain evaluation and revision invalidation. | M0.3–M0.4 | Mandatory checks cannot be waived by policy; changed assertions cannot reuse assessments; failing factual claims remain staged; routine fixture proposals become eligible without human approval. |
|
||
| M0.6 Implement durable publication | Repository/workflow: staged revisions, journal, exclusive writer, final authority/base/policy rechecks, private Git index, expected-head publication, committed acceptance manifest and immutable receipt reconciliation. | M0.5; identity port | One commit per accepted revision; changed-payload retry conflicts; concurrent writes serialize; unrelated files are excluded; committed-tree reads work before indexing. |
|
||
| M0.7 Prove recovery and replacement boundaries | Recovery/module contracts: inject crashes/write failures around proposal, artifact, ref, receipt and index boundaries; block new publication while reconciliation is incomplete. | M0.6 | Ref-before-receipt retry reconstructs the original receipt without another commit; live retries and unrelated queued work respect the recovery gate; a second writer is excluded. Subprocess termination immediately after ref update and before receipt write survives fresh repository/workflow adapter construction; an empty derived index rebuilds from the recovered canonical commit and returns its citations. The shared repository contract passes against Git and an independent in-memory adapter, and the node reader works against the substitute unchanged. A bad pending-manifest digest blocks an unrelated proposal before it is staged; after reconciliation is repaired, the original receipt is recovered and the queued proposal succeeds. The index transaction is killed during rebuild over an existing generation; SQLite retains that generation, then a normal search rebuilds and serves the complete current canonical set. |
|
||
|
||
M0 exits with executable contracts and a disposable accepted source/note that can be read at its committed revision. Real-source semantic evaluation and all 20-source M0 fixtures remain visible until supplied and checked; synthetic fixtures alone cannot close that PRD requirement.
|
||
|
||
### M1 — Authenticated service, MCP and retrieval
|
||
|
||
| Task | Owning module and deliverable | Depends on | Acceptance evidence |
|
||
| --- | --- | --- | --- |
|
||
| M1.1 Establish identities and authority | Identity/control: admin/editor/viewer, stable agents and hashed revocable keys, owner grants, browser sessions, restricted bootstrap and separate owner client/challenges. | M0.2; ownership decisions for pilot | Viewer proposals denied; browser admin still cannot author; rotation retains attribution; revoked key fails at publication; agent cannot impersonate assessor/owner. Protected authorization is digest-bound and expires; replay/change/grant-revocation and recovery cases pass. |
|
||
| M1.2 Expose one application core | MCP/read HTTP: catalog-backed reads, upload/proposal/status/revision/withdraw/evaluate and run operations, bounded errors/diagnostics. | M0.4–M0.7, M1.1 | Actual transports enforce capabilities and scope; no generic HTTP/CLI knowledge writes or raw queries; accepted reads disclose canonical/index revisions; diagnostics contain no secrets. |
|
||
| M1.3 Bridge selected local files | Connector: stdio MCP to service, explicit-file byte transfer, bounded upload resume, service identity checks and protocol-only stdout. | M1.2 | Selected file reaches service by bytes/hash, including a remote service; arbitrary VPS-local path interpretation fails; byte payloads and credentials stay outside model-visible responses. |
|
||
| M1.4 Deliver scoped lexical reading | Index/query: FTS, typed edges, project/shared filters and `default_includes` resolution, canonical revision reads, neighbors/history, read-time freshness and synthesis dependency warnings, complete citation groups and tokenizer accounting. | M0.7, M1.2 | IDs/redirects/graph/citations obey scope; omitted project includes resolve defaults plus shared for one project context, explicit empty includes suppress defaults, and every expanded project is authorized; stale warnings survive budgets; too-small budget returns a defined error; index deletion/rebuild preserves authority and citations; index lag is disclosed. |
|
||
| M1.5 Qualify hybrid retrieval | Embedding/vector/ranking adapters: model/native-package probes, scoped keyword/vector fusion, capped graph expansion, language fixtures and isolated index generations. | M1.4; retrieval/benchmark decisions | Top-five source recall meets the PRD target on labeled questions; old citations survive model/parser changes; offline assets work; keyword degradation is labeled. Real provider swap and generation rollback pass consumer contracts without search-consumer changes. |
|
||
|
||
M1.1–M1.4 support the pilot. M1.5 and all four native client flows are required to close full M1/v1. Authentication and enforced data isolation precede any non-disposable shared use.
|
||
|
||
### M2 — Local agent integration and input coverage
|
||
|
||
| Task | Owning module and deliverable | Depends on | Acceptance evidence |
|
||
| --- | --- | --- | --- |
|
||
| M2.1 Bind the existing GEWU pack | Agent profile/skills/pipelines: use real tool/schema contracts, pin pack/module hashes, record scope/lens, source coverage, skips, proposal IDs and receipts in durable runs. Update the separate pack submodule deliberately. | M1.2–M1.4 | Installed pack completes ingest/query, attributes same-agent assessments, reports unavailable P1 capabilities, and resumes an uncertain outcome through service status without regeneration or duplicate acceptance. |
|
||
| M2.2 Build additive installation | Harness adapters: preserve existing personas, skills and MCP entries; dedicated/integrated workspace routing; thin discovery pointers including needed GEMINI routing; copy fallback and owned-entry rollback. | M2.1, M1.3 | Install/uninstall leaves unrelated configuration intact; discovery, connection and workflow verification recorded separately; newer pack installation cannot silently alter a pinned active run. |
|
||
| M2.3 Expand source intake | Fetch/parser adapters: URL/HTML, PDF, supported local files, pasted text and supplied NotebookLM exports; transcript locators where applicable; archived downloads, source identity/revision changes, reference-only failure records. | M0.4, M1.3; format limits | Exact locations verified on real fixtures; changed URL creates a revision; failed capture preserves prior bytes; scanned/encrypted/unsupported material returns honest status. Parser replacement retains old extraction resolution. Required input paths are qualified; unsupported document subtypes are listed explicitly. |
|
||
| M2.4 Complete linking and consistency | Entity/relationship application services plus ingest skills: canonical aliases, duplicate candidates, reversible merge history, evidence/rationale-bound typed links, contradictions and supersession. | M0.5–M0.6, M2.1 | Shared dedupe preserves provenance; substantive links include conditions/support; similarity does not create accepted edges; destructive merges require authority; contradictions retain both claims. |
|
||
| M2.5 Qualify native hosts and quality | Acceptance/client fixtures: Codex, Claude, Antigravity and OpenCode on declared supported OS pairs; five common documents through two agents; sampling and labeled coverage/error evaluation. | M2.1–M2.4; quality/coverage decisions | Each supported pair transfers, rejects invalid evidence, auto-accepts eligible work, retries/resumes and retrieves citations. Convergence compares findings/support errors; no pass based on matching prose or tool listing. |
|
||
|
||
For the pilot, complete M2.1–M2.2 and the selected-format/client portions of M2.3–M2.5. Remaining matrix cells and source paths stay NOT RUN or BLOCKED until verified; they remain full-v1 obligations.
|
||
|
||
### M3 — Read-only application and pilot evaluation
|
||
|
||
| Task | Owning module and deliverable | Depends on | Acceptance evidence |
|
||
| --- | --- | --- | --- |
|
||
| M3.1 Build reading primitives | Design system/app shell: editorial tokens, typography, scope selector, result/evidence/relationship rows, freshness and system states; generated read-client types. | M0.2, M1.2; UI fixtures may start earlier | Real rendered review under the [visual protocol](development/VISUAL_DESIGN_PROTOCOL.md); keyboard navigation and error/empty/stale/index-lag states; semantic tokens replace feature-local styling. |
|
||
| M3.2 Connect search and citations | Search/node/source/history features: URL-owned scope/filters/query/selected node/revision, archived revision download/location preview, typed neighbors and citing notes. | M1.4, M3.1 | Scope → search → note → quote → exact archived revision works with agents stopped; direct query/filter URLs restore after authentication, selected node and history revision deep links and browser back/forward restore the reader; typing query/facets does not submit until Search; obsolete responses do not replace current scope; logout clears private caches; safe Markdown/preview rendering. History is bounded to the latest 20 revisions; real two-minute tasks and visual/accessibility qualification remain open. |
|
||
| M3.3 Deliver problem/topic reading | Problem/topic/staleness features: approaches, assumptions, cases, source references, facets and separately labeled related suggestions. | M2.4, M3.2 | A viewer explores multiple approaches and their conditions; excluded projects remain explicit; no capture/edit/accept/generate controls or hidden mutation client. Search supports URL-owned exact tag/audience/use/domain facets, applied in SQLite before pagination and bound into cursors. “Browse problems” lists scoped problem records without a text query, with type-bound pagination; cross-project explanations and visual/accessibility acceptance remain open. Evidence panel replacement preserves public API/navigation. |
|
||
| M3.4 Evaluate the pilot | Acceptance/quality: real two-project corpus, shared-source reuse, viewer/editor/admin and two agent identities, retry/restart, quality sampling and timed reading tasks. | Pilot portions of M0–M3; production pilot decisions | Person finds a cited answer within two minutes; location validity and assessment coverage meet PRD requirements; semantic errors and reviewer effort recorded against agreed limits. All missing full-v1 features remain listed. |
|
||
|
||
Inspect the primary path at the visual protocol's desktop/mobile sizes, 200% zoom and 320-pixel reflow. Combine automated accessibility checks with keyboard and screen-reader evidence. The pilot requires enough real UI to test research usefulness; a service-only demonstration cannot close M3.4.
|
||
|
||
### Full-v1 hardening and local-to-VPS release
|
||
|
||
This gate spans M0–M3; it is not a new product milestone. Finish all P0 coverage before labeling a release v1.
|
||
|
||
| Task | Deliverable | Required evidence |
|
||
| --- | --- | --- |
|
||
| R1 Enforce deployment boundaries | Restricted service identity, private data/credential mounts, parser sandbox/resource limits, bounded queues/uploads, authenticated artifacts, URL/DNS/redirect restrictions and MCP admission controls. | Traversal/symlink escape, malicious source instructions, secret-read, private-network fetch, unsafe preview and denied-capability cases fail through actual boundaries. Separate-process existence alone does not prove sandboxing. |
|
||
| R2 Exercise offline and portable operation | Native Windows/macOS/Linux paths, safe storage keys, non-symlink installation, local model/tokenizer/font assets and parser/vector packaging. | Existing-source validation, ingestion, index/rebuild and reading work offline; configured local-model workflow demonstrated where claimed; new external fetching reports unavailable. Declared host/OS matrix passes. |
|
||
| R3 Measure product and capacity targets | Fixed workload/corpus/hardware, latency percentiles, retrieval quality, queue saturation and cold/warm rebuild profiles. | PRD search target under 300 ms at 10,000 project nodes; 100,000-node scale and under-ten-minute rebuild evaluated with stated cache assumptions; ≥80% top-five source recall on 20–30 questions per project; acceptance rate tracked toward ≥70% after tuning. Exceptions remain visible, not redefined as passes. |
|
||
| R4 Prove backup, restore and migrations | Consistent canonical/artifact/workflow/control backups; protected owner-key recovery; schema/index migrations and rollback procedures. | Independent restore verifies hashes, grants, manifests, runs, receipts and citation reads, then disposable acceptance. Index deletion loses no authority. Revoked access is not revived; owner-key recovery and missing-key reset paths tested; recovery objectives measured. |
|
||
| R5 Transfer authority and qualify VPS | Locked release images, Compose/Caddy, private volumes, freeze/transfer/verify local state, rebuild, client endpoint switch and controlled rollback/handback. | One writer only; authenticated HTTPS/browser reading and native remote upload/accept/retry/resume/retrieval evidence; restored test copy cannot become a second authority; local and deployed revisions recorded separately. |
|
||
| R6 Close release coverage | P0 traceability and acceptance ledger, runbooks, limitations and first-month reuse follow-up. | No required gate is FAIL, BLOCKED or NOT RUN. Record shared-node reuse during the first month rather than claiming it passed on release day. |
|
||
|
||
### M4 and M5 — Selected follow-up
|
||
|
||
M4 adds P1 analysis/task briefs, synthesis compilation/refresh, deterministic evidence bundles or saved briefs, curator reports, audience lenses, matrices, comparisons, timelines and graph exploration. Decide brief semantics and release priority before implementation. Existing P0 synthesis dependency warnings and source/relationship reading are already delivered in M1/M3.
|
||
|
||
M5 adds P1 shared promotion and project export/import with redirects, provenance, policy-domain authority and durable receipts. Imports enter through MCP proposals. Exercise three projects for this later milestone; the confirmed pilot has two. P0 cross-project scope and dedupe are already enforced in M0–M2. Keep project confidentiality, automatic hooks, clustering and slide export under their existing P2 priorities.
|
||
|
||
## P0 traceability
|
||
|
||
This maps implementation ownership; the release ledger must still record each requirement separately, including the PRD's non-tabular evidence, lifecycle and source-retention rules.
|
||
|
||
| PRD requirement | Work packages |
|
||
| --- | --- |
|
||
| FR-P1, FR-P2, FR-P5 | M0.2–M0.3, M1.2, M1.4, M2.1, M3.2 |
|
||
| FR-P3 | M0.4, M2.4 |
|
||
| FR-P8 | M1.1–M1.2, M0.6–M0.7, R1/R4 |
|
||
| FR-I1, FR-I2, FR-I9 | M0.4, M1.3, M2.3 |
|
||
| FR-I3, FR-I4, FR-I7 | M0.3–M0.5, M2.1, M2.5 |
|
||
| FR-I6 | M2.4 |
|
||
| FR-I8, FR-I11 | M0.6–M0.7, M1.2, M2.1–M2.2 |
|
||
| FR-V1, FR-V2, FR-V3 | M0.5, M1.1–M1.2, M3.4 |
|
||
| FR-V4 | M2.4, M1.4, M3.2 |
|
||
| FR-V5 | M0.6–M0.7, M1.1–M1.2 |
|
||
| FR-A1, FR-A9, FR-A10 | M0.2, M1.2–M1.4, M2.2, M2.5 |
|
||
| FR-A2 | M1.5 |
|
||
| FR-A3 | M0.2, M1.4–M1.5 |
|
||
| FR-A5 | M0.6, M1.1–M1.2, R1/R5 |
|
||
| FR-A6, FR-A11 | M2.1–M2.2, M2.5 |
|
||
| FR-C3; displayed synthesis warnings (§6.3.2) | M1.4, M3.2–M3.3 |
|
||
| FR-U1, FR-U2, FR-U3, FR-U6, FR-U9 | M3.1–M3.4 |
|
||
| Non-functional requirements and success metrics | M0.1–M0.7, M2.5, M3.4, R1–R6 |
|
||
|
||
## Task completion and evidence
|
||
|
||
Each implementation task records: task ID, observable outcome, requirement IDs, owning module/public contract, dependencies, failure cases, state migration/rollback where applicable, checks and evidence. Apply the [development protocol](development/DEVELOPMENT_PROTOCOL.md) and [modularity protocol](development/MODULARITY_PROTOCOL.md); do not duplicate their contracts in every task.
|
||
|
||
The [acceptance ledger](acceptance/ledger.md) tracks each M0–M3 and R1–R6 gate, its current status, evidence scope and missing proof. Detailed fixture and command results remain in dated acceptance records. Each gate must distinguish structural pack validation, installed discovery, connection, native workflow and semantic quality; no fixture-only evidence closes a pilot or release gate.
|
||
|
||
Dependencies determine sequence, not fixed calendar dates. Staffing and source/decision availability are unknown; estimate effort after M0 contracts and the first vertical flow expose parser, client and recovery costs. Review progress at the disposable-flow, pilot-evaluation and full-v1 gates, with unresolved decisions and failed evidence visible.
|
||
|
||
## First implementation batch
|
||
|
||
1. Complete M0.1 with isolated temporary data roots, actual verification scripts and initial import-boundary checks.
|
||
2. Implement M0.2–M0.3 for a shared text source and one project factual note, plus wrong-location, stale-assessment and scope fixtures for both projects.
|
||
3. Implement M0.4–M0.7 for that bundle under an explicitly named test policy; prove exact retry and ref-before-receipt recovery.
|
||
4. Add the minimal M1 identity/MCP/connector/read path, bind `kb-ingest`/`kb-query`, and connect a minimal M3 citation view.
|
||
5. Demonstrate the disposable source-to-citation flow, then finalize real pilot inputs/policy and expand the same contracts. Do not scaffold P1 workflows before this path works.
|