feat: online verification, node removals, and knowledge browser
CI / quality (push) Canceled after 0s

Adds claim-bound online evidence records and citation verification scripts,
node removal transport/adapter, retrieval benchmark and owner-review tooling,
GraphCanvas/SourceDetail knowledge browser UI, and the ingest handoff helper.
Bumps the agents/gewu pack to the online-verification branch.
This commit is contained in:
vasily committed 2026-10-16 20:02:29 +08:00
1 parent a3156f4e08
commit 78033bb830
83 files changed
+13476 -1051

No files matched your search

+248
View File
@@ -0,0 +1,248 @@
# Knowledge quality: implementation, verification, and benchmark plan
Status: partially implemented and demo-verified, 16 October 2026. The detailed design and qualification gates below remain the target; the completed slice and evidence are recorded in the [demo acceptance report](acceptance/2026-10-16-knowledge-quality-demo.md). Scope covers the GEWU backend and reader, and the separate [GEWU_agent repository](../agents/gewu/README.md), mounted at `agents/gewu`. The root [SPEC.md](../SPEC.md) retains its existing relevance-review scope.
## Goal and observed problems
A reader or consuming agent should find a useful explanation, understand when it applies, inspect supporting and conflicting evidence, and open the exact source passage. Ingesting another source should improve existing knowledge without duplicating concepts or losing prior evidence.
The live-store review found 103 nodes and 14 sources at canonical revision `137db3a282ffa9f8b3589d9946c085dc38f37c2b`; the search index matched that revision. This is a dated baseline, not a permanent corpus expectation:
| Problem | Observed baseline | Consequence |
| --- | --- | --- |
| Missing classification | All nodes have empty tags, audience, use_for, and domain | Facet browsing cannot help readers find relevant knowledge |
| No accumulated support | Every node has one claim, one citation, and one source | Repeated findings do not become better-supported knowledge |
| Insufficient explanation | Summary median 25 words, range 13–46; no assumptions or limitations | Readers cannot reliably understand or apply a technique |
| No knowledge connections | Zero typed edges | Problems cannot collect approaches, tradeoffs, or cases |
| Source navigation gap | UI displays an exact excerpt but has no contextual reader navigation | Users still search through source material |
| Lost original format | All sources captured as pasted text; 13 lack original URL and publication date | Native PDF pages and original-source provenance are unavailable |
| Support quality gap | All 103 stored quotes match their spans; sampled BallJam claim adds a channel count absent from its attached quote | Valid locations alone do not establish semantic support |
The ingestion pipeline says to update an existing stable ID, but `validate_proposal` rejects an existing ID with `NODE_ID_EXISTS`. The contribution protocol instead prescribes replacement nodes with supersession. This conflict must be fixed in the service contract and pack together. Facets are optional, and normalization is reached through `ingest_review_create`; the reviewed store had zero such review items.
Product basis: [GOAL](GOAL.md), [PRD sections 6.3–6.5 and success metrics](PRD.md), [acceptance contract](architecture/ACCEPTANCE_CONTRACT.md), and [module protocol](development/MODULARITY_PROTOCOL.md). Preserve short summaries for scanning; add the explanation needed to reuse the knowledge.
## Scope, invariants, and completion boundary
V1. Canonical knowledge changes only through authenticated, validated MCP proposals. Git remains authoritative; SQLite indexes remain rebuildable.
V2. Enrichment retains node identity, prior supported claims, evidence provenance, and readable history. Removed or changed accepted assertions require an explicit correction, not a silent additive update.
V3. Every factual detail, equation, qualifier, assumption, limitation, and substantive relationship binds to applicable evidence or is explicitly marked as interpretation. Interpretation never becomes measured fact through formatting.
V4. Exact quote validity, semantic support, source reliability, relevance, and independent corroboration remain separate judgments.
V5. Retries, concurrent proposals, crashes, provider failures, and stale indexes cannot duplicate accepted updates, lose evidence, or falsely report completion.
V6. Node/source/history/artifact reads and evidence additions enforce project/shared authority. New write behavior does not introduce a browser knowledge-write path.
V7. New ingestion records a classification disposition. Provider unavailability is explicit; it cannot silently erase existing facets or weaken evidence rules.
V8. Citation links bind immutable source/extraction versions. Readers either reach the cited passage or receive an explicit unavailable/fallback state; never highlight a guessed match.
V9. Backend and installed pack versions are compatible and pinned. Benchmarks record the actual CLI model selection, rather than trusting a node's authored model label.
V10. Benchmarks run in isolated stores and workspaces. Live knowledge, credentials, existing frozen questions, and historical raw results remain untouched.
Completion means implemented backend and agent behavior, passing deterministic and browser gates, qualified native-client demo, and benchmark results meeting the frozen thresholds below. A prepared plan or passing unit suite alone does not close this work. Full cross-platform/four-client qualification and VPS deployment remain separate release gates.
## Proposed design
### 1. Versioned enrichment of accepted nodes
Keep the existing create-only `upsert_node` behavior for compatibility; document its meaning. Add a distinct proposed `update_node` operation rather than silently changing old clients' behavior. Finalize names and schema in an ADR before implementation.
The update carries the stable node ID, expected current node revision/digest, update intent (`enrich` or `correct`), proposed complete next record, and a change rationale. A deterministic diff validates intent. Use the existing expected-head publication and proposal replay machinery; compare the target node revision again inside the publication boundary.
- Enrichment may append claims/evidence, expand supported detail, and add classifications; it cannot remove prior evidence or change an existing claim's meaning.
- Corrections explicitly identify changed/removed claims or evidence and reasons, retain history, and follow applicable correction policy. Contradictory findings remain available and linked rather than being overwritten as agreement.
- Claim IDs remain stable across updates; new claims get new IDs. Recompute assertion-path and evidence/assessment digests when record layout changes. Old assessment attribution is retained for unchanged claims; newly assessed evidence has attributable assessment history. Resolve this explicitly because current validation requires assessor identity to match the proposer.
- Repeated evidence identity is a no-op; do not collapse different passages or independently captured provenance merely because quote text matches. Define equality using revision, extraction, locator, and assertion identity.
- Reassess the complete changed claim/evidence set. Preserve earlier assessment events separately so another agent's enrichment does not rewrite authorship.
- Stale updates return a typed conflict with the current target revision. The agent rereads and prepares a merged proposal; no last-writer-wins replacement.
- A proposal containing a source registration, node update, and edges publishes atomically under every affected policy. Unknown targets, unauthorized shared writes, and incompatible lifecycle changes fail before publication.
Represent source lineage where known: a review, export, and original paper can be distinct evidence records without being counted as three independent confirmations. Unknown independence stays unknown.
### 2. Substantive, evidence-bound node content
Add optional, versioned structured content sections to `KnowledgeNode`, leaving existing records readable without rewriting them. Keep the short summary. Proposed section kinds: explanation, mechanism, procedure, applicability, worked example, math, and figure. Each section references stable claim IDs; interpretation sections declare that status and cite their basis. Retain the existing separately assessed assumptions and limitations.
Store bounded Markdown/math content and archived figure references, with explicit byte/count limits and an overall record/read budget. Figure references bind source revision, page/region or extraction provenance, caption, and supporting claims. Never accept agent filesystem paths as artifact references. Existing PDF extraction preserves text, not figure assets or highlight geometry; figure capture/mapping is an explicit adapter deliverable.
Render safe Markdown, math, and accessible diagrams/figures in the reader. Disable raw HTML and active content; sanitize links and figure captions. Pin renderer dependencies during implementation. If Mermaid is supported, use a restricted renderer; otherwise use inert diagrams. A reconstructed diagram must be labeled as such and trace to its supporting claims.
Detail is selected by informational value, not minimum word count. A technique normally explains how it works, when it applies, tradeoffs, and a supported example. If the source lacks details, report the gap. Do not manufacture equations, complexity guarantees, limitations, or illustrative source figures.
For branch explosion, distinguish authored content size, possible playthrough count, and state-space size. A model may derive a labeled illustration such as a full b-ary tree's node count, but it cannot attribute that derivation to a source that does not contain it. State-based techniques do not automatically eliminate state-space growth.
Search indexes the substantive content with a versioned profile. Search results retain bounded summaries and claim-owned citations; full detail is fetched explicitly. Verify token budgets and all-or-nothing citation groups so added content does not break agent retrieval.
### 3. Classification as an explicit ingestion stage
Provide a dedicated facet-resolution interface usable for ordinary ingestion, rather than requiring a human relevance-review flag. Reuse the existing project catalog, embedding comparisons, and optional backend Jev adapter behind a public application boundary.
The agent proposes up to five tags and three labels for each other facet, with useful definitions for new vocabulary. Existing canonical labels can be reused without a provider call. Preserve current Jev promotion thresholds unless a separate measured change justifies altering them.
Return per-candidate outcomes: canonical match, approved new label, pending candidate, rejected candidate, or provider unavailable. Accepted nodes carry approved labels; unresolved candidates/dispositions remain attributable and retrievable as follow-up work. Catalog registration alone must not be reported as attaching a label to a node.
Validate approved facet membership on new/updated records under the versioned contract. Legacy empty/free-text facets stay readable and are flagged for migration. Empty facets require a recorded reason for new ingestion; provider failures permit a visible pending-classification state and later retry. Do not force labels into facets that are inapplicable, or turn classification into a semantic evidence gate.
### 4. Contextual source reader and precise links
Add a typed citation target carrying project, node/claim reference, source revision, extraction, block/span, and optional PDF page/cue times. Encode it in the reader URL so refresh, sharing, back/forward, and historical revisions restore the same location.
- Text/Markdown/HTML: load a bounded context window around the stored normalized span; scroll and highlight the exact occurrence, including repeated-quote cases. Navigation can expand context without fetching the entire document.
- PDF: archive the native original, preserve page-aware extraction, and render the cited page with a text layer. Map extraction spans to page coordinates with a versioned adapter; handle ligatures, columns, line breaks, rotation, and Unicode offsets. Page-only navigation plus an exact extracted passage is an explicit fallback when geometric matching is ambiguous. Never claim a visual highlight passed on that fallback.
- Transcript: retain cue-time links and show the exact passage; media seeking is available only when a supported archived/media target exists.
- Images/scanned or encrypted PDFs: report unsupported/OCR-unavailable states. General OCR implementation is outside this first scope; no fabricated page-text mapping.
- Figures: open the archived figure/page with its caption and provenance, while retaining original download.
Review artifact response headers, authorization, range requests if needed by the viewer, and bounded extraction reads. Citation opening is a read operation. Keyboard focus, screen-reader announcement, mobile layout, loading, denied, stale-link, and unavailable states are acceptance cases.
For current pasted captures, preserve valid text citations. When originals become available, capture native revisions and propose new evidence bindings; do not pretend pasted offsets are native PDF coordinates.
### 5. GEWU_agent behavior and distribution
Update `skills/kb-ingest`, `skills/kb-curate`, ingestion/curation/query pipelines, contribution/evidence protocols, capability notes, generated contracts, and manifest in the separate GEWU_agent repository. Keep the backend as contract owner; export synchronized schemas/catalogs instead of hand-editing divergent copies.
The ingestion procedure becomes:
1. Discover compatible tools; create/resume a pinned run and load project intent/catalog.
2. Capture the selected original through the connector or supported URL/text capture. A PDF must use the PDF upload path, not a pasted-text shortcut. Retain original URL/type/date and declared export provenance where available; unknown remains unknown.
3. Read sufficient contextual sections and record coverage. Build a finding plus mechanism, constraints, examples, equations/figures where useful and supported.
4. Search multiple terms and inspect likely matches. Classify each finding as create, enrich existing claim, extend existing node, contradict, unrelated, or unresolved identity.
5. For a match, collect the current stable node revision and preserve prior content. Submit an evidence-backed update; an already known concept is not a reason to skip a new source's support.
6. Resolve facets and propose substantive typed relationships with evidence or labeled rationale. Use `addresses` when `solves` would overclaim.
7. Prepare, evaluate, submit eligible proposals, verify receipts, and checkpoint actual source/finding dispositions. Report held, pending-classification, unsupported, and budget-deferred work separately.
Query behavior distinguishes source claims from derivations and returns usable explanations with evidence. Curate audits unsupported clauses, missing classifications/details, duplicate identity, orphan nodes, and missing relationships. Provider or context exhaustion saves a resume point rather than dropping details silently.
Version the pack and service compatibility requirements. Unsupported update/content capabilities produce an explicit blocker or read-only fallback, never a duplicate node to bypass the missing operation. Test the installed pack through Antigravity, not only source files or a handwritten API driver. Update both repository revisions and the parent submodule pointer as one coordinated release.
## Isolated demo project and credentials
Create a unique run directory outside `.data/gewu`, containing an isolated backend data root, backend checkout, independently pinned agent checkout/installation, client workspace, fixture inbox, and private evidence directory. Inventory existing dirty changes in both repositories before implementation; preserve them and use isolated implementation branches/worktrees with explicit inclusion of intended prerequisites.
Use project `kb-quality-demo-<run>` plus a second scope for authorization tests and explicit shared fixtures. Project intent: help a developer compare and apply ways to control interactive narrative content growth. Record owner-approved test policies separately from production policy; permit routine supported demo enrichments without granting owner authority to the ingest agent.
Provision distinct demo principals/credentials for bootstrap admin, ingest agent A, enrichment agent B, and viewer. Scope writers to demo resources; include shared write only in the explicit shared-policy scenario. The second agent proves another contributor can add support while preserving the first contributor's attribution. Derive effective capabilities from actual role rules, not assumed role names.
Existing building blocks are `scripts/seed_test_registry.py`, `scripts/provision_credential.py`, the authenticated project/credential interfaces, and the pack installer. The provisioning script prints a secret once: invoke it through a private setup wrapper that captures output directly into a permission-restricted credential/config file, never a model-visible transcript, committed fixture, screenshot, or manifest. Give the agent its non-secret principal ID and selected file IDs explicitly. Keep bootstrap/owner credentials outside the client workspace.
Start the service on unused loopback ports with explicit data root, configure the client MCP endpoint and selected-file connector using isolated host configuration, and verify diagnostics, project context, authenticated read, denied writes, and installed pack pins. Record configuration shape with secrets redacted. Native-client host discovery/configuration remains a tested step, not an assumed consequence of installation.
At completion revoke demo credentials, stop owned processes, and retain private artifacts under a documented retention path. Keep a reusable demo only with credentials revoked; future sessions provision fresh keys. Failure cleanup has the same requirements. No live-project policy or credential needs to change.
## Fixtures and benchmark design
### Controlled four-file quality corpus
Author and freeze a small, clearly labeled synthetic research corpus with deterministic reference truth. It is a test fixture, not external research or production knowledge:
| File | Content and purpose |
| --- | --- |
| A: native PDF | Branch-growth problem, explicit equation, assumptions, and a foldback method with a diagram |
| B: captured HTML | Independent support for the same method plus an applicability condition; repeated quote occurrences test exact location |
| C: native multicolumn PDF | State-based alternative, procedure, a table/figure, and a limiting case that qualifies an overbroad claim in A |
| D: Markdown case | Concrete implementation/cost example linking problem and techniques, with audience/use/domain signals |
Store expected assertions, relationships, allowed facet equivalents, equations/figures, source lineage, and exact page/block/span truth separately from the ingest workspace. Freeze hashes and scoring rules before runs. Mark mandatory learning units and optional findings so node count is never a quality proxy.
Run A first; then B using the second agent identity; then C and D. Verify intermediate snapshots: the original technique ID survives, both support passages attach to the appropriate claim, and the qualification is preserved. Replay and reorder ingestion in additional fresh runs. Use side fixtures for corrupted citations, provider outage, injected source instructions, scanned PDFs, changed revisions, duplicate capture, and unauthorized scope.
### Representative regression corpus
Reuse the four original files and manifest from the existing [four-file plan](acceptance/four-file-validation-plan.md), not copied pasted-text substitutes. Preserve prior results and reference partitions. Audit whether any reserved material was already exposed; exposed questions become tuning-only. Create a new withheld partition when needed rather than asserting a compromised set is blind. Include a deterministic imported legacy snapshot to verify old records remain readable.
Freeze 30 controlled questions: 20 tuning and 10 withheld. Cover mechanism, applicability, tradeoffs, math/figure interpretation, multi-source support, contradictions, facets, problem-to-technique traversal, and exact source opening. Ingestion cannot access answer keys or withheld questions. Report semantic coverage against reference learning units even when policy-held units prevent publication.
### Comparable runs and model pin
1. Measure an unchanged baseline using the recorded pre-change backend and installed-pack snapshots on fresh stores. Pin any existing dirty prerequisites as explicit snapshot artifacts; do not label an arbitrary clean HEAD as the reviewed live baseline.
2. Use the same source hashes, machine, client, budgets, project intent/policies, retrieval mode, and embedding/parser configuration for the candidate. Contract-required policy additions are recorded as a manifest diff.
3. Pass `antigravity --model gemini-3.8-flash-low` explicitly for ingestion, answer generation, and model-based validation in these four-file runs. Verify the CLI's available-model list and invocation/result metadata. If unavailable, report BLOCKED and the available list before any substitute is used. Authored `model_id` fields are insufficient proof.
4. Run separate extraction-only, backend deterministic, native-agent ingestion, retrieval/answer, and browser-navigation measurements. This distinguishes infrastructure defects from semantic/model quality.
5. Tune only on tuning data, then freeze candidate and thresholds. Run withheld evaluation once on the locked candidate; any subsequent change needs a new qualification cycle and unexposed holdout.
6. Perform three fresh, comparable candidate runs, not three replays of one accepted store. Record every run and variability; no best-run-only reporting. Compare ingestion order as a separate stress case.
Semantic review uses two separately prompted, isolated passes with the required same model, labeled correlated assessments rather than independent-model consensus. Disagreement/uncertainty triggers a reported review item. The current `run_semantic_review.py` requires two distinct models: adapt its contract for same-model passes and pin pass/prompt identity; never let its default models silently override the project pin. Deterministic fixture expectations remain the hard oracle for seeded failures. Real-source semantic scores remain machine assessments until owner inspection; do not call them human-verified.
## Verification matrix
| Layer | Required cases and evidence |
| --- | --- |
| Domain/schema | Valid create/enrich/correct; immutable IDs/project; stable claim identity; evidence dedupe; new provenance retained; classification dispositions; section/figure bounds; malformed/unsupported content rejected |
| Publication/recovery | Exact replay yields same receipt and no extra commit; stale-base conflict; concurrent evidence additions converge after explicit rebase; crash at publication checkpoints; source/update/edge bundle atomicity; preserved old revisions and authorship |
| Evidence/support | Exact page/span/quote/digest checks; wrong page/extraction/claim rejected; added channel count, date, unit, negation, or qualifier flagged by semantic evaluation; changing evidence renews assessment; conflicting sources not flattened |
| Scope/auth | Viewer cannot propose; writer cannot update another project/shared record without authority; historical reads, artifact access, and new citation endpoints enforce scope; tokens revoked successfully |
| Index/retrieval | Rebuild from canonical files matches latest content/facets/edges; delayed index disclosed; old revision reads stable; bounded rich-content retrieval retains citation groups and scope |
| Facets/providers | Canonical reuse; spelling variants resolve; appropriate new label promotion; low-confidence candidate retained pending; missing Jev/embeddings and provider timeout recorded; enrichment preserves existing facets |
| Extraction/reader | Native PDF capture, Unicode/ligatures, columns, repeated quote, page rotation, figure provenance, long document windows, changed source revision, page-only fallback, missing extraction, scanned/encrypted states |
| Agent workflow | Installed discovery/version mismatch; second-source enrichment; distinct findings kept separate; contradiction; no direct database/file writes; budget checkpoint/resume; uncertain submit recovery; source injection cannot authorize tools |
| Browser | Project → problem → technique → claims → each source passage; refresh/back/share/history; scroll/highlight correctness; keyboard/focus and narrow layout; inaccessible source and provider-pending states |
| Compatibility | Old records/packs remain readable; unsupported writes fail clearly; schema exports and generated API types agree; replacement adapters honor public contracts; restore/rebuild retains update history and source artifacts |
Use temporary stores and fixed deterministic fixtures for automated tests; no live provider calls in unit suites. Extend existing tests under `tests/domain`, `tests/application`, `tests/api`, `tests/adapters`, pack tests, and `web/e2e`. Browser acceptance uses the T3 collaborative preview when available; record the actual tested environment.
Existing verification entry points: `make check`, `make typecheck`, `make test`, `make boundaries`, `make web-check`, and `make browser-test`. Regenerate contracts/types with `make api-types` and the pack manifest with `make pack-manifest` after contract changes; verify generated diffs and installed checksums. Run focused checks per slice, then the affected full integration gates. Partial implementation results and remaining gates are recorded in the [demo acceptance report](acceptance/2026-10-16-knowledge-quality-demo.md); they do not qualify the full completion boundary above.
## Frozen acceptance targets
These are proposed gates to freeze before scored implementation runs, not measured results. Denominators include misses, holds, and deferred reference learning units; exclude only fixture units declared out of scope before the run. Report every metric by source and node type where meaningful.
| Metric | Gate |
| --- | --- |
| Authority, publication, replay, history, recovery | 100% required deterministic cases; zero unauthorized disclosures/writes, lost evidence, or duplicate replay commits |
| Exact evidence validity | 100% accepted factual claims and substantive factual edges bind valid source revisions and locations |
| Controlled support correctness | All seeded unsupported/contradictory mutations identified; zero unsupported accepted reference assertions |
| Representative semantic support | At least 95% of assessed accepted atomic assertions fully supported; zero material number/unit/date/negation errors; list uncertainty separately |
| Enrichment | Every required repeated finding retains stable node/claim identity and all distinct valid evidence; exact replay adds nothing |
| Classification | 100% new findings have dispositions; at least 90% of reference-applicable node/facet pairs have correct canonical labels and at least 95% assigned-label precision; no credit for merely pending candidates |
| Useful detail | At least 90% reference learning-unit coverage; every mandatory mechanism, condition, limitation, equation, and figure present with valid provenance; no word-count target |
| Relationships | All mandatory controlled problem/technique/case links present and supported; zero false `solves` or `contradicts` links |
| Retrieval | Correct source/node in top five for at least 80% of frozen questions, matching the PRD floor; no decline from comparable baseline |
| Answers | At least 90% required answer points covered; all factual answer clauses cited; zero material unsupported claims in the controlled set; derivations labeled |
| Source navigation | All controlled text links reach exact highlighted occurrence; all supported PDF targets open correct page; all unambiguous PDF mappings highlight correctly; ambiguous mappings visibly fall back; report fallback rate separately |
| Reader usefulness | All mandatory browse/apply/compare tasks succeed; owner can find a cited answer in under two minutes on the demo, matching PRD intent; human timing stays NOT RUN until observed |
| Latency and resources | No more than 20% p95 regression versus equivalent baseline for metadata/search reads; report rich-detail/viewer work separately; ingestion cost/time is measured, not hidden by fewer findings |
Latency uses a fixed seeded 1,000-node fixture and fixed hardware/configuration, 20 warmups followed by 200 requests per operation at concurrency 1 and 5. Measure cold-start/viewer first-open separately. Report sample count, p50/p95/max, response sizes, peak memory, index build time, and cache state. This is a bounded regression benchmark, not a production capacity claim.
Record agent wall time, tokens when available, tool calls, retries, accepted/held/rejected updates, coverage, and cost per supported learning unit. If billing data is unavailable, report cost UNKNOWN rather than estimating a measured value. Budgets must match baseline/candidate; exhaustion remains a measured failure/defer, not silent scope reduction.
All hard gates must pass in each fresh run; report aggregate quality and each run's result. A missing model, blocked native host, unavailable PDF environment, or unrun owner task stays BLOCKED/UNSUPPORTED/NOT RUN and prevents the corresponding qualification claim.
## Delivery sequence and ownership
| Task | Owner / boundary | Deliverable and exit gate |
| --- | --- | --- |
| Q1 | Backend + agent contracts | Freeze baseline, fixture/reference hashes, policies, budgets, model, and metrics; ADR for update/content/citation contracts and compatibility |
| Q2 | Backend domain/application/Git adapter | Stable-ID enrichment/correction, assessment history, conflicts/replay/atomicity; focused domain and crash/recovery tests pass |
| Q3 | Backend facets + GEWU_agent | Dedicated resolution/disposition flow; installed agent attaches/reuses labels and records provider failure without losing content |
| Q4 | Backend content/extraction + reader | Evidence-bound detailed content, math/figures, safe rendering and bounded retrieval; native source artifacts preserved |
| Q5 | Backend read API + reader | Shareable exact citation target, context/highlight, native PDF page mapping and explicit fallbacks; browser cases pass |
| Q6 | GEWU_agent repository | Coordinated protocols/pipelines/curation/query, generated contracts, manifest, installer compatibility, installed native-client tests |
| Q7 | Harness/tests | Private demo setup/credential lifecycle, baseline/candidate runner, same-model review passes, machine-readable scoring and failure injection |
| Q8 | Integrated release candidate | Full regression matrix, tuning, locked withheld evaluation, three fresh runs, owner demo and evidence report |
| Q9 | Curation rollout | Dry-run report against current 103-node snapshot; proposal batches enrich facets/details/evidence/edges and repair unsupported clauses; validate each accepted receipt |
Q2–Q6 are coordinated vertical slices: each includes its backend contract, pack consumer, reader behavior where applicable, and tests before being marked complete. Q7 fixture/baseline preparation starts with Q1; benchmark qualification waits for the integrated candidate. Implementation work is in GEWU and GEWU_agent feature branches; the final parent pointer pins the tested agent release. Do not fold unrelated dirty changes into this work.
## Migration, rollout, and rollback
Read old records with optional new fields absent. New operation/content support is capability/version gated. Additive index changes must rebuild cleanly from Git; any durable auxiliary assessment/classification data needs an export/restore contract rather than becoming hidden SQLite-only truth. Preserve immutable historical schemas or provide explicit versioned readers.
Before live curation, back up canonical Git, source artifacts/extractions, identity/workflow stores, and required auxiliary records; verify restoration in a disposable root. Build a dry-run per-node plan: enrich, clarify support, recapture original, attach facets, add edges, unresolved. Missing originals remain explicit; no automatic conversion of pasted text to PDF evidence.
Migrate through validated proposals in bounded batches, preserving source lineage and history. Verify canonical reads, rebuilt index, representative queries, source links, and receipts after each batch. Protect shared-project policy and owner-required decisions. The user has authorized planning and demo testing; live-corpus curation is a distinct execution phase of the plan, not something performed while drafting it.
Rollback means disable incompatible new writes, deploy the last compatible reader/service pair, and rebuild derived indexes from a recorded canonical snapshot in an isolated restore first. Do not delete accepted history or reset the live canonical branch to undo enrichment. Accepted content corrections use subsequent validated proposals. Record irreversible external/model costs separately from reversible code/config changes.
## Evidence and final report
Each run writes an exclusively created manifest with backend commit/snapshot digest, agent commit/manifest/installed hashes, source/reference/question hashes, policies, schema/parser/viewer versions, actual CLI model flags, retrieval/embedding pins, machine/runtime, budgets, and redacted configuration. Save immutable raw outputs, proposal receipts, before/after node revisions, deterministic and semantic judgments, scores, browser recordings/screenshots, timing/resource samples, and cleanup/revocation outcomes.
Reuse the existing manifest, citation-verification, retrieval, semantic-review, and orchestration scripts where their contracts fit. Extend them for stable updates, rich-content references, facets, source navigation, and required model pinning; do not assume the existing result-file scorer already drives a full native ingestion run. Keep runner execution, raw artifacts, and scoring separate so scores cannot manufacture missing evidence.
The final report compares baseline/candidate by requirement and metric, identifies remaining failures and fallbacks, and distinguishes implemented, locally tested, native-client verified, machine-assessed, and owner-observed outcomes. Link the exact evidence for every gate. Update the acceptance ledger and handbook only with the results actually obtained.