GEWU/docs/PRD.md

# Multi-Project Knowledge Base for Humans and AI Agents

Working name: KB | Version 0.2 (draft) | 12 Oct 2026 | Status: for review
## Summary

KB is a local-first, git-based knowledge base that turns collected material (web pages, papers, reviews, NotebookLM exports, talks, internal notes) into verified, linked, reusable knowledge. It serves two kinds of readers equally: people (through a web UI) and AI agents (through MCP and an HTTP API). One installation hosts many projects, each identified by a project_id (for example interactive-video, dome-performance, agentic-design). Projects keep their own goals, taxonomies and curated notes, and can draw on a shared library of sources, entities and concepts.
The core idea is three layers: immutable sources, atomic notes extracted from them with evidence, and compiled syntheses (briefs, market overviews, timelines, comparison tables) built from the notes. Agents write only by submitting proposals that pass a validator.
## Problem and opportunity

- Research for a project is scattered across browser tabs, PDFs, NotebookLM notebooks and chat histories, and is hard to reuse by a different person, agent or project.
- AI agents working on development, writing, presentations, budgets or marketing plans start each session without the project's accumulated knowledge, or get it as unverified summaries.
- Existing note tools store text but do not enforce provenance, evidence strength, freshness or typed relationships, so market numbers go stale, contradictions go unnoticed and claims lose their sources.
- Knowledge that applies to several projects (for example agentic design patterns used in both an interactive video tool and a dome show pipeline) is duplicated or lost.
## Goals and non-goals

Goals
1. One knowledge base, many projects. Every node belongs to exactly one project or to the shared library, and queries can be scoped accordingly.
2. Evidence-backed knowledge. Every claim traces to a source and a quote location that a validator can check.
3. Dual-audience access. Humans browse, learn and search in a UI. Agents retrieve cited, budgeted, structured answers through MCP and API.
4. Agent-agnostic. Works from OpenCode, Claude, Codex and Antigravity through shared files, skills and MCP, with no logic locked inside one tool.
5. Goal-aware ingestion. A local agent pipeline digests new material against the project's goal (three horizons) and proposes notes, links and insights.
6. Multi-audience value. The same nodes serve developers, product managers, project managers and sales through facets and saved lenses.
7. Trustworthy over time. Dated claims, staleness tracking, contradiction links and a full audit trail.
Non-goals (v1)
- Real-time multi-user collaborative editing (git-based collaboration only).
- Hosting copyrighted source texts for redistribution. KB stores notes, short quotes and links, and keeps archived copies private.
- Replacing project management, CRM or code search tools. KB feeds them knowledge but does not manage tasks or deals.
- A general-purpose chatbot. Agents use KB as a tool, and the UI is for navigation, review and reading.
- A story-map authoring editor (a canvas for drawing branching story flows) and an embedded in-UI copilot. Agents reach KB through MCP, and story authoring is a separate product.
## Users and use cases

User
Typical needs
Example
Developer
Problems and proven techniques, architecture precedents, case studies
How do interactive movies avoid combinatorial explosion of branches?
Product manager
Market shape, competitors, user problems, feature precedents, comparisons
Compare how three titles handled player choice pacing.
Project manager
History, timelines, cost and schedule datapoints, risks
Timeline of interactive film production tooling for a kickoff deck.
Sales
Market overview, positioning, proof points, objections, as-of numbers
Current market size claims and who made them.
Ingest agent
Digest a source against the project goal and propose notes
Process a new paper into proposals with evidence and edges.
Consumer agent
Retrieve cited knowledge during coding, writing, budgeting, marketing
Fetch a brief for a presentation with token budget 2,000.
Curator agent
Find stale, duplicate, contradictory or orphan nodes
Weekly report of stale market datapoints.
Project owner
Define the goal, review proposals, set autonomy levels
Approve new notes, update the north star.
## Multi-project model

This section defines how one database serves many projects.
### 5.1 Project registry

Each project has a registry entry projects/<project_id>/project.md with frontmatter:
- project_id: lowercase slug, unique, immutable (for example interactive-video, dome-performance, agentic-design).
- title, description, status (active, paused, archived), owners.
- north_star: the project's main goal (Horizon 3), human-owned. Agents may propose edits through review, never change it directly.
- key_questions, watch_list (competitors, technologies, titles, people), out_of_scope.
- audiences: which audiences the project cares about.
- taxonomy_extensions: project-specific domain tags and node subtypes added to the core vocabulary.
- autonomy_policy: per node type, one of suggest, auto-merge-safe, research-on-gap.
- default_includes: other projects or shared whose nodes appear in this project's searches by default.
Horizons 1 (current request) and 2 (session objective) are not stored in the registry. The agent derives them per task and records them in the run log.
### 5.2 Scope of every node

Every node has project_id, which is either a registered project or the reserved value shared.
Scope
What lives there
Rule
Project
Notes, syntheses, project-specific cases and datapoints
Owned by one project. Reviewed against that project's lens.
Shared
Sources, entities (companies, people, tools, titles), general concepts, cross-domain techniques
Deduplicated globally. Any project can reference them.
Rules:
1. Sources and entities default to shared, so one paper or company is stored once and referenced by many projects.
2. Notes default to the project that triggered ingestion. A note can be promoted to shared when it is useful beyond the project, through a review action that records who promoted it and why.
3. Cross-project links are allowed and explicit. A project node can link to a shared node or to a node in another project with a typed edge. The edge records the project pair, and the UI and API mark it as cross-project.
4. Relevance is per project. A node has lens_fit keyed by project: a note about agentic design patterns can carry a high score for agentic-design and a medium score for interactive-video, without being copied.
5. Default query scope is the active project plus its default_includes and shared. Other scopes are opt-in (projects=[...] or all).
### 5.3 Identity and layout

- Node IDs are globally unique and stable: <project_id>:<slug> for project nodes and shared:<slug> for shared nodes. Renames create redirects.
- Repository layout (single repo, namespaced):
kb/
  shared/{sources,entities,notes,synthesis}/
  projects/<project_id>/{project.md,notes,synthesis,staging,runs,lens/}
  schema/ protocols/ skills/ src/ docs/
- A separate repo per project is not used in v1, because shared sources and cross-project links would need duplication or submodules. The repo can still be split later because every node carries its project_id.
### 5.4 Isolation and export

- v1 is a single-owner or small-team setup. Isolation is logical: every API call takes an explicit scope, and the validator rejects nodes with an unknown project_id.
- Per-project export (nodes, linked shared nodes, and an index) must be possible so a project can be handed over or archived.
- Per-project access control (read, propose, merge) is a P2 requirement and the schema must not block it.
## Content model

### 6.1 Layers

1. Source (immutable): archived snapshot or reference, hash, bibliographic and reliability metadata. Never edited by agents.
2. Note (atomic): one finding, claim, problem, technique, case, datapoint or event, with evidence quote and location.
3. Synthesis (compiled): topic pages, briefs, market overviews, timelines and comparisons, regenerated from notes.
### 6.2 Node types

Source, Finding/Claim, Problem, Technique/Solution, Case, Entity, Datapoint, Event, Concept, Synthesis. Projects may add subtypes through taxonomy_extensions.
Project subtype packs. A project adds subtypes and fields through its registry, and the same validator checks them. The interactive-video pack starts with these four, taken from the deep-research report:
Subtype
Based on
Key fields
Production case
Case
release year, studio, delivery medium (FMV, real-time 3D, streaming player, theatrical), budget, reported revenue, units sold, total and average playthrough runtime, shooting ratio, decision points, endings
Narrative mechanism
Technique
paradigm (branch-and-bottleneck, state variable, storylet, database cinema, foldback), complexity class, authoring overhead, agency profile
Technical constraint
Problem
domain (buffering, state storage, bandwidth, platform parity), severity 1-5, mitigation
Market metric
Datapoint
metric type (market size, conversion, ticket premium, completion drop-off), demographic, value, unit, sample size, reporting year
Other projects define their own packs. All subtype fields follow the evidence rules in 6.3.
### 6.3 Metadata

All nodes: id, project_id, type, title, summary (1-2 sentences for agents to scan), created, updated, status (draft, reviewed, stale, superseded), confidence (low, medium, high), tags, schema_version.
Facets: audience (dev, product, pm, sales), domain (project-extensible core set: tech, design, market, history, legal, production, marketing), use_for (presentation, budget, comparison, architecture decision, pitch).
Lens fit: lens_fit: {<project_id>: {score, horizon, key_question, reason}}.
Provenance: derived_from: [{source, quote, loc}], run_id of the agent run that proposed it, agent and model identity.
Source-specific: URL, author, publisher, publication date, access date, source type (paper, review, press, vendor, internal), reliability grade, archive path, license note.
Claims and datapoints: evidence_quote (short, with page or timestamp), evidence_type (measured, reported, opinion, vendor marketing), as_of, stale_after.
Datapoint values: value, unit, sample_size, reporting_year and demographic (where relevant) are separate fields from the quote, so numbers can be compared and charted.
Entities: aliases (alternate names) and merged_from (IDs merged into this node, so a merge can be undone).
### 6.4 Typed edges

solves, causes, supports, contradicts, instance_of, part_of, precedes, supersedes, compared_with, derived_from, plus applies_to (node to project) for cross-project relevance. Edges live in frontmatter and are validated: the target must exist and the edge type must be allowed between the two node types.
Projects add extension edges in their registry, each with allowed source and target types. The interactive-video pack adds implements (case to mechanism), introduces (mechanism to constraint, a project-level form of causes), mitigates (technique to constraint), has_metric (case to datapoint) and incurs_cost (problem to datapoint). documented_in is the core derived_from edge. The core set stays small so other projects are not forced to carry these.
## Functional requirements

Priority: P0 required for v1, P1 important, P2 later.
### 7.1 Project management

ID
Requirement
Pri
FR-P1
Create, archive and list projects from a registry entry, and the validator rejects nodes with unregistered project_id.
P0
FR-P2
Every read and write API takes a project scope, with the documented default scope.
P0
FR-P3
Shared library with global dedupe of sources and entities.
P0
FR-P4
Promote a note to shared, and mark links as cross-project in API and UI.
P1
FR-P5
Per-project lens (north-star, key questions, watch list) read by ingest and curation.
P0
FR-P6
Per-project export and import.
P1
FR-P7
Per-project access control.
P2
### 7.2 Ingestion

ID
Requirement
Pri
FR-I1
Capture from URL, PDF, file, text paste and NotebookLM export through an inbox/ folder, CLI and API.
P0
FR-I2
Archive source snapshot, hash, normalize metadata, dedupe by hash and URL across the whole base.
P0
FR-I3
Triage against the project lens: relevant or skip, with score and reason.
P0
FR-I4
Digest extracts schema-valid notes with quote and location for every claim.
P0
FR-I5
Analyze writes learnings and points of interest per horizon, labeled as the agent's interpretation and kept separate from source claims.
P1
FR-I6
Link resolves entities to a canonical name plus aliases (name normalization and embedding similarity find candidates, the agent judges borderline matches), logs every merge so it can be undone, proposes typed edges and flags contradictions.
P0
FR-I7
Verify re-checks quotes, numbers and dates against the archived source and assigns evidence grade.
P0
FR-I8
All writes go to staging/ as proposals, with a run log.
P0
FR-I9
Tag each source with the projects it was ingested for, so one source can serve several projects.
P0
FR-I10
Task brief: agent derives H1 and H2 for each run and stores them in the run record.
P1
### 7.3 Validation and review

ID
Requirement
Pri
FR-V1
kb validate checks schema, required fields, link resolution, edge-type rules, project scope and that every quote appears in the archived source. A proposal that fails cannot be committed.
P0
FR-V2
Review inbox with accept, edit, reject and merge-duplicate actions. Decisions are logged and reused as examples for the skills.
P0
FR-V3
Autonomy policy per node type and per project, with graduated promotion from suggest to auto-merge once acceptance rate is demonstrated.
P1
FR-V4
Contradictions are linked, never overwritten. Superseded nodes stay with superseded_by.
P0
### 7.4 Retrieval and agent interface

ID
Requirement
Pri
FR-A1
MCP server and HTTP API expose: search, get_node, neighbors, timeline, compare, brief, list_projects, propose_note, flag_stale. All take project scope.
P0
FR-A2
Hybrid retrieval: keyword plus embeddings, then graph expansion by typed edges.
P0
FR-A3
Responses are structured JSON with citations, confidence, as_of and project of origin, bounded by a token budget parameter.
P0
FR-A4
brief(topic, audience, purpose, budget) returns a compiled, cited summary for one audience.
P1
FR-A5
Agents cannot write to notes/ or synthesis/ directly. They can only submit proposals.
P0
FR-A6
Single canonical AGENTS.md, with thin CLAUDE.md and GEMINI.md pointers, a shared skills folder, and an install script that links skills and MCP config into OpenCode, Claude, Codex and Antigravity.
P0
FR-A7
Optional hook integration so a coding harness can inject relevant KB nodes at turn start, with high-confidence full items and low-confidence hints.
P2
FR-A8
MCP resources (schema, project registry, node by ID) and prompts (per-project roles, for example an architecture advisor) alongside the tools.
P1
FR-A9
Typed tools only. No raw query language (Cypher, SQL) is exposed to agents, and every write goes through a proposal.
P0
### 7.5 Compilation and curation

ID
Requirement
Pri
FR-C1
Regenerate affected syntheses when notes change and show a diff for review.
P1
FR-C2
Curator agent reports stale nodes, duplicates, orphans, contradictions and cross-project promotion candidates.
P1
FR-C3
Staleness: nodes past stale_after are flagged in the UI and returned with a warning by the API.
P0
FR-C4
Optional topic-cluster summaries: detect clusters in the graph and write a summary per cluster for broad questions. Enabled only when a project outgrows its syntheses.
P2
### 7.6 Web UI

ID
Requirement
Pri
FR-U1
Project switcher, with an "all projects" view and clear marking of shared and cross-project items.
P0
FR-U2
Global search with filters for type, audience, domain, date, confidence and project.
P0
FR-U3
Node pages with summary, evidence, source links, typed neighbors and history.
P0
FR-U4
Problem to solutions to cases matrix, market datapoint table with as-of badges, timeline view, comparison view.
P1
FR-U5
Graph explorer.
P1
FR-U6
Review inbox and staleness list.
P0
FR-U7
Audience lenses: saved filters plus brief templates for developers, product, project managers and sales.
P1
FR-U8
Export a synthesis or timeline to slides or markdown.
P2
## Architecture summary

1. Source of truth: markdown and YAML files in git.
2. Derived index: SQLite (full text and vectors) or Postgres plus pgvector, with an edges table. It is disposable and can always be rebuilt from the files.
3. KB service: one core with MCP, HTTP and CLI transports derived from a single registry of tools. It is the only code path that reads the index and commits to the repo.
4. Skills: judgment procedures (capture, triage, digest, analyze, link, verify, compile, report), kept as one canonical set with a shared concerns to protocol mapping.
5. UI: a client of the same API with no privileged access.
6. Principle: deterministic strictness lives in code (validator, indexer), not in prompts, so different agents and models are held to the same floor.
Reuse from existing work: the one-core-with-thin-adapters and registry pattern, confidence tiers, observe-then-enforce promotion and acceptance receipts from Jev, and the skill template, state-machine pipelines and symlink-based cross-agent setup from Winnow.
Storage decision (v1). Files in git plus SQLite (full text, vectors and an edges table). A graph database such as Neo4j was evaluated and is not used in v1: it adds a server to run and back up, and it moves the source of truth out of git, which loses diffs, undo and direct file access for agents. A graph engine can be added later as a second derived index. Revisit when the base passes the 100,000-node target, when multi-hop path queries become too slow in SQLite, or when live multi-user editing is required. Also left out of v1: cloud document parsers (local-first), separate entity-tagging models and heavy entity-matching pipelines. Agents get typed tools only, never a raw query language.
## Non-functional requirements

Area
Requirement
Performance
Search under 300 ms for a project of 10,000 nodes on a laptop. brief under 5 s excluding model time.
Scale targets
10,000 nodes per project and 100,000 total, with full index rebuild under 10 minutes.
Local-first
Everything runs offline with local models as an option. Cloud models are configurable.
Portability
Plain files, no lock-in. Works on Windows, macOS and Linux. No required symlinks.
Reliability
Index is rebuildable. Every write is a git commit. Ingest runs are resumable from file artifacts.
Auditability
Every node records run, agent and model. Every proposal decision is logged.
Security
Source text is treated as data, never as instructions. Ingest agents have no write access beyond proposals. Secrets are kept out of the repo.
Legal
Store notes, short quotes and links. Archived copies stay private. License field on every source.
Versioning
schema_version on every node and run record, with migration scripts.
Agent-agnostic
No required use of tool-specific hooks, plugins or subagents in the core pipeline.
## Success metrics

Metric
Target
Citation validity (quotes found at stated location)
100% of merged notes
Proposal acceptance rate after tuning
At least 70% for notes, tracked per skill and per agent
Retrieval quality on a per-project question set of 20-30 questions
Correct source in the top 5 for at least 80% of questions
Cross-agent convergence
Same 5 documents through two different agents give validator-clean, mostly overlapping notes
Freshness
No datapoint older than its stale_after served without a warning
Reuse
At least one project reuses shared nodes from another project in its first month
Time to answer
A person finds a cited answer to a typical question in under 2 minutes
## Milestones

Milestone
Scope
Exit criteria
M0 Foundations
Node schema, edge vocabulary, project registry, validator, 20 hand-guided sources across two projects, starting with the deep-research report as the first ingest test (graded as a secondary, AI-synthesized source)
Validator catches seeded errors. Notes validate. Verify flags unsupported claims in the report.
M1 Service
Index, hybrid search, MCP and HTTP with project scoping, proposals to staging/
Agents can search and propose from all four tools.
M2 Ingest
capture, triage, digest, verify, link, run logs, task brief
Cross-agent convergence test passes.
M3 UI
Project switcher, search, node pages, review inbox, staleness
A person reviews and navigates without touching files.
M4 Synthesis
analyze, compile, brief, matrix, timeline, curator report
Briefs produced for each audience.
M5 Multi-project hardening
Promotion to shared, cross-project edges, export and import, acceptance receipts per agent
Three projects live on one base without cross-contamination.
## Risks and mitigations

Risk
Mitigation
Different agents extract inconsistent notes
Shared schema, validator gate, golden test documents, short example-driven skills
Hallucinated or misquoted claims
Quote-location validation, separate verify step, second agent for verification where possible
Low-value auto-captured notes
Lens-based triage, relevance thresholds, review-first autonomy
Projects pollute each other
Required project_id, scoped queries, per-project lens fit, explicit cross-project edges, promotion only through review
Duplicate sources and entities across projects
Global dedupe on hash, URL and entity resolution in the shared layer
Prompt injection from ingested sources
Treat source text as data, restrict ingest agent permissions, no direct writes
Market data goes stale
as_of and stale_after required for datapoints, curator reports, API warnings
Vendor or tool lock-in
Files plus MCP, no dependence on any one agent's hooks or plugins
Index drift
Index is derived and rebuilt from files, with a doctor command to check it
Over-building the stack (graph server, extra pipeline stages)
Start with files, SQLite and typed tools. Add a graph engine, clustering or heavier entity matching only when a revisit trigger in section 8 is hit.
## Open questions

1. Should KB live in its own repo or inside one of the existing agent repos?
2. Embedding model: local or API, and how to handle re-embedding when the model changes?
3. UI stack and hosting: local web app only, or also hosted for a team?
4. Do projects ever need hard isolation (separate access rules), or is logical separation enough?
5. Should shared notes require two-step review (project owner plus a shared curator)?
6. How should NotebookLM exports be captured: manual export, or an automated step?
7. Which node subtypes does each of the three starting projects need beyond the core set?
## Appendix A. Example project registry entry

---
project_id: interactive-video
title: Interactive video knowledge
status: active
north_star: <main goal, human-owned>
key_questions:
  - How do others control branch explosion at feature-film scale?
  - Who pays for interactive video, and how much?
audiences: [dev, product, pm, sales]
default_includes: [shared, agentic-design]
autonomy_policy: {source: auto-merge-safe, note: suggest, datapoint: suggest}
---
Other starting projects: dome-performance, agentic-design.
## Appendix B. Example note

---
id: interactive-video:state-variables-vs-branching
project_id: interactive-video
type: technique
title: Track state variables instead of multiplying branches
summary: Store choices as flags or values and vary scenes by state so paths do not multiply with each choice.
audience: [dev, product]
domain: [tech, design]
confidence: medium
status: reviewed
solves: [interactive-video:combinatorial-explosion]
lens_fit:
  interactive-video: {score: 0.9, horizon: H3, key_question: branch explosion}
  agentic-design: {score: 0.4, horizon: H2, reason: state tracking in agent workflows}
derived_from:
  - {source: shared:src-example-talk, quote: "...", loc: "t=14:32"}
schema_version: 1
---
## Appendix C. Agent tool signatures (illustrative)

search(query, project="interactive-video", scope="project+shared", filters={type, audience, domain, since}, limit=10)
get_node(id)
neighbors(id, edge_types=[], depth=2, scope="project+shared")
timeline(topic, project, from, to)
compare(ids)
brief(topic, project, audience, purpose, token_budget)
list_projects()
propose_note(project, node_bundle, run_id)
flag_stale(id, reason)
Resources: kb://schema, kb://projects/{project_id}, kb://nodes/{id}. Prompts: per-project roles, for example architecture_advisor(project_type, budget_tier) for interactive-video.