Files
GEWU/docs/acceptance/2026-10-14-hybrid-probe.md
2026-10-14 19:21:35 +08:00

3.9 KiB

Hybrid retrieval dependency probe

Status: implementation and compatibility evidence. This record does not meet the PRD's labeled top-five recall target or close FR-A2.

Runtime

  • Linux x86_64, Python 3.14.7, SQLite 3.53.4, CPU inference.
  • Disposable environment: sentence-transformers==6.1.0, torch==2.14.1+cpu, and sqlite-vec==0.1.9. These packages were not installed into the project runtime.
  • sqlite-vec extension loaded into an in-memory SQLite database and reported v0.1.9; a vec0 table accepted normalized float32 vectors and returned the nearest row first (distance 0.00612 versus 0.88957 for the second row).
  • Model: intfloat/multilingual-e5-small, repository revision 614241f622f53c4eeff9890bdc4f31cfecc418b3; local packaged tree SHA-256 b96279dfc10e6c6c05782feead2f0155d7e16975d3ebc440cda0c46b9a038094.
  • Output was 384-dimensional and normalized to norm 1.0. Both query and passage calls used the required query: and passage: prefixes.
  • English query/passage cosine: 0.84662. Chinese query/passage cosine: 0.94244. English query/Chinese passage cosine: 0.79640. This small hand-written probe is not a relevance benchmark and makes no recall claim.
  • End-to-end composition used the real local model with the Git, FTS, vector and application search adapters. It returned retrieval_mode=hybrid with four results. The vector generation and canonical index both reported revision 734c31c64a22803134a76bba30fec25c8f322c8f; model ID, artifact digest, content profile and dimensions matched, and both indexes used the same SQLite file.

Code delivered

src/gewu/application/ports.py defines the replaceable embedding port. src/gewu/adapters/embeddings.py loads only an already-installed local model, checks its tree digest, disables remote code and network loading, supplies E5 prefixes, and validates row count, dimensions, finite values, and normalization. src/gewu/adapters/sqlite_vectors.py stores scoped vectors in immutable generations, records revision/model/artifact/content-profile/dimension metadata, and atomically switches the active generation using compare-and-swap. Canonical FTS and vectors publish in the same SQLite transaction. Prior generations remain available for rollback, and active-generation deletion is rejected. The retrieval optional dependency set is pinned and included in uv.lock; sqlite-vec is also in the dev extra so CI exercises its native adapter. Unit tests use an injected encoder to verify model-adapter contracts without downloading weights. Vector-index tests cover project isolation, activation, rollback, and invalid-vector rejection (tests/adapters/test_sqlite_vectors.py). The application tests cover fusion, stable pages, cursor invalidation on model change, generation replacement, scope/filter enforcement, labeled degradation, and bounded one-hop graph expansion. They also verify that an injected failure after vector activation rolls back both vector and FTS publication (tests/application/test_hybrid_search.py).

Still required before hybrid acceptance

  • Expand citation/scope fixtures against the PRD's labeled corpus; current tests verify graph-origin citation propagation, filter enforcement and that cross-project neighbors do not escape scope.
  • Qualify old-citation stability across real source/parser revisions and measure cold re-embedding and generation publication against target hardware.
  • Run the PRD's labeled top-five recall set and language/corpus fixtures.
  • Package model assets for offline installation and probe the native vector runtime on Windows and macOS; qualify cold rebuild time on target hardware.

The model card documents the candidate model. The sqlite-vec project identifies its current pre-v1 compatibility risk; using it in the probe does not qualify it for production storage.