feat: refresh shared skills catalog
This commit is contained in:
1 parent
98bb35e6fb
commit
418792d1ec
49 files changed
+2273
-2390
No files matched your search
@@ -1,91 +1,31 @@
|
||||
---
|
||||
name: database-design
|
||||
description: >
|
||||
Designs high-performance relational schemas, SQLite WAL storage, repository data access patterns,
|
||||
composite indexing, and entity models. Use when creating or modifying database tables, defining entity models,
|
||||
optimizing query performance, or planning migrations.
|
||||
concerns: [system-one-balance]
|
||||
Design schemas, migrations, and queries or diagnose query performance using the project database and measured workload.
|
||||
scope: global
|
||||
---
|
||||
|
||||
# Database Design & Storage Architecture
|
||||
# Database design
|
||||
|
||||
Governs persistence, entity modeling, and high-concurrency relational/SQLite storage patterns across backend projects.
|
||||
1. Inspect the engine/version, existing models, migration tooling, queries, expected
|
||||
volume, concurrency, and durability requirements.
|
||||
2. Define keys, constraints, relationships, deletion behavior, and transaction
|
||||
boundaries. Bind values using driver parameters; validate dynamic identifiers
|
||||
separately. Keep consistency-sensitive checks and writes in one transaction.
|
||||
3. Design indexes around actual filters, joins, and ordering; inspect query plans
|
||||
when optimizing. Cardinality alone does not determine whether an index helps.
|
||||
4. Bound interactive list queries using product-appropriate limits. Use stable
|
||||
keyset ordering for deep pagination; exports and internal batch jobs may stream
|
||||
larger datasets. Do not impose pagination on every query.
|
||||
5. Use the existing migration mechanism and plan data/backward compatibility.
|
||||
Verify affected reads, constraints, and migration behavior under
|
||||
[shared verification](../../protocols/workflow.md#verification).
|
||||
|
||||
---
|
||||
## SQLite when applicable
|
||||
|
||||
## Phase 0: Setup and Protocols
|
||||
|
||||
1. **Storage Invariants**:
|
||||
- **SQLite WAL Mode**: Always enable Write-Ahead Logging (`PRAGMA journal_mode=WAL; PRAGMA synchronous=NORMAL;`) for concurrent read/write throughput.
|
||||
- **Connection Scoping**: Thread-safety and connection pooling must be explicit. Never share open database cursor connections across concurrent threads or asynchronous tasks.
|
||||
- **Minimal Transaction Footprint**: Prepare data and run validation *before* opening a transaction. Commit writes swiftly to minimize table/row lock contention.
|
||||
- **Entity Separation**: Keep domain entities (dataclasses, Pydantic models) decoupled from low-level SQL row drivers. Repositories map raw rows to domain objects.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Ingest and Check
|
||||
|
||||
1. **Model Requirements**:
|
||||
- Identify entities, relations (1:1, 1:N, N:M), cascade behaviors, and foreign key constraints.
|
||||
- Determine write concurrency levels and access patterns (read-heavy vs. write-intensive).
|
||||
- Identify query filter combinations to determine mandatory indexing.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Processing (Core Logic)
|
||||
|
||||
### Step 1: Entity Definition
|
||||
Define pure domain entities decoupled from SQL execution:
|
||||
```python
|
||||
@dataclass
|
||||
class UserSession:
|
||||
id: str
|
||||
user_id: str
|
||||
token_hash: str
|
||||
created_at: datetime
|
||||
expires_at: datetime
|
||||
```
|
||||
|
||||
### Step 2: DDL & Migration Schema
|
||||
Author DDL with strict types, primary keys, and foreign keys:
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS user_sessions (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
token_hash TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
expires_at TEXT NOT NULL,
|
||||
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
### Step 3: Composite Indexing
|
||||
Create composite indexes on high-cardinality filters and join columns:
|
||||
```sql
|
||||
CREATE INDEX IF NOT EXISTS idx_user_sessions_user_expires
|
||||
ON user_sessions (user_id, expires_at);
|
||||
```
|
||||
|
||||
### Step 4: Query Optimization & Pagination
|
||||
- **Parameterized SQL**: Always bind query parameters with `:named` or `?` syntax. String interpolation or raw concatenation in SQL statements is strictly prohibited.
|
||||
- **Clamped Limits**: Never execute unbounded `SELECT *` queries. Enforce default limits (`1 <= limit <= 50`).
|
||||
- **Keyset Pagination**: For deep feeds or log tables, prefer keyset pagination (`WHERE id > :last_id ORDER BY id ASC LIMIT :limit`) over high-offset scans (`OFFSET 10000`) which cause linear disk scans in relational engines.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: The System One Balance Pass
|
||||
|
||||
- **Augment, Don't Handcuff**:
|
||||
- Keep migration scripts straightforward and forward-compatible.
|
||||
- Do NOT create premature indexes on low-cardinality flags (e.g. `is_active` boolean columns) where table scans are faster.
|
||||
- Choose the simplest storage architecture that satisfies requirements (e.g. SQLite WAL is sufficient for thousands of writes/sec without external DBMS overhead).
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Output Execution & Verification
|
||||
|
||||
1. Verify:
|
||||
- Foreign key constraints enabled (`PRAGMA foreign_keys=ON;`).
|
||||
- All parameters sanitized via parameterized queries.
|
||||
- Composite indexes cover common query filters and sorting keys.
|
||||
2. Deliver the entity definition, DDL schema, repository access methods, and migration step.
|
||||
WAL can improve concurrent reads/writes on supported local storage; it is not a
|
||||
universal engine choice or a promise of a particular write rate. Choose `FULL` for
|
||||
power-loss durability, or `NORMAL` only when its durability tradeoff is acceptable.
|
||||
Enable foreign keys per connection, use explicit connection lifetimes, and choose
|
||||
busy handling/checkpoint behavior for the workload. Consult the installed version's
|
||||
[pragma documentation](https://www.sqlite.org/pragma.html) for exact semantics.
|
||||
Reference in new issue
Block a user