91 lines
3.7 KiB
Markdown
91 lines
3.7 KiB
Markdown
|
---
|
||
|
|
name: database-design
|
||
|
|
description: >
|
||
|
|
Designs high-performance relational schemas, SQLite WAL storage, repository data access patterns,
|
||
|
|
composite indexing, and entity models. Use when creating or modifying database tables, defining entity models,
|
||
|
|
optimizing query performance, or planning migrations.
|
||
|
|
concerns: [system-one-balance]
|
||
|
|
---
|
||
|
|
|
||
|
|
# Database Design & Storage Architecture
|
||
|
|
|
||
|
|
Governs persistence, entity modeling, and high-concurrency relational/SQLite storage patterns across backend projects.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Phase 0: Setup and Protocols
|
||
|
|
|
||
|
|
1. **Storage Invariants**:
|
||
|
|
- **SQLite WAL Mode**: Always enable Write-Ahead Logging (`PRAGMA journal_mode=WAL; PRAGMA synchronous=NORMAL;`) for concurrent read/write throughput.
|
||
|
|
- **Connection Scoping**: Thread-safety and connection pooling must be explicit. Never share open database cursor connections across concurrent threads or asynchronous tasks.
|
||
|
|
- **Minimal Transaction Footprint**: Prepare data and run validation *before* opening a transaction. Commit writes swiftly to minimize table/row lock contention.
|
||
|
|
- **Entity Separation**: Keep domain entities (dataclasses, Pydantic models) decoupled from low-level SQL row drivers. Repositories map raw rows to domain objects.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Phase 1: Ingest and Check
|
||
|
|
|
||
|
|
1. **Model Requirements**:
|
||
|
|
- Identify entities, relations (1:1, 1:N, N:M), cascade behaviors, and foreign key constraints.
|
||
|
|
- Determine write concurrency levels and access patterns (read-heavy vs. write-intensive).
|
||
|
|
- Identify query filter combinations to determine mandatory indexing.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Phase 2: Processing (Core Logic)
|
||
|
|
|
||
|
|
### Step 1: Entity Definition
|
||
|
|
Define pure domain entities decoupled from SQL execution:
|
||
|
|
```python
|
||
|
|
@dataclass
|
||
|
|
class UserSession:
|
||
|
|
id: str
|
||
|
|
user_id: str
|
||
|
|
token_hash: str
|
||
|
|
created_at: datetime
|
||
|
|
expires_at: datetime
|
||
|
|
```
|
||
|
|
|
||
|
|
### Step 2: DDL & Migration Schema
|
||
|
|
Author DDL with strict types, primary keys, and foreign keys:
|
||
|
|
```sql
|
||
|
|
CREATE TABLE IF NOT EXISTS user_sessions (
|
||
|
|
id TEXT PRIMARY KEY,
|
||
|
|
user_id TEXT NOT NULL,
|
||
|
|
token_hash TEXT NOT NULL,
|
||
|
|
created_at TEXT NOT NULL,
|
||
|
|
expires_at TEXT NOT NULL,
|
||
|
|
FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
|
||
|
|
);
|
||
|
|
```
|
||
|
|
|
||
|
|
### Step 3: Composite Indexing
|
||
|
|
Create composite indexes on high-cardinality filters and join columns:
|
||
|
|
```sql
|
||
|
|
CREATE INDEX IF NOT EXISTS idx_user_sessions_user_expires
|
||
|
|
ON user_sessions (user_id, expires_at);
|
||
|
|
```
|
||
|
|
|
||
|
|
### Step 4: Query Optimization & Pagination
|
||
|
|
- **Parameterized SQL**: Always bind query parameters with `:named` or `?` syntax. String interpolation or raw concatenation in SQL statements is strictly prohibited.
|
||
|
|
- **Clamped Limits**: Never execute unbounded `SELECT *` queries. Enforce default limits (`1 <= limit <= 50`).
|
||
|
|
- **Keyset Pagination**: For deep feeds or log tables, prefer keyset pagination (`WHERE id > :last_id ORDER BY id ASC LIMIT :limit`) over high-offset scans (`OFFSET 10000`) which cause linear disk scans in relational engines.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Phase 3: The System One Balance Pass
|
||
|
|
|
||
|
|
- **Augment, Don't Handcuff**:
|
||
|
|
- Keep migration scripts straightforward and forward-compatible.
|
||
|
|
- Do NOT create premature indexes on low-cardinality flags (e.g. `is_active` boolean columns) where table scans are faster.
|
||
|
|
- Choose the simplest storage architecture that satisfies requirements (e.g. SQLite WAL is sufficient for thousands of writes/sec without external DBMS overhead).
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Phase 4: Output Execution & Verification
|
||
|
|
|
||
|
|
1. Verify:
|
||
|
|
- Foreign key constraints enabled (`PRAGMA foreign_keys=ON;`).
|
||
|
|
- All parameters sanitized via parameterized queries.
|
||
|
|
- Composite indexes cover common query filters and sorting keys.
|
||
|
|
2. Deliver the entity definition, DDL schema, repository access methods, and migration step.
|