49 lines
2.3 KiB
Markdown
49 lines
2.3 KiB
Markdown
|
---
|
|||
|
|
name: benchmark-refactor
|
|||
|
|
description: >
|
|||
|
|
Measure baselines, evaluate optimization candidates in isolated branches, and record MADRs for performance-critical refactoring.
|
|||
|
|
scope: global
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Benchmark-driven refactoring
|
|||
|
|
|
|||
|
|
**"The model proposes, but the engine measures."**
|
|||
|
|
Optimization and refactoring must never rely on theoretical claims or synthetic
|
|||
|
|
unit test speedups. Every performance-critical change must be empirically proven
|
|||
|
|
against a deterministic baseline.
|
|||
|
|
|
|||
|
|
## 4-Phase refactoring workflow
|
|||
|
|
|
|||
|
|
### 1. Baseline profiling
|
|||
|
|
Before altering any production code, establish and record the empirical baseline:
|
|||
|
|
1. Select the profiling harness appropriate for the stack:
|
|||
|
|
- Python: `pytest-benchmark`
|
|||
|
|
- Rust: `Criterion.rs`
|
|||
|
|
- CLI / Compiled binaries: `hyperfine`
|
|||
|
|
- C / C++: `google/benchmark`
|
|||
|
|
2. Run the suite under clean conditions to capture mean execution duration ($\mu$),
|
|||
|
|
standard deviation ($\sigma$), memory overhead, and throughput.
|
|||
|
|
3. Export the baseline as a machine-readable JSON artifact (e.g. `benchmark-baseline.json`).
|
|||
|
|
|
|||
|
|
### 2. Hypothesis & candidate generation
|
|||
|
|
1. Formulate 2–3 discrete optimization strategies (e.g. vectorization, streaming iterators,
|
|||
|
|
in-memory caching).
|
|||
|
|
2. Implement each candidate in an isolated Git worktree or ephemeral branch
|
|||
|
|
(`candidate-a`, `candidate-b`). Do not mutate the baseline tree.
|
|||
|
|
|
|||
|
|
### 3. Differential benchmarking & elimination
|
|||
|
|
1. Execute the benchmark suite against each candidate under identical conditions.
|
|||
|
|
2. Apply strict statistical thresholds:
|
|||
|
|
- Candidate is valid only if it demonstrates statistically significant gain
|
|||
|
|
($p < 0.01$ and $\ge 15\%$ improvement in throughput or latency).
|
|||
|
|
- Zero regression across all unit, integration, and type checks.
|
|||
|
|
3. Immediately eliminate and hard-reset candidates that fail tests or fall below
|
|||
|
|
the performance threshold.
|
|||
|
|
4. Select the winning candidate balancing performance gain against code complexity.
|
|||
|
|
|
|||
|
|
### 4. Automated Architecture Decision Record (MADR)
|
|||
|
|
Refactoring without documentation produces code that future agents will revert.
|
|||
|
|
Formalize the decision by generating a Markdown Architectural Decision Record in
|
|||
|
|
`docs/adr/NNNN-[slug].md` using [the MADR template](references/madr-template.md).
|
|||
|
|
Commit the MADR alongside the refactored code and benchmark JSON outputs.
|
|||
|
|
Follow [shared verification](../../protocols/workflow.md#verification).
|