# Benchmark Regression Detection (INT-13)

This document describes the CI infrastructure for detecting performance regressions in clawhdf5 benchmarks.

## Overview

Performance regressions can degrade user experience and increase operational costs. This system enables automated detection of regressions >5% in key benchmarks, with early warning before changes merge.

## Scripts

### benchmark-regression-check.sh

Located at `scripts/benchmark-regression-check.sh`, this script:

1. Runs the full benchmark suite (`cargo bench --no-fail-fast`)
2. Compares results against a baseline (`BENCHMARKS_BASELINE.json`)
3. Reports regressions exceeding the threshold
4. Exit code 0 = no regressions, 1 = regression detected

**Usage:**
```bash
./scripts/benchmark-regression-check.sh
# or with custom threshold
THRESHOLD=10 ./scripts/benchmark-regression-check.sh
```

## CI Integration

Add to your CI workflow (GitHub Actions, CircleCI, etc.):

```yaml
- name: Check benchmark regressions
  run: ./scripts/benchmark-regression-check.sh
  env:
    THRESHOLD: 5  # Allow up to 5% regression
```

## Baseline Management

The baseline is stored in `BENCHMARKS_BASELINE.json`. To update:

```bash
./scripts/benchmark-regression-check.sh  # Creates new baseline if none exists
git add BENCHMARKS_BASELINE.json
git commit -m "Update benchmark baseline"
```

## Regression Policy

- **Threshold:** 5% by default (configurable via `THRESHOLD` env var)
- **Action:** CI fails if regression exceeds threshold
- **Approval:** Regressions can be approved by:
  - Performance review of the code change
  - Documentation in the PR explaining the tradeoff
  - Deliberate update to the baseline after review

## Key Benchmarks

Focus areas for regression detection:

- `clawhdf5::read_f64` — main read path performance
- `clawhdf5::chunked_read` — chunked dataset reads
- `clawhdf5::filter_decompress` — decompression overhead (INT-07)
- `clawhdf5::alignment_check` — zero-copy alignment validation (INT-05)

## References

- BENCHMARKS.md — comprehensive benchmark suite documentation
- arXiv:2206.14761 — reasoning on benchmark methodology
- INT-05, INT-07 — performance items these regressions detect