# Regression Test Promotion Playbook

This playbook documents the promotion, demotion, and quarantine rules for the Yorvana regression test suite based on stable scenario IDs.

## Flake-Rate Semantics

The following table defines how test outcomes are categorized to compute the nightly flake budget and pass rates:

| Outcome | Counts as | Description |
|---|---|---|
| Passed first try | pass | Test succeeded on the first attempt without retries. |
| Failed first try, passed on retry | **flake** | Test failed initially but succeeded on a retry (signals flakiness). |
| Failed all retries | failure | Test failed all retry attempts (counts against pass-rate, not flake rate). |
| `Assume.assumeTrue` skip | excluded from denominator | Test skipped cleanly due to missing environment pre-requisites. |
| `@Ignore` | excluded entirely | Test ignored / disabled at source-level. |
| Infrastructure failure | excluded | Emulator crash, OOM, or task timeout before the test ran. |
| Macrobenchmark task failure | excluded from flake rate | Recorded separately as a benchmark performance regression. |
| Run cancelled | excluded entirely | Manual cancellation or run superseded by a newer commit. |

---

## Promotion, Demotion, & De-Quarantine Policy

### 1. Promotion (RegressionFull -> Regression)
To promote a test from the extended/nightly-only tier (`@RegressionFull`) to the PR-gated core tier (`@Regression`):
- **Criterion**: Must achieve **100 consecutive nightly passes** on the stable scenario ID.
- **Runtime**: Median execution duration must be `< 30 seconds`.
- **Infrastructure**: No infrastructure noise events (e.g. emulator timeout, process crash) during the streak.

### 2. Demotion (Regression -> RegressionFull / Quarantined)
To demote an active test due to instability:
- **Criterion**: Any nightly test failure not reproduced as a real regression within **24 hours**, or runtime drift **> 50%**.
- **Action**: Demote the test to `@RegressionFull` or wrap it with `@Quarantined` if flakiness is high.

### 3. De-Quarantine Threshold
To promote a quarantined test back to its owning tier (`@Regression` or `@RegressionFull`):
- **Criterion**: Must achieve **30 consecutive green nightly runs** (either pass or flake) on the stable scenario ID.
- **Authority**: This document (Phase 7 #120) is the authoritative source for the de-quarantine threshold. If this threshold is changed, it must be updated here first, and corresponding sections in the main test plan (#116 / `docs/regression-tests.md`) must be synchronized in the same PR.