# What the verdict tests

A verdict is the answer to one question: did this strategy earn its result, or did the search find luck? A run passes through stages, cheap ones first. A strategy stops at the first stage it fails, and the verdict names that stage.

## The stages

1. **Cheap checks.** Do costs leave any profit, are there enough trades to say anything, and is the strategy's list of settings short enough to trust? A strategy that fails here never reaches the expensive stages.
2. **Hold-out.** The last part of the history is kept back while the search runs. The strategy must still work on it. See [the hold-out test](https://oathra.dev/docs/concepts/verdict-checks/hold-out).
3. **Rolling consistency.** The earlier history is cut into consecutive windows. The strategy must work in most of them, not in one lucky stretch. See [the rolling test](https://oathra.dev/docs/concepts/verdict-checks/k-fold).
4. **The expensive tier.** Four tests that ask how likely it is that the winner is an accident:
   - [The probability of backtest overfitting](https://oathra.dev/docs/concepts/verdict-checks/chance-this-is-luck): how often the best setting of one half of the history is below average on the other half.
   - [The deflated Sharpe ratio](https://oathra.dev/docs/concepts/verdict-checks/deflated-sharpe): the Sharpe ratio, reduced for how many things were tried.
   - **Plateau:** whether the neighbours of the winning settings also work, or only that exact point.
   - [Walk-forward](https://oathra.dev/docs/concepts/verdict-checks/walk-forward): settings chosen on one window, then tried on the next, again and again.

The cost of the trades is tested again with wider spreads. A strategy that dies when the spread triples was living on the spread. See [the cost stress](https://oathra.dev/docs/concepts/verdict-checks/cost-check).

> **Trap.** Trying more settings makes the best result look better and be less reliable. The count of what was tried is part of the verdict.

## What the words on a result mean

A result says Validated when a candidate cleared every stage. Otherwise it says where the strategy was killed, for example at the gates, at the hold-out or at the expensive tier. Killed is a result, not an error: it tells you which question the strategy could not answer.

> **Trap.** A strategy that passes is not a promise of future profit. It says the result was not explained by luck or by the search alone.