Flaky Test Calculator
Calculate flaky test rate and estimated CI/CD time wasted per month from test run statistics. Quantify the cost of flaky tests in your pipeline.
How to Use the Flaky Test Calculator
- Enter total CI test runs in your measurement period.
- Enter number of flaky (non-deterministic) runs.
- Enter total test suite size.
- Click Calculate to see flake rate and CI time wasted.
Casos de Uso
- •Quantifying the business cost of flaky tests.
- •Building the case for flaky test remediation sprints.
- •Setting flake rate SLAs for CI/CD pipelines.
- •Tracking improvement as flaky tests are fixed.
Fórmula
Flake rate = flaky runs / total runs × 100%. CI impact = flaky runs × suite size × 5 min avg / 60 / 30 hours/month.
Preguntas Frecuentes
What is a flaky test?
A flaky test passes and fails non-deterministically without code changes, causing unreliable CI/CD pipeline results.
What flake rate is acceptable?
Below 1% flake rate is generally acceptable. Above 5% significantly impacts developer productivity.
How do I fix flaky tests?
Identify and fix timing issues, add proper waits, isolate test state, use deterministic test data, and fix async issues.