Playwright Retries and Flaky Tests: Detect, Debug and Fix Them

Flaky tests destroy trust in automation. In this lesson you will configure retries the right way, detect unstable tests before they reach CI, debug failures with traces, and fix the root causes that make Playwright tests flaky.

---

🎲 What Is a Flaky Test?

A flaky test passes sometimes and fails sometimes without any change to the code. Flaky tests are dangerous: teams stop trusting red builds, start re-running pipelines “until green”, and real bugs slip through.

Playwright gives you tools to detect, diagnose and fix flakiness. Retries are part of that toolkit, but they are a safety net, not a cure.

---

🔁 Configure Retries

By default Playwright does not retry failed tests. Turn retries on in playwright.config.ts:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: process.env.CI ? 2 : 0,   // retry twice in CI, never locally
});

Or from the command line, or for one group of tests:

npx playwright test --retries=2

test.describe('payment gateway (third-party sandbox)', () => {
  test.describe.configure({ retries: 3 });
  test('card payment', async ({ page }) => { /* ... */ });
});

How retries work

When a test fails, Playwright discards the worker process and runs the test again in a fresh one, including its beforeAll hooks. The final result is one of:

Status	Meaning
passed	Passed on the first attempt
flaky	Failed at least once, then passed on a retry
failed	Failed on every attempt

The HTML report has a Flaky filter, and the terminal summary lists flaky tests separately. Treat every flaky test as a bug to fix.

---

🔍 Detect Flaky Tests Before They Reach CI

Run a test many times in a row to see whether it is stable:

npx playwright test tests/cart.spec.ts --repeat-each=10            # 10 runs per test
npx playwright test tests/cart.spec.ts --repeat-each=10 --workers=4 # also under parallel load
npx playwright test --fail-on-flaky-tests --retries=1              # fail the run if any test is flaky

---

🔬 Diagnose the Failure with Traces

Record a trace for the retry so you can see exactly what happened in the failed attempt:

use: {
  trace: 'on-first-retry',          // records a trace when a test is retried
  screenshot: 'only-on-failure',
  video: 'retain-on-failure',
},

Open it from the HTML report or with npx playwright show-trace. The trace shows every action, a DOM snapshot before and after, network calls and console errors, so you can compare a passing and a failing attempt. Screenshots and videos are covered in screenshots and video recording.

Inside a test you can also react to retries:

test('import report', async ({ page }, testInfo) => {
  if (testInfo.retry > 0) {
    console.log('Retry #' + testInfo.retry + ': cleaning up data from the previous attempt');
    // e.g. delete the half-imported report through the API
  }
  // ...
});

---

🧯 Common Causes of Flaky Tests and How to Fix Them

Cause	Symptom	Fix
Hard waits (waitForTimeout)	Passes on fast machines, fails on slow CI	Wait for a condition: web-first assertions, waitForResponse (waits)
Checking a value once	expect(await el.textContent()) fails while the page is updating	Use await expect(el).toHaveText(), which retries
Missing await	Random failures, actions out of order	Await every action; enable no-floating-promises (async/await)
Tests depend on each other	Pass in order, fail alone or in parallel	Make each test create its own data and state
Shared test data	Two parallel tests edit the same user or record	Unique data per test, e.g. add testInfo.workerIndex or a timestamp
Unstable locators	Element sometimes not found or wrong element clicked	Role and test-id locators, no nth() on changing lists (locators)
Animations and overlays	“Element intercepts pointer events”	Wait for the overlay to be hidden; never use force: true as a fix
Slow or unreliable third-party services	Timeouts on payment, maps, analytics	Mock those requests with page.route (Network Mocking lesson, Module 8)
Environment differences	Fails only in CI	Same browser versions, viewport, time zone and locale in config; run CI in Docker

Example: from flaky to stable

// ❌ Flaky: fixed sleep and a one-time check
await page.getByRole('button', { name: 'Search' }).click();
await page.waitForTimeout(2000);
expect(await page.locator('.result').count()).toBeGreaterThan(0);

// ✅ Stable: wait for the response, then a retrying assertion
const response = page.waitForResponse(r => r.url().includes('/api/search') && r.ok());
await page.getByRole('button', { name: 'Search' }).click();
await response;
await expect(page.locator('.result').first()).toBeVisible();

Example: unique test data per worker

test('register a new user', async ({ page }, testInfo) => {
  const email = 'user_' + testInfo.workerIndex + '_' + Date.now() + '@test.com';
  await page.goto('/register');
  await page.getByLabel('Email').fill(email);
  // ...
});

---

🏷️ Managing Known Flaky Tests

While you investigate, keep the main pipeline trustworthy:

• Mark the test with test.fixme() and a link to the ticket, or tag it @flaky and exclude it with --grep-invert @flaky (tags and grep).
• Give genuinely slow tests more time with test.slow() instead of raising the global timeout (annotations).
• Track flaky tests over time; a rising count is an early warning about the app or the environment.

test('export to PDF', { tag: '@flaky' }, async ({ page }) => {
  test.fixme(true, 'Flaky in CI, see JIRA QA-123');
  // ...
});

---

✅ Retry Policy Checklist

• retries: 0 locally so you notice flakiness immediately.
• retries: 1 or 2 in CI as a safety net, with trace: 'on-first-retry'.
• Review the flaky list in every report and fix the root cause.
• Consider --fail-on-flaky-tests for release pipelines.
• Never increase retries to hide a problem.

---

📝 Assignment

1. Set retries: process.env.CI ? 2 : 0 and trace: 'on-first-retry' in your config.
2. Write a test that fails randomly (for example expect(Math.random()).toBeGreaterThan(0.5)), run it with --retries=3 and find it under Flaky in the HTML report.
3. Run one of your real Sauce Demo tests with --repeat-each=10 --workers=4 and confirm it is stable.
4. Take this flaky snippet and rewrite it to be stable: await page.click('#load'); await page.waitForTimeout(3000); expect(await page.locator('#status').innerText()).toBe('Done');
5. Log testInfo.retry in a test and observe its value on each attempt.
6. Bonus: tag one test @flaky, exclude it with grepInvert in the config, and confirm it no longer runs.

---

❓ FAQ

🤔 How do I retry failed tests in Playwright?

Set retries in playwright.config.ts (for example retries: 2), pass --retries=2 on the command line, or use test.describe.configure({ retries: 2 }) for a group.

🤔 What does “flaky” mean in the Playwright report?

The test failed at least once and then passed on a retry. Playwright reports it separately so you can find and fix unstable tests.

🤔 How do I find flaky tests in Playwright?

Run them repeatedly with --repeat-each=10 (ideally with several workers), and check the Flaky filter in the HTML report from CI runs with retries enabled.

🤔 Should I use retries in Playwright?

Use a small number in CI as a safety net, combined with traces on retry. Keep retries at zero locally, and always fix the root cause of flaky tests.

🤔 What is the most common cause of flaky Playwright tests?

Timing: fixed sleeps, one-time checks instead of web-first assertions, and missing await. Replacing them with auto-waiting actions and retrying assertions fixes most flakiness.

---

🎯 Next Steps

• 🎭 Data-Driven (Parameterized) Tests
• 🎭 Playwright Waits Explained
• 🎭 Custom Fixtures with Examples
• 🎭 Playwright Tutorial with TypeScript