pytest Tutorial for Beginners: Test Discovery, assert and Running Tests

pytest is the test runner behind every Playwright Python suite in this course. Before going deeper into fixtures and plugins, it pays to understand pytest itself: how it finds tests, why a plain assert statement gives such helpful failure messages, how to test for exceptions and floating-point values, what each test outcome means, and how to read the results. Most examples in this lesson use plain Python so you can focus on pytest; the last one brings Playwright back in.

---

🤔 Why pytest?

Python ships with a test framework called unittest, modelled on Java’s JUnit: tests live in classes that inherit from unittest.TestCase and use methods such as self.assertEqual(). pytest, the most popular alternative, lets you write tests as plain functions with plain assert, and adds fixtures, parametrization and a large plugin ecosystem. It can also run existing unittest tests, so teams can switch gradually.

	unittest	pytest
A test is	A method in a TestCase subclass	Any test_ function (or method in a Test class)
Checking a value	self.assertEqual(a, b)	assert a == b
Setup and teardown	setUp / tearDown methods	Fixtures with yield
Plugins	Few	More than a thousand, including pytest-playwright

---

🔍 How pytest Finds Tests

When you run pytest, it walks through the current folder (or the testpaths you configured) and collects:

1. files named test_*.py or *_test.py;
2. inside them, functions whose names start with test;
3. classes whose names start with Test and that have no __init__ method, and inside them, methods whose names start with test.

Everything else is ignored. Helper functions without the test prefix are never run as tests, which is exactly what you want.

---

✅ Plain assert, Powerful Messages

Create tests/test_basics.py:

📄 tests/test_basics.py

import pytest


def to_price(text: str) -> float:
    """Convert '$29.99' to 29.99."""
    return float(text.replace("$", ""))


def test_price_is_parsed():
    assert to_price("$29.99") == 29.99


def test_total_with_tax():
    total = to_price("$29.99") + to_price("$9.99")
    assert total * 1.08 == pytest.approx(43.1784)


def test_bad_price_raises_value_error():
    with pytest.raises(ValueError, match="could not convert"):
        to_price("free")


def test_tags_contain_smoke():
    tags = ["smoke", "login", "regression"]
    assert "smoke" in tags
    assert len(tags) == 3

What each test shows:

• **assert expression**: if the expression is false, the test fails. You need no special methods; ==, in, <, is None and every other Python expression work.
• **pytest.approx()**: floating-point maths is not exact (0.1 + 0.2 is 0.30000000000000004), so compare floats with approx, which allows a tiny tolerance.
• **pytest.raises()**: a context manager that passes only if the code inside raises the given exception. match= checks the error message with a regular expression.

⌨️ Terminal

pytest -q tests/test_basics.py

🖥️ Output

....                                                                     [100%]
4 passed in 0.02s

What a failure looks like

pytest rewrites assert statements while importing test files, so when one fails it can show you the values involved, not just “AssertionError”. These tests fail on purpose (tests/test_failures.py):

📄 tests/test_failures.py

def test_compare_lists():
    expected = ["Backpack", "Bike Light", "Onesie"]
    actual = ["Backpack", "Bolt T-Shirt", "Onesie"]
    assert actual == expected


def test_compare_dicts():
    expected = {"browser": "chromium", "retries": 2, "headless": True}
    actual = {"browser": "chromium", "retries": 0, "headless": True}
    assert actual == expected

⌨️ Terminal

pytest -q tests/test_failures.py

🖥️ Output

_____________________________ test_compare_lists ______________________________

    def test_compare_lists():
        expected = ["Backpack", "Bike Light", "Onesie"]
        actual = ["Backpack", "Bolt T-Shirt", "Onesie"]
>       assert actual == expected
E       AssertionError: assert ['Backpack', ...rt', 'Onesie'] == ['Backpack', ...ht', 'Onesie']
E
E         At index 1 diff: 'Bolt T-Shirt' != 'Bike Light'
E         Use -v to get more diff

tests\test_failures.py:4: AssertionError
_____________________________ test_compare_dicts ______________________________

    def test_compare_dicts():
        expected = {"browser": "chromium", "retries": 2, "headless": True}
        actual = {"browser": "chromium", "retries": 0, "headless": True}
>       assert actual == expected
E       AssertionError: assert {'browser': '...adless': True} == {'browser': '...adless': True}
E
E         Omitting 2 identical items, use -vv to show
E         Differing items:
E         {'retries': 0} != {'retries': 2}
E         Use -v to get more diff

tests\test_failures.py:10: AssertionError
=========================== short test summary info ===========================
FAILED tests/test_failures.py::test_compare_lists - AssertionError: assert ['...
FAILED tests/test_failures.py::test_compare_dicts - AssertionError: assert {'...
2 failed in 0.06s

For the list, pytest pinpoints the first difference (“At index 1 diff”). For the dictionary, it hides the two identical items and shows only the key that differs: {'retries': 0} != {'retries': 2}. The long values are shortened with ...; add -v or -vv to see a full diff.

Shorter tracebacks: –tb

When many tests fail, long tracebacks are hard to scan. --tb controls them: --tb=short (shorter), --tb=line (the error plus one location line per failure) or --tb=no (none).

⌨️ Terminal

pytest -q --tb=line tests/test_failures.py

🖥️ Output

E   AssertionError: assert ['Backpack', ...rt', 'Onesie'] == ['Backpack', ...ht', 'Onesie']

      At index 1 diff: 'Bolt T-Shirt' != 'Bike Light'
      Use -v to get more diff
C:\pwv\pytest-tutorial-for-beginners\tests\test_failures.py:4: AssertionError: assert ['Backpack', ...rt', 'Onesie'] == ['Backpack', ...ht', 'Onesie']
E   AssertionError: assert {'browser': '...adless': True} == {'browser': '...adless': True}
      ...
C:\pwv\pytest-tutorial-for-beginners\tests\test_failures.py:10: AssertionError: assert {'browser': '...adless': True} == {'browser': '...adless': True}
2 failed in 0.01s

The code listings are gone; only the assertion message and the file and line of each failure remain.

---

🚦 The Five Test Outcomes

Every collected test ends in one of these outcomes. This file produces all of them (tests/test_outcomes.py):

📄 tests/test_outcomes.py

import sys

import pytest


@pytest.fixture
def broken_setup():
    raise RuntimeError("database not reachable")


def test_passes():
    assert 2 + 2 == 4


def test_fails():
    assert "Products" == "Product"


def test_errors_in_setup(broken_setup):
    assert True


@pytest.mark.skipif(sys.platform == "win32", reason="Linux-only check")
def test_skipped_on_windows():
    assert True


@pytest.mark.xfail(reason="known bug #123: cart badge shows 0")
def test_known_bug():
    assert 1 == 0

Outcome	Letter	Meaning
passed	.	All asserts held
failed	F	An assert or exception inside the test body
error	E	Something went wrong outside the test body, in a fixture’s setup or teardown
skipped	s	The test was not run (by marker or pytest.skip())
xfailed	x	Expected to fail, and did; xpassed (X) means it unexpectedly passed

The -r option prints a short summary for chosen outcomes; -rA means “all”:

⌨️ Terminal

pytest -q -rA tests/test_outcomes.py

🖥️ Output

.FEsx                                                                    [100%]
...
=========================== short test summary info ===========================
PASSED tests/test_outcomes.py::test_passes
SKIPPED [1] tests\test_outcomes.py:23: Linux-only check
XFAIL tests/test_outcomes.py::test_known_bug - known bug #123: cart badge shows 0
ERROR tests/test_outcomes.py::test_errors_in_setup - RuntimeError: database n...
FAILED tests/test_outcomes.py::test_fails - AssertionError: assert 'Products'...
1 failed, 1 passed, 1 skipped, 1 xfailed, 1 error in 0.07s

The progress line .FEsx shows one letter per test. The skip appears because this run was on Windows; on Linux that test would pass. pytest shortens long summary lines with ... to fit the terminal width.

The difference between failed and error matters when you read CI results: a failure usually means the application is wrong, an error usually means your test setup is broken. Markers such as skipif and xfail are covered in detail in pytest Markers.

---

🗃️ Grouping Tests in a Class

Classes are a simple way to group related tests. pytest creates a new instance for every test method, so tests never share instance attributes (tests/test_cart_class.py):

📄 tests/test_cart_class.py

class TestCart:
    items = []                          # class attribute: shared, avoid mutating

    def test_starts_empty(self):
        cart = []
        assert cart == []

    def test_add_item(self):
        cart = ["Backpack"]
        cart.append("Onesie")
        assert cart == ["Backpack", "Onesie"]

    def test_instances_are_separate(self):
        self.note = "set in this test only"
        assert hasattr(self, "note")

---

🎭 And with Playwright

Everything above applies unchanged to browser tests. The only addition is the page fixture (tests/test_ui.py):

📄 tests/test_ui.py

import pytest
from playwright.sync_api import Page, expect


def test_practice_page_heading(page: Page):
    page.goto("https://sqatools.in/automation-practice-page/")
    expect(page.get_by_role("heading", name="Web Table")).to_be_visible()


def test_table_has_three_rows(page: Page):
    page.goto("https://sqatools.in/automation-practice-page/")
    rows = page.locator("table tbody tr")
    assert rows.count() == 3                       # plain assert on a Python value
    with pytest.raises(AssertionError):
        expect(rows).to_have_count(4, timeout=500)  # expect raises AssertionError

• rows.count() returns a plain int, so a plain assert is fine.
• A failing expect() raises AssertionError, which is why pytest reports it as a normal test failure. Here pytest.raises proves it, using a short timeout to keep the demo fast.

---

🔢 Exit Codes for CI

pytest’s exit code tells CI servers what happened. This script runs pytest from Python and prints the code for three situations (exit_codes.py):

📄 exit_codes.py

import subprocess

PY = r"C:\pwv\_venv0\Scripts\python"
cases = {
    "all passed": ["tests/test_basics.py"],
    "some failed": ["tests/test_failures.py"],
    "no tests collected": ["tests/test_basics.py", "-k", "nothing_matches"],
}
for label, args in cases.items():
    code = subprocess.run([PY, "-m", "pytest", "-q", "-p", "no:cacheprovider", *args], capture_output=True).returncode
    print(f"{label:20} -> exit code {code}")

🖥️ Output

all passed           -> exit code 0
some failed          -> exit code 1
no tests collected   -> exit code 5

Code	Meaning
0	All collected tests passed
1	Some tests failed
2	Run interrupted (for example Ctrl+C or a collection error)
3	Internal pytest error
4	Command-line usage error (for example an unknown option)
5	No tests were collected

---

🧭 Everyday Command-Line Flags

Flag	Effect
-v / -q	More / less output
-k "login and not slow"	Select by name expression
-m smoke	Select by marker
-x / --maxfail=3	Stop after the first / third failure
--lf / --ff	Run only last failures / run them first
-r fE	Summary of failed and errored tests
--tb=short	Shorter tracebacks
--durations=5	Show the 5 slowest tests
--collect-only -q	List tests without running them
-s	Show print() output

---

❓ Frequently Asked Questions

🤔 Can pytest run my old unittest tests?

Yes. pytest collects unittest.TestCase classes automatically, so you can migrate step by step.

🤔 Why does pytest show values in assertion errors but plain Python does not?

pytest rewrites the assert statements in test modules (and conftest.py files) when it imports them, adding code that records the values. Plain Python only knows the assertion failed.

🤔 Should I use assert or expect in Playwright tests?

Use expect() for anything on the page, because it waits and retries. Use plain assert for values you already have, such as a count you read or data you computed.

🤔 What is the difference between a failure and an error?

A failure happens in the test body; an error happens in setup or teardown, usually in a fixture. Errors point at broken test infrastructure rather than a broken application.

---

🏁 Summary

• pytest collects test_*.py files, test functions and Test classes without __init__.
• Plain assert plus assertion rewriting gives detailed failure messages; use pytest.approx for floats and pytest.raises for exceptions.
• Outcomes are passed, failed, error, skipped and xfailed/xpassed; -rA summarises them.
• Exit codes (0, 1, 2, 3, 4, 5) tell CI what happened; 5 means nothing was collected.

Next, you will learn fixtures, the feature that makes pytest so powerful for setup, teardown and sharing resources such as browsers.