Target models
Qwen3.8-MaxQwen3.8-Flash-NextQwen3.8-27BQwen3.8 FamilyFuture Qwen Models
Name
pytest
Category
Testing
Description
A pytest suite that stays fast and trustworthy — layout, fixtures with the right scope, parametrize instead of loops, markers that mean something, mocking only at boundaries you own, coverage as a diagnostic, async tests, and keeping the whole run under a minute.
License
MIT
Author
Agent.md maintainers
Last verified
2026-09-13
Reviewed by
unreviewed
#Task boundary
- Implement only what the task names; no extra abstractions or files.
- English-only comments and identifiers.
- Stop when the checklist passes.
#Purpose
Rules for organising and writing tests with pytest. What to assert is
Testing/unit; this package is about the harness: how tests are found, shared,
isolated and kept fast.
#Layout and configuration
bashtests/
conftest.py # shared fixtures, no tests
unit/test_money.py
integration/conftest.py test_orders_repo.py
e2e/test_checkout.py
toml[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q --strict-markers --strict-config -ra"
xfail_strict = true
filterwarnings = ["error"]
markers = ["slow: takes >1s", "integration: needs the database"]
--strict-markers: a typo'd@pytest.mark.integratonis an error, not a silently unselected test.filterwarnings = ["error"]: deprecation warnings fail the suite while they are cheap to fix.xfail_strict: anxfailthat starts passing fails, so fixed bugs get their marker removed.src/layout with the package installed editable, soimport ordersin tests is the installed package. →Backend/python-conventions
#Fixtures and scope
python@pytest.fixture(scope="session")
def engine(): # expensive: once per run
e = create_engine(TEST_URL); yield e; e.dispose()
@pytest.fixture
def session(engine): # cheap and isolating: once per test
conn = engine.connect(); tx = conn.begin()
s = Session(bind=conn, join_transaction_mode="create_savepoint")
yield s
s.close(); tx.rollback(); conn.close()
@pytest.fixture
def order(session) -> Order:
return OrderFactory(session=session)
- Default scope is
function. Widen it only for things that are expensive to build and immutable (an engine, a compiled schema, a loaded model). A session-scoped fixture that holds mutable state is order-dependent flakiness. - Fixtures compose:
orderdepends onsession, which depends onengine. Ask for what you need by name; do not build the world in one mega-fixture. yieldfixtures for teardown; the code afteryieldruns even when the test fails.conftest.pyper directory for fixtures that belong to that layer; a 600-line rootconftest.pyis a sign the layers are not separated.autouse=Trueis a global; use it for things every test genuinely needs (freezing the clock, disabling network) and nothing else.
#Parametrize, don't loop
python@pytest.mark.parametrize(("qty", "discount"), [
(0, 0), (1, 0), (9, 0), (10, 100), (11, 110),
], ids=["zero", "one", "below", "threshold", "above"])
def test_bulk_discount(qty: int, discount: int) -> None:
assert discount_for(qty) == discount
- One case per parameter set, each reported and re-runnable by id
(
-k threshold). Aforloop inside a test stops at the first failure and hides the rest. ids=for readable failures; the defaultqty0-discount0tells you nothing.pytest.param(..., marks=pytest.mark.xfail(reason="#123"))for a known failing case inside the table, instead of deleting it.
#Markers and selection
bashpytest -m "not slow and not integration" # the pre-commit run
pytest -m integration # CI job with the database
pytest --lf # only what failed last time
pytest -x --ff # stop at first, run failures first
- Declare every marker in config. Markers are the contract between the suite and CI; undeclared ones are noise.
@pytest.mark.skipif(sys.platform == "win32", reason="...")with a reason, always. A bareskipis a test that quietly stopped existing.
#Mocking at boundaries you own
pythondef test_sends_receipt(order, mailer_spy): # fake of OUR interface
send_receipt(order, mailer=mailer_spy)
assert mailer_spy.sent == [Receipt(order.id)]
def test_retries_on_timeout(monkeypatch):
monkeypatch.setattr(payments, "charge", AsyncMock(side_effect=[TimeoutError, "ok"]))
- Fake the interface you defined (
Mailer,PaymentGateway), not the vendor SDK. Mockingstripe.Charge.createtests your guess about Stripe's API. monkeypatchoverunittest.mock.patchdecorators: it undoes itself, works on env vars and attributes, and reads top-to-bottom.AsyncMockfor async callables; aMockreturns a non-awaitable and the test passes for the wrong reason.- Block real network in the suite (
pytest-socket, or an autouse fixture that raises onsocket.connect). A test that hits a real API is an outage waiting for CI. →Testing/unit
#Async tests
tomlasyncio_mode = "auto" # pytest-asyncio: every `async def test_*` just runs
pythonasync def test_fetch_times_out(monkeypatch):
with pytest.raises(TimeoutError):
async with asyncio.timeout(0.01):
await fetch_slow()
Await events and futures; never await asyncio.sleep(0.2) to let something
finish. Use pytest.raises with the specific exception, not Exception.
→ Backend/python-async
#Coverage and speed
bashpytest --cov=src --cov-report=term-missing --cov-fail-under=80 # a floor, not a target
pytest -n auto # pytest-xdist
pytest --durations=10 # find the slow ones
- Coverage is a diagnostic for untested code; 100% with assertion-free tests is worse than 70% with real assertions. Set a floor to stop regressions, then ignore the number.
- The unit run should finish in seconds. Anything slower is I/O leaking in:
find it with
--durations, mark itsloworintegration, move the real work behind a fake. -n autorequires tests that do not share state — which the fixture rules above already guarantee.- Freeze time (
freezegun/time-machine) and seed randomness; a test that depends on the wall clock fails at midnight.
#Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Undeclared markers | Typos silently unselect tests | --strict-markers |
| Warnings allowed | Deprecations pile up until removal | filterwarnings = ["error"] |
| Session-scoped mutable fixture | Order-dependent flakes | Function scope, or immutable |
One giant root conftest.py | Layers tangled | Per-directory conftest.py |
autouse for convenience | Hidden global behaviour | Explicit fixture args |
for loop over cases | First failure hides the rest | parametrize with ids |
Bare @pytest.mark.skip | Test silently vanishes | skipif with a reason |
| Mocking the vendor SDK | Tests an assumption | Fake your own interface |
mock.patch decorators stacked five deep | Unreadable, order-sensitive | monkeypatch |
Mock() for async code | Returns non-awaitable | AsyncMock |
| Real network in tests | Flaky, slow, outages | Block sockets |
asyncio.sleep() to wait | Slow and still flaky | Await an event |
pytest.raises(Exception) | Passes on the wrong error | Specific exception |
| Coverage as a target | Assertion-free tests | Floor + mutation testing |
| Unit run over a minute | I/O leaked in | --durations, mark and move |
| Wall clock in tests | Fails at midnight/DST | Freeze time |
#Checklist
-
pyproject.tomlsets--strict-markers,--strict-config,filterwarnings = ["error"],xfail_strict - Every marker declared and used to split unit from integration runs
- Fixtures default to function scope; wider scope only for immutable, expensive resources
- Database tests run inside a rolled-back transaction fixture
-
conftest.pyper layer;autousereserved for clock and network guards - Tables of cases use
parametrizewithids - Every skip has a reason; every
xfailreferences an issue - Fakes target interfaces we own; vendor SDKs are never mocked directly
-
monkeypatchandAsyncMockused; real network blocked -
asyncio_mode = "auto"; no real sleeps in async tests - Coverage floor enforced; number treated as diagnostic
- Unit run finishes in seconds;
-n autoworks - Time and randomness are controlled