Writing testable extensions

What furnace needs from an extension, and the mistakes that make a suite untestable on anything but its author's machine.

Put integration tests inside the module

app/code/<Vendor>/<Module>/Test/Integration/

Furnace installs module directories into a cell, by composer require or by copy. Tests at the repository root (dev/tests/integration/, tests/integration/) never arrive and cannot run. Across the Qoliber suites, 40 of 79 keep integration tests at the repository root and 9 keep them only there; those are the ones that go untested.

A run reports this rather than staying quiet, and the skip blocks certification:

- integration  tests live outside the module directories and are never installed
               into the cell: dev/tests/integration -- move them into the module
               (<Module>/Test/Integration) to run them

Unit tests live at Test/Unit and run from the source checkout; they need no cell.

Ship the tests in the released package

A release is judged by the tests it ships. With --release or a harness, the installed package's own Test/Integration runs against the installed code; a harness has no checkout to fall back to at all. A .gitattributes that export-ignores /Test therefore makes a package untestable from the registry. Either keep the tests in the dist (a few hundred kilobytes a customer never executes), or accept that the module is only ever certified from a checkout.

Declare no hardcoded version

Module composer.json should let the release tag define the version. A version key that disagrees with the tag makes Packagist and the file disagree about what shipped.

Typed properties in integration tests must be nullable

Magento 2.4.6 and 2.4.7 null every declared property of a test class after the suite, through reflection. A non-nullable typed property makes that fatal, and PHPUnit reports zero tests — indistinguishable from a suite that ships none.

private FpcProbe $probe;          // fatals on 2.4.6 / 2.4.7
private ?FpcProbe $probe = null;  // fine everywhere

private array $ids = [];          // same problem
private ?array $ids = [];         // widen the type, keep the default

2.4.8 onwards is unaffected, so this only appears on your oldest supported versions.

PHPStan will report the nullable type as unnecessary, because the null is assigned by reflection where static analysis cannot see it. Scope an ignore:

ignoreErrors:
    - identifier: property.unusedType
      path: app/code/*/Test/Integration/*

Do not call setAccessible()

A no-op since PHP 8.1 and deprecated in 8.5. Magento's test bootstrap turns deprecations into exceptions, so it becomes a hard failure on PHP 8.5 only.

Dispatch controllers through the framework's area

A controller test that calls execute() directly leaves the design area unset. 2.4.9 and MageOS tolerate it; 2.4.6 to 2.4.8 throw "Incorrect theme identification key".

Bootstrap::getInstance()->loadArea(\Magento\Framework\App\Area::AREA_FRONTEND);

Playwright suites must be portable

Two rules, both learned from suites that passed only on their author's machine.

Read the target from the environment. Furnace sets BASE_URL, ADMIN_USER, ADMIN_PASS and ADMIN_PATH. A config that pins baseURL to a literal is refused, because the same config serves every store view and a literal silently tests the wrong storefront.

// every project, not just the top-level `use`
baseURL: process.env.BASE_URL || 'https://fallback.test',

Seed the content your browser tests assert on

A spec that navigates to /faq/shipping/how-long-does-delivery-take assumes data that exists on no clean cell. Ship a seeder and furnace runs it in the cell, before the production gate, so the reindex that follows makes the content searchable:

tests/furnace/e2e-data.php

It bootstraps the cell named by FURNACE_CELL and creates whatever the suite needs, through Magento's own APIs:

$cell = getenv('FURNACE_CELL');
require $cell . '/app/bootstrap.php';

$om = \Magento\Framework\App\Bootstrap::create($cell, $_SERVER)->getObjectManager();
$om->get(\Magento\Framework\App\State::class)
   ->setAreaCode(\Magento\Framework\App\Area::AREA_ADMINHTML);

PHP rather than API calls, for a reason: store views cannot be created over REST, and a suite asserting on a de storefront needs one to exist.

Outside app/code, so it is never copied into a cell as module code and never ships to a customer. No teardown is needed — the next run restores the cell from its golden snapshot, which drops the database outright.

An extension without a seeder reports n/a, not a gap. Only a suite that asserts on data it never creates is incomplete, and that shows up as failing browser tests.