magento.works

I built magento.works to plan Magento upgrades on servers that are not containerised, where the database and search engine are whatever the host already runs and changing them is a project of its own. Adobe’s system requirements list the stack each release is tested on, not the full range it runs on. So “will Magento 2.4.9 run on what this box already has?” had no answer short of trying it.

The harness tries it, for every Magento 2.4.2 to 2.4.9 and Mage-OS 1.0 to 3.5 release, and publishes the results at magento.works.

#How does it test each release?

Each release gets a baseline stack, taken from Adobe’s or Mage-OS’s published requirements. The harness installs the baseline once, then swaps one service at a time to every other version it knows about: PHP 7.4 to 8.5, MySQL, MariaDB, Percona, Elasticsearch, OpenSearch, Redis, Valkey, RabbitMQ, ActiveMQ Artemis, Varnish, Apache and nginx.

A run passes when Magento installs, the smoke checks pass and three Playwright flows complete: storefront, guest checkout and admin. Every failure is classified by the Go runner as a compatibility, harness or infrastructure failure, so a broken harness never reads as a broken stack.

The site is static, and every page is also JSON:

curl -s https://magento.works/api/v1/health.json

{
  "status": "ok",
  "total": 4057,
  "passed": 2676,
  "failed": 1381,
  "lastRun": "2026-10-05T13:04:35Z"
}

The last full run went three installs at a time on my M4 Max MacBook, alongside everything else it was doing, from 30th September to 5th October 2026. That was 190.8 hours of cumulative step time. The median passing install took 100 seconds, smoke checks 46 and the Playwright flows 27.

#What has it found?

Caches, queues and Varnish ran on every release tested. The failures are in databases and search engines.

Out of 94 releases, with that one service swapped in (as of 5th October 2026):

• MariaDB 10.11 and Magento 2.4.8. Every 2.4.8 release up to p5 rejects MariaDB 10.11, the version Debian 12 and Ubuntu 24.04 install by default. 2.4.7-p6 and 2.4.9 both accept it. The allow-list in app/etc/di.xml dropped 10.11 between the tags, and the install stops with Current version of RDBMS is not supported.
• OpenSearch 2 and 3 on old releases. Magento 2.4.2 to 2.4.5 install cleanly on OpenSearch 2.19 and 3, then fail on the first product save, because they index through a mapping-type endpoint OpenSearch 2 removed.
• Patches are the norm. 1,862 of the 2,676 passes needed at least one patch or workaround. The libxml2 XSD fix alone (magento/magento2@308e261, by Pieter Hoste) was needed on 2,181 runs. All twelve are listed in the repo’s workarounds.json.

#History

• 18th May 2026: first commit, v0.0.1.
• 30th September 2026: launched as magento.works.
• 5th October 2026: full matrix rerun finished, 4,057 installs.
• 7th October 2026: v0.0.13.

#What has it not tested?

Combinations. Each run swaps one service away from the baseline, so two non-baseline versions together have not been run as a set.

Results also come from Alpine (musl) PHP images only. The site is still marked alpha.