The Problem
Simple CI pipelines — “does it compile, do the unit tests pass” — catch a narrow class of issues. Performance regressions, security vulnerabilities, frontend degradation, and code style inconsistencies accumulate silently until they’re expensive to address. By the time a pattern of regressions is visible in production metrics, it represents weeks or months of accumulated changes.
The second problem is gate fatigue. Single-dimensional CI pipelines that developers learn to work around, ignore, or override are worse than no CI at all — they create false confidence.
The Solution
CI pipelines gating on six distinct quality dimensions simultaneously. Every dimension must pass. Nothing merges without clearing every gate.
Gate 1 — Static Analysis (PHPStan + PHPCS)
PHPStan at maximum level. PHPCS with a custom ruleset covering PSR-12 plus Magento-specific conventions (method naming, DI patterns, frontend template security). Zero tolerance policy: no new violations, no ratchet. The bar doesn’t move to accommodate a deadline.
Runs on changed files plus any files they call, catching violations introduced indirectly through interface changes.
Gate 2 — Unit & Integration Tests
Full PHPUnit suite with coverage requirements for modified files. New code without test coverage for changed classes doesn’t merge. Integration tests run against a real MySQL instance (not mocked) — the incident that created this requirement involved mocked tests passing while a real migration failed in production.
Gate 3 — APM Regression Check
New Relic performance baseline comparison. Key Magento controller and API endpoint response times benchmarked pre-PR. If the PR introduces measurable P95 regression against the baseline in a headless load test, the gate flags and requests justification before merge. Catches expensive queries, missing index usage, and N+1 patterns before they reach production load.
Gate 4 — Core Web Vitals Delta
Lighthouse CI runs against the staging deployment. LCP, CLS, and INP measured against baseline. Regressions beyond the defined threshold (currently: LCP +200ms, CLS +0.05, INP +50ms) block merge. Frontend performance no longer degrades silently between releases — every PR that ships frontend changes is accountable to measurable user experience metrics.
Gate 5 — Security Scanning
SanSec Ecomscan: Malware patterns, known web shells, credit card skimmer signatures. Runs against the full codebase on PRs targeting production.
Semgrep: Custom rulesets for Magento-specific security patterns — unescaped output in PHTML templates, SQL string concatenation, direct object reference without ACL, session fixation patterns. Runs on changed files per PR.
Composer audit: Known CVEs in installed PHP dependencies surfaced per PR. New vulnerable package versions block merge until assessed.
Gate 6 — Peer Review
Two-engineer sign-off for standard changes; three-engineer sign-off for changes touching checkout, payment flows, or authentication. Review comments must be resolved or explicitly acknowledged with reasoning — “LGTM” without engaging with open comments is not a valid sign-off.
The other five gates run in parallel while review is in progress. By the time review is complete, all automated gates have results. Engineers review already-validated code, not code they’re validating simultaneously.
Implementation
All gates run as parallel GitHub Actions jobs. The fastest (static analysis, security scan) complete within 90 seconds of PR open. The slowest (E2E tests, load test benchmark) complete within 10-12 minutes. Total wall-clock time from PR open to a first full result: under 12 minutes for most changes.
Failed jobs produce PR check annotations with finding details attached to the relevant lines. The developer sees exactly what failed and why, in the PR interface, without visiting a separate CI dashboard.
Impact
The shift is from quality assurance as a periodic activity to quality assurance as a continuous, per-change process. Issues caught in CI cost 5-15 minutes to fix. The same issues caught post-deployment cost hours in incident response, rollback overhead, and production impact.
The pipeline has fundamentally changed what gets to production: only code that has passed static analysis, test coverage, performance benchmarks, security scanning, and peer review.