So obvious in retrospect
If you listen to the Cypress fail event without re-throwing then you swallow all test failures
"fun"
When reviewing #17919 I knew the Cypress tests would have to be failing which is what prompted me to check
Introduced in bbb7ed9 (July 10th!)