Yesterday, @Cypress_io introduced test retries into their newest release. Are they promoting bad testing practices?

Short answer: No.

Now for the long answer.

1. When talking about flakiness, we cannot pretend like it has a single cause. Of course, badly written test will always be a badly written test and you should put in your effort to fix that. Or maybe think about how much you need that test if you are jumping over hoops with it.

2. From day one, Cypress built their testing tool to be integrated within your application repo, with continuous integration setup in mind. Basically, e2e tests written in Cypress are a part of your test step in your pipeline.

Running e2e tests within your build pipeline is quite a powerful idea, but it is also an area where lot of things can go wrong. That is why teams are often hesitant to make e2e tests a part of their build.

You can see e2e tests being ran overnight, on separate testing server, basically divorced from your app. Too many things can go wrong and what you don’t want to do is to dig through a pile of false negatives that just make the whole company wait for your next release.

The way I see it, Cypress actually makes an active effort to improve quality of shipped apps, by optimizing for a usage in CI.

3. It made a less fuss, but couple of days ago, Cypress updated their dashboard with improved analytics. Analytics automatically look for the most flaky tests, or tests that run too slow.

Adding test retries to Cypress needs to be read in this context. What you try to do as a tester/developer is to optimize for speed. Make sure your tests run fast, so you can ship fast and with a great confidence. Analytics help you identify roadblocks.

Even better, if you see a test retrying, you can identify those even before it affects your build. Maybe it fails just once and then go green. But that way you gain two benefits - not break your build, and make sure to do some hardening your test so it does not go rogue.

Every app is different and all depends on context you are testing in. It may be a no-go for you to retry your tests. It may actually help a lot. BTW Google Chrome team retries their tests too (check minute 15)

And get this. It’s up to you whether you want to turn on retries. They are off by default. If you choose to turn them on, you can do it via config, you can choose how many times you want to do a retry or you can go really specific and just enable retry for a single test.

The cool thing about folks in Cypress is that they actually give you freedom to chose your path.

BTW if you are looking to learn Cypress, you can do so in my course. Or just DM me, I’m always happy to help. https://www.udemy.com/course/cypress-test-automation-for-people-in-a-hurry/?referralCode=21A875FF429457CF641D