Why run Screenshot Tests with Screenshotbot.io
In the previous introduction, we talked about why use Screenshot tests at all.
Perhaps you're now convinced that Screenshot Testing is a good idea to add to your test pyramid. But now let's see how that's typically implemented (outside of Facebook).
you would write your tests, start an emulator, and run
:recordScreenshots. This would store all the screenshots in your
repository, that you then have to commit. This can already be
problematic for some people because that can be a lot of screenshots
to store which can make your repository large and slow.
But that's not all. You now have to make sure all your developers
check against the recorded screenshots with
:verifyScreenshots. So you have this new developer join the team, and
they run that command on an emulator of their choice, and they'll find
that there's a mismatch. They complain to you, and you'll tell them
well, you need to run the tests on a precisely identical emulator:
same screen size, density, locale.. otherwise they'll get a mismatch.
Dealing with dependencies
As your company grows, you will many libraries and dependencies. A single change in one small core dependency will force your developers to re-record screenshots on almost of your codebase. This is exhausting, and you will most like not be paying attention to the actual changes or getting the changes reviewed by the team that owns the screenshots.
Screenshotbot solves this by assuming that most changes are intentional, not regressions.
So when something changes, we'll gently create a task and just let you (and the owners of the screenshot) know that something has changed. You can close out the task, you don't have to explain each and every change you made. If you think a task is actually a bug, then you can go ahead and start addressing it, but say, reverting the change or following up with the author.
This keeps everybody happy and productive, while still catching all of your regressions. Your happy and productive engineers are more likely to write new tests.