Swift snapshot tests without Git LFS

Point-Free's swift-snapshot-testing is the most popular snapshot testing library for iOS. A single assertSnapshot call renders a view or view controller, and compares it against a reference image recorded earlier.

Those reference images are PNGs in a __Snapshots__ directory next to each test file, and they're meant to be committed to your repository. Once the images start to add up, most teams move them into Git LFS. For a handful of snapshots that works fine. As your team writes more snapshot tests, it starts to hurt.

In this post we'll look at why Git LFS gets painful as swift-snapshot-testing suites grow, and how to run the same snapshot tests with Screenshotbot without storing a single reference image in your repository.

How swift-snapshot-testing works with Git LFS

The standard workflow looks like this:

1. A developer changes some UI, and re-runs the tests on their machine with recording turned on.
2. swift-snapshot-testing writes the new reference images to __Snapshots__/<TestFile>/.
3. The developer commits the images along with the code. Git LFS stores the image contents on the LFS server, and leaves a small pointer file in the repository.
4. CI runs the tests, which fail if any snapshot doesn't match its reference image.

To use LFS, you add a line like this to .gitattributes:

**/__Snapshots__/**/*.png filter=lfs diff=lfs merge=lfs -text

This is a reasonable default. It keeps reference images next to the tests that produced them, and it doesn't need any extra infrastructure. But every one of those four steps gets worse as the number of snapshots grows.

Where it starts to hurt

Your repository only ever grows

Every time a snapshot changes, a new copy of the image is stored forever. Change a font or a color in your design system and you may re-record hundreds or thousands of snapshots in one commit. And if you test each view on several devices, in light and dark mode, and with a few Dynamic Type sizes, each view multiplies into many images. LFS keeps the large files out of the Git packfiles, but they still count against your storage and bandwidth quota on GitHub, GitLab or Bitbucket, and they still have to be downloaded.

We wrote about this in more detail in Can Git LFS scale?

Every CI job pays for it

A CI job that runs your snapshot tests needs the reference images, so it has to fetch them from LFS. That's extra time and bandwidth on every build, including the builds that didn't touch any UI. CI caches help, but they're one more thing to tune and keep warm.

Recording on a laptop isn't the same as recording on CI

Image snapshots depend on the simulator, the iOS version and the Xcode version they were rendered with. A reference image recorded on one developer's Mac can differ by a few pixels from the one CI renders, which turns into failures that nobody can reproduce locally. Teams end up pinning everyone to the same simulator and Xcode version, loosening the precision of their assertions until real regressions slip through, or adding a CI job that records and pushes a commit back to the branch.

Merge conflicts on binary files

Two branches that change the same screen will both change the same reference images. Git can't merge PNGs, so whoever merges second has to re-record and push again. On a busy repository with a shared design system, this happens a lot.

Reviewing changes is awkward

When a pull request changes fifty snapshots, the reviewer has to click through fifty image diffs in the code review tool, one file at a time, mixed in with the code changes. It's easy to miss the one snapshot that changed for the wrong reason.

A different approach: don't store reference images in Git

Reference images are build artifacts. They're generated from your code, so they don't need to be stored next to it in version control: they only need to be stored somewhere you can compare against, and tied to the commit that produced them.

That's what Screenshotbot does. Instead of recording on a developer's machine and committing the result:

1. CI runs your snapshot tests in record mode on every commit, for both your main branch and your pull requests.
2. The snapshots are uploaded to Screenshotbot, which stores them against the commit.
3. For a pull request, Screenshotbot compares its snapshots against the ones from the merge base on your main branch, and reports the changes as a build status on the pull request.
4. A reviewer looks through the changes side by side, with the differences highlighted, and accepts or rejects them.

No images are committed, so the repository stops growing, there's nothing to fetch from LFS on CI, and there are no merge conflicts on images. And because every snapshot is recorded on CI, they're all rendered with the same simulator and the same Xcode.

This scales to tens of thousands of snapshots, which is where the LFS approach tends to fall over.

Setting up swift-snapshot-testing with Screenshotbot

You don't need to rewrite your assertSnapshot calls. The simplest way to integrate is to switch to our fork of swift-snapshot-testing, which we keep in sync with the latest upstream release. In your Package.swift, or in Xcode's package dependencies, point swift-snapshot-testing at:

https://github.com/tdrhq/swift-snapshot-testing

With the fork, image snapshots are always recorded, and never fail the test. Screenshotbot does the comparison instead. Text snapshots, like .dump or .json, behave exactly as before, so they can stay in your repository. If you'd rather stay on upstream, you can wrap assertSnapshot in a helper that passes record: true for image snapshots.

Next, create an API key from your Screenshotbot account, and add it to your CI as the SCREENSHOTBOT_API_KEY and SCREENSHOTBOT_API_SECRET environment variables. Then, after your tests run, upload every __Snapshots__ directory. In Fastlane, that looks like:

lane :screenshotbot_ci do
  tests
  sh "curl https://cdn.screenshotbot.io/recorder.sh | sh"
  Dir.glob("../**/__Snapshots__/").each do |dir|
    channel_name = dir[3..-("/__Snapshots__/".length + 1)]
    sh "~/screenshotbot/recorder --channel #{channel_name} --directory #{dir} --recursive"
  end
end

Run bundle exec fastlane screenshotbot_ci on CI instead of your usual test lane, and you'll see the snapshot changes on your next pull request. If you use Xcode Cloud, the setup is a little different, since the upload runs from a post-build script. The iOS documentation covers both, and there's a complete example in tdrhq/swift-snapshot-testing-example.

Moving your existing snapshots out of LFS

Once Screenshotbot is running on CI, you can delete the reference images from your repository. Remove the LFS rule for __Snapshots__ from .gitattributes, then:

git rm --cached '*/__Snapshots__/*.png'
echo '**/__Snapshots__/**/*.png' >> .gitignore
git commit -m "Move image snapshots to Screenshotbot"

This only removes the PNGs, so any text snapshots stay where they are. The .gitignore entry stops anyone from accidentally committing newly recorded images. Older commits still reference the old images in LFS, so keep them around until you no longer need to build those commits.

Running snapshot tests locally

Developers can still check their changes before pushing, by recording a baseline and then verifying their changes against it with Screenshotbot's command line tool. See local runs for how this works.

Wrapping up

Git LFS is a fine way to start with swift-snapshot-testing. But reference images behave differently from source code: they change in bulk, they can't be merged, and they need a visual review. Once you have more than a few hundred of them, it's worth moving them out of Git and into a tool built for them.

Screenshotbot works with swift-snapshot-testing and the other popular iOS snapshot testing libraries, as well as Android and web. Try it for free, or get in touch if you'd like help migrating a large snapshot test suite.