Flutter golden tests without Git LFS

Flutter has golden tests built in. Call matchesGoldenFile() in a widget test, and flutter_test renders the widget and compares it against a golden image recorded earlier. There's nothing to install, and no emulator or device to manage.

Those golden images are PNGs in your test directory, and they're meant to be committed to your repository. Once they start to add up, most teams move them into Git LFS. For a handful of goldens that works fine. As your team writes more golden tests, it starts to hurt.

In this post we'll look at why Git LFS gets painful as Flutter golden test suites grow, and how to run the same golden tests with Screenshotbot without storing a single golden image in your repository.

How Flutter golden tests work with Git LFS

A golden test looks like this:

testWidgets('MyHomePage golden test', (WidgetTester tester) async {
  await tester.pumpWidget(const MyApp());
  await tester.pumpAndSettle();

  await expectLater(
    find.byType(MyApp),
    matchesGoldenFile('goldens/home_page.png'),
  );
});

And the standard workflow looks like this:

1. A developer changes some UI, and runs flutter test --update-goldens on their machine.
2. Flutter writes the new golden images next to the tests.
3. The developer commits the images along with the code. Git LFS stores the image contents on the LFS server, and leaves a small pointer file in the repository.
4. CI runs flutter test, which fails if any widget doesn't match its golden image.

To use LFS, you add a line like this to .gitattributes:

**/goldens/**/*.png filter=lfs diff=lfs merge=lfs -text

This is a reasonable default. It keeps golden images next to the tests that produced them, and it doesn't need any extra infrastructure. But every one of those four steps gets worse as the number of goldens grows.

Where it starts to hurt

Your repository only ever grows

Every time a golden changes, a new copy of the image is stored forever. Change a color in your theme and you may re-record hundreds or thousands of goldens in one commit. And if you test each widget at several screen sizes, in light and dark mode, and with a few text scales, each widget multiplies into many images. LFS keeps the large files out of the Git packfiles, but they still count against your storage and bandwidth quota on GitHub, GitLab or Bitbucket, and they still have to be downloaded.

We wrote about this in more detail in Can Git LFS scale?

Every CI job pays for it

A CI job that runs your golden tests needs the golden images, so it has to fetch them from LFS. That's extra time and bandwidth on every build, including the builds that didn't touch any UI. CI caches help, but they're one more thing to tune and keep warm.

Goldens recorded on a Mac don't match goldens rendered on Linux

This is the one every Flutter team runs into. Golden images depend on the platform they were rendered on: text and anti-aliasing come out slightly differently on macOS, Linux and Windows. A developer records goldens on their Mac, CI runs on Linux, and the tests fail on pixels nobody can see. Teams end up recording goldens inside a Linux Docker image, skipping golden tests on every platform except one, or loosening the comparison until real regressions slip through.

Merge conflicts on binary files

Two branches that change the same screen will both change the same golden images. Git can't merge PNGs, so whoever merges second has to re-record and push again. On a busy repository with a shared theme, this happens a lot.

Reviewing changes is awkward

When a pull request changes fifty goldens, the reviewer has to click through fifty image diffs in the code review tool, one file at a time, mixed in with the code changes. It's easy to miss the one golden that changed for the wrong reason.

A different approach: don't store goldens in Git

Golden images are build artifacts. They're generated from your code, so they don't need to be stored next to it in version control: they only need to be stored somewhere you can compare against, and tied to the commit that produced them.

That's what Screenshotbot does. Instead of recording on a developer's machine and committing the result:

1. CI runs flutter test --update-goldens on every commit, for both your main branch and your pull requests.
2. The golden images are uploaded to Screenshotbot, which stores them against the commit.
3. For a pull request, Screenshotbot compares its goldens against the ones from the merge base on your main branch, and reports the changes as a build status on the pull request.
4. A reviewer looks through the changes side by side, with the differences highlighted, and accepts or rejects them.

Nothing is committed, so the repository stops growing, there's nothing to fetch from LFS on CI, and there are no merge conflicts on images. And because every golden is rendered on CI, they all come from the same platform, so the Mac versus Linux problem goes away.

This scales to tens of thousands of goldens, which is where the LFS approach tends to fall over.

Setting up Flutter golden tests with Screenshotbot

You don't need to change your golden tests. On CI, run your tests in update mode so they write every golden instead of comparing, then upload the goldens with the Screenshotbot CLI:

flutter test --update-goldens
curl https://cdn.screenshotbot.io/recorder.sh | sh
~/screenshotbot/recorder --directory test/goldens --channel my-app

The CLI reads your API key from the SCREENSHOTBOT_API_KEY and SCREENSHOTBOT_API_SECRET environment variables, which you can create from your Screenshotbot account and add to your CI secrets. That's it: you'll see the golden changes on your next pull request.

Once the goldens are no longer in the repository, a plain flutter test would fail locally because there's nothing to compare against. To fix that, add a test/flutter_test_config.dart that always writes the goldens to test/goldens and passes, so golden tests behave the same way locally and on CI:

import 'dart:async';
import 'dart:io';
import 'dart:typed_data';
import 'package:flutter_test/flutter_test.dart';

class AlwaysUpdateGoldenFileComparator extends LocalFileComparator {
  AlwaysUpdateGoldenFileComparator(Uri testFile) : super(testFile);

  @override
  Future<bool> compare(Uint8List imageBytes, Uri golden) async {
    // Ensure the golden file goes in test/goldens
    final Uri testGoldensUri = Uri.parse('test/goldens/');
    final String goldenPath = golden.path;
    
    // Remove 'goldens/' prefix if present and rebuild the path
    final String fileName = goldenPath.startsWith('goldens/') 
        ? goldenPath.substring('goldens/'.length)
        : goldenPath;
    
    final Uri finalGoldenUri = testGoldensUri.resolve(fileName);
    final File goldenFile = File.fromUri(finalGoldenUri);
    
    await goldenFile.parent.create(recursive: true);
    await goldenFile.writeAsBytes(imageBytes);
    return true; // Always pass
  }

  @override
  Future<void> update(Uri golden, Uint8List imageBytes) async {
    // This method is called when --update-goldens is used
    // Use the same logic as compare() to ensure consistent paths
    await compare(imageBytes, golden);
  }
}

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  // Set up always-update golden file comparator for all tests
  goldenFileComparator = AlwaysUpdateGoldenFileComparator(Uri.parse('test/'));
  
  // Run the tests
  return testMain();
}

The Flutter documentation has the full details, and there's a complete example, including a CircleCI config, in screenshotbot/flutter_example.

Moving your existing goldens out of LFS

Once Screenshotbot is running on CI, you can delete the golden images from your repository. Remove the LFS rule for goldens from .gitattributes, then:

git rm -r --cached test/goldens
echo 'test/goldens' >> .gitignore
git commit -m "Move golden images to Screenshotbot"

The .gitignore entry stops anyone from accidentally committing goldens after running the tests. Older commits still reference the old images in LFS, so keep them around until you no longer need to build those commits.

Running golden tests locally

Developers can still check their changes before pushing, by recording a baseline and then verifying their changes against it with Screenshotbot's command line tool. See local runs for how this works.

Wrapping up

Git LFS is a fine way to start with Flutter golden tests. But golden images behave differently from source code: they change in bulk, they can't be merged, they depend on the platform that rendered them, and they need a visual review. Once you have more than a few hundred of them, it's worth moving them out of Git and into a tool built for them.

Screenshotbot works with Flutter golden tests, as well as native Android, iOS and web. Try it for free, or get in touch if you'd like help migrating a large golden test suite.