Screenshot testing Android Apps

NEW! We now have a Gradle plugin to automate all of this in one line. Check out our Blog Post for more information.

In this document we'll describe how to integrate Screenshotbot with popular screenshot testing frameworks such Facebook's screenshot-tests-for-android, Karumi's Shot, and Cash App's Paparazzi.

We suggest using Screenshotbot instead-of the in-built record/verify modes provided by these libraries. It's a better developer experience for your colleagues, as they will not be required to manually record screenshots after every run. Screenshotbot will give you automatic notifications on your Pull Requests.

Paparrazi has the advantage of not requiring an emulator to generate screenshots, but this comes with certain limitations of what you can screenshot.

We'll work on this in two steps, first we'll show you how you can generate runs on your local desktop or laptop, and we'll then show you how you'll integrate with your CI.

Initial setup

First, let's install the CLI tools used to interact with Screenshotbot

        curl https://cdn.screenshotbot.io/recorder.sh | sh

          PS C:\> curl https://screenshotbot.io/recorder.exe -O recorder.exe

Then we need to set up Screenshotbot's API keys. The preferred method is to use environment variables, although you can also pass it as command line arguments.

You can get the API Key and Secret from your dashboard.

        export SCREENSHOTBOT_API_KEY=<api-key>
        export SCREENSHOTBOT_API_SECRET=<api-secret>

        # You might have to restart the shell or you can set `set` instead
        C:\> setx SCREENSHOTBOT_API_KEY <api-key>
        C:\> setx SCREENSHOTBOT_API_SECRET <api-secret>

Paparazzi

Paparazzi is straightforward to integrate with Screenshotbot.

By default, Paparazzi stores screenshots generate to src/test/snapshots/images. Add this directory to your .gitignore, we will not be committing the screenshots to your repository.

Run your Paparazzi tests as follows:

      $ ./gradlew recordPaparazziDemoDebug

This should populate src/test/snapshots/images, so we can now upload it using the Screenshotbot CLI:

      $ ~/screenshotbot/recorder --directory src/test/snapshots/images
                                 --channel any-arbitrary-name

Remember, all these steps are automatically running on CI. Developers will not need to run these locally.

Sample Code

See a fully working sample repository with Paparazzi here.

screenshot-tests-for-android

We'll assume you already have configured screenshot-tests-for-android for your project, and can run recordDebugAndroidTest or similar.

These same instructions should work for Karumi's Shot which uses screenshot-tests-for-android under the hood.

Keep your screenshot output directory in your .gitignore. You'll not be commiting the screenshots to your repository.

At this point you can run your screenshots on your development branch as follows:

    $ ./gradlew :recordDebugAndroidScreenshotTests

If your output directory is ./screenshots/ you can now run the CLI tool to upload all the screenshots.

      $ ~/screenshotbot/recorder --directory ./screenshots
                                 --channel any-arbitrary-name

If you now go to https://screenshotbot.io/runs, you’ll be able to view your recent runs. You can select multiple runs and you can compare them. Note that any developer run will not be “promoted”: i.e. we won’t send tasks for changes. We’ll get to how to set up promotions in the section on CircleCI and Jenkins.

Using Firebase for emulators

By default screenshot-tests-for-android does not support Firebase. The screenshotbot CLI can however parse output from Firebase. See here for more detail or feel free to contact us.

Sample Code

See a fully working sample repository here.

pedrovgs/Shot (a.k.a Karumi/Shot)

Shot uses a similar workflow as screenshot-tests-for-android. Our CLI tool will be able to parse the different when needed.

Sample Code

See a fully working sample repository using Firebase for emulators here.

Next Steps

In order to get the full benefits of Screenshotbot we encourage you to integrate the scripts from above into your CI jobs, for example CircleCI or Jenkins.

You can then integration with your Code Review platform of you choice (for example, GitHub), to get build statuses.