Automated testing in Flutter: An overview

The success of any application depends on its quality. For customers to love an app and evangelize it via word-of-mouth advertising, it must provide the highest quality possible and withstand adverse conditions.

Quality assurance plays an important role in addressing an application’s defects before it reaches production. Almost all software teams have some form of QA as part of their development lifecycle, even if there is no dedicated QA team that only does this job.

It’s the nature of software engineering that new features are built on top of the existing codebase. Hence, whoever is responsible for QA will have to test not only the new features, but the existing features as well to ensure the app works nicely with the new features integrated.

Now the problem is: the time spent in QA will increase with every new feature, and there is a very real chance that not everything will be well-tested. Bugs can easily slip into the user’s hand.

Automation testing really helps here by automating some of the work that QA would do manually. We can write an automation test for those features that QA has already tested so the team can focus on testing new features while the old features will be tested automatically. This saves a lot of time and brings a higher level of confidence in shipping the app to production.

In this tutorial, we’ll introduce automated testing for Flutter and review how to write each type of automation test with an example.

Here are the three types of tests we’ll cover:

• Unit tests
• Widget tests
• Integration tests

Reviewing our example Flutter app

Let’s have a look at the sample app we’ll be testing:

For the purposes of this tutorial, our requirement is that the list of all products should be available on the app homepage. The user can add the product to the cart by clicking the cart icon beside it. Once added, the cart icon should be changed.

Clicking on the Cart text should open up a cart page, which displays a list of all the products added to the cart. The products can be removed from the cart either via the cancel button or a swipe to dismiss.

Writing the tests for our Flutter app

As discussed above, we’ll automate three types of tests for our Flutter app: unit tests, widget tests, and integration tests. An app can have several combinations of these three tests, but it’s up to you to design and implement the tests in a way that provides the most confidence for your use case.

Unit tests

Let’s begin with the unit test for the app. This tests the single method of the class by ensuring the method provides the expected result based on the input given to it. It helps you to write more testable and maintainable code.

Our goal is to write unit tests for the Cart class — to be more specific, we will make sure that adding and removing methods for products gives the correct result.

First, dd the test dependency:

dev_dependencies:
  test: ^1.14.4

Here is the Cart class, which has methods to add and remove items:

import 'package:flutter/material.dart';

/// The [Cart] class holds a list of cart items saved by the user.
class Cart extends ChangeNotifier {
  final List<int> _cartItems = [];

  List<int> get items => _cartItems;

  void add(int itemNo) {
    _cartItems.add(itemNo);
    notifyListeners();
  }

  void remove(int itemNo) {
    _cartItems.remove(itemNo);
    notifyListeners();
  }
}

Next, we’ll create a file to write test cases. Inside the test folder (at the root of the project), create a new file cart_test.dart. It should look something like this:

Now add the below code inside it:

N.B., make sure to give a test file named as (classToTest)_test.dart.

import 'package:flutterdemos/testingdemo/models/cart.dart';
import 'package:test/test.dart';

void main() {
  group('Testing Cart class', () {
    var cart = Cart();

    //Test 1
    test('A new product should be added', () {
      var product = 25;
      cart.add(product);
      expect(cart.items.contains(product), true);
    });

    // Test 2
    test('A product should be removed', () {
      var product = 45;
      cart.add(product);
      expect(cart.items.contains(product), true);
      cart.remove(product);
      expect(cart.items.contains(product), false);
    });
  });
}

Here, Test 1 verifies the added item should exist in the cart list, and Test 2 checks whether the removed item does not exist in the cart. The expect() method is a way to validate our output with expectation.

Now we’ll run the unit test. Simply hit the play button in the IDE.

You can also try with the terminal using the following command:

flutter test test/cart_test.dart

Widget tests

As its name suggests, the widget test focuses on a single widget. Unlike the unit test, the widget test makes sure that a particular widget is looking and behaving as expected. You should write a widget test for at least all common widgets.

Our goal here is to write a widget test to ensure that the homepage is working as expected.

First, add one more test dependency:

dev_dependencies:
  test: ^1.14.4 # for unit test
  flutter_test:   
    sdk: flutter

Similar to the cart_test.dart file we created in the previous section, we’ll now create one more file home_test.dart inside the test folder. Let’s add the below code to it.

Widget createHomeScreen() => ChangeNotifierProvider<Cart>(
      create: (context) => Cart(),
      child: MaterialApp(
        home: HomePage(),
      ),
    );

void main() {
  group('Home Page Widget Tests', () {
    // Test 1
    testWidgets('Title should be visible', (tester) async {
      await tester.pumpWidget(createHomeScreen());
      expect(find.text('Shopping App Testing'), findsOneWidget);
    });
  });
}

The methods below are the building blocks for writing our widget test:

• createHomeScreen() – provides the UI for the home screen that we would normally do in the main.dart file
• testWidgets() – creates the WidgetTester that provides ways to interact with the widget being tested
• await tester.pumpWidget() – renders the provided widget
• find.text() – finds the widget with the given text. Sometimes we may have the same text in the UI, so find.byKey(Key('string')) becomes really helpful
• expect() – takes the found widget and compares it with the expected Matcher, which can be findsOneWidget, findsNothing, etc.

Let’s cover a couple other important test cases that we would otherwise have to perform manually. Here, we test that the product list is visible on the homepage:

testWidgets('Product list should be visible', (tester) async {
  await tester.pumpWidget(createHomeScreen());
  expect(find.byType(ListView), findsOneWidget);
});

And here, we test that the user is able to scroll the product list:

testWidgets('Scroll test', (tester) async {
  await tester.pumpWidget(createHomeScreen());
  expect(find.text('Product 0'), findsOneWidget);

  await tester.fling(find.byType(ListView), Offset(0, -200), 3000);
  await tester.pumpAndSettle();

  expect(find.text('Product 0'), findsNothing);
});

A full list can be found here.

Now run the test.

Integration tests

Integration tests help to achieve end-to-end testing for the app. They enable us to understand whether users are able to complete the full flow of the app. It’s essentially like testing a real application.

Unlike unit tests and widget tests, integration tests run on a real device, so we get a chance to see how tests are being performed. In a perfect world, we’d write and run as many tests as we need. But if we have limited time, we should absolutely write an integration test at the very least.

Our goal here is to test that the user is able to add and remove products to and from the cart. Here’s the dependency required for the integration test:

dev_dependencies:
  test: ^1.14.4 # for unit test
  flutter_test: # for widget test
    sdk: flutter
  flutter_driver:
    sdk: flutter
  integration_test: ^1.0.1

Now we create the integration_test folder at the project root and add a file driver.dart inside it with the following code:

import 'package:integration_test/integration_test_driver.dart';

Future<void> main() => integrationDriver();

Then we’ll create a file app_test.dart and add the below code:

void main() {
  group('Testing full app flow', () {
    IntegrationTestWidgetsFlutterBinding.ensureInitialized();

    testWidgets('Add product and remove using cancel button', (tester) async {
      await tester.pumpWidget(MyApp());

      //Add
      await tester.tap(find.byIcon(Icons.shopping_cart_outlined).first);
      await tester.pumpAndSettle(Duration(seconds: 1));
      expect(find.text('Added to cart.'), findsOneWidget);

      // Move to next page
      await tester.tap(find.text('Cart'));
      await tester.pumpAndSettle();

      //Remove via cancel button
      await tester.tap(find.byKey(ValueKey('remove_icon_0')));
      await tester.pumpAndSettle(Duration(seconds: 1));
      expect(find.text('Removed from cart.'), findsOneWidget);
    });

  });
}

As we can see in the code above, there are instructions to perform actions and verify the results, just as we would do manually:

• await tester.tap() – clicks on the specified widget
• await tester.pumpAndSettle() – when users click on a UI element, there might be an animation. This method ensures that the animation has settled down within a specified duration (e.g., if we think the required widget is not yet available), after which period we’re good to go for new instructions

We also have a provision for removing products via swipe. The code for achieving this behavior goes here:

//Remove via swipe
await tester.drag(find.byType(Dismissible), Offset(500.0, 0.0));
await tester.pumpAndSettle(Duration(seconds: 1));
expect(find.text('Removed from cart.'), findsOneWidget);

Finally, we’ll run the test on a real device. Run following command in the terminal:

flutter drive — driver integration_test/driver.dart — target integration_test/app_test.dart

Conclusion

In this tutorial, we’ve introduced automation testing in Flutter and walked through the various types of tests we can write via examples. You can view the complete source code with all the test cases in my GitHub.