Auto-testing Flutter Apps using Artificial Intelligence

Mobile application development has been growing faster than ever. Almost every business needs a mobile app to stay competitive in the market. As cross-platform mobile app development frameworks like React Native allow companies to build both iOS and Android apps using the single source code and single programming language, Flutter is another hot cross-platform mobile application development framework backed by Google.

Flutter is an open-source, multi-platform, mobile SDK which can be used to build iOS and Android apps with the same source code. Flutter has been around for quite some time now, but it's got more attention since Google released a first stable release. The apps built with Flutter looks almost like native apps which might be the reason big companies like Alibaba, Groupon, and many others started using it. Flutter apps can be tested using various testing layers.

Flutter Apps

Flutter applications use slightly different technologies than the usual tech stack being used in the market. Flutter uses Dart programming language to develop cross-platform apps which are not common programming language for developers. However, setting up the first Flutter app isn't that difficult.

Let's create a demo Flutter app. Flutter has great documentation on how to install and set up the environment for iOS and Android apps, and we can demonstrate that easily. The process of installing Flutter SDK is very simple: you need to download the SDK and add it to your local PATH. Here you have the detailed instructions to install the Flutter SDK. Once you can access Flutter globally, the next thing is to setup iOS and Android environments. The flutter doctor command will guide you to the entire setup. Once all setup is done, you can runflutter doctorto make sure there are no setup issues.

Flutter can generate boilerplate for the sample app by running a single command. Let's create a Flutter-Bitbar app by using the following command.

This will create a Flutter app inside the flutter_bitbar directory. If we got iOS simulator or Android emulator open then we can simply run the Flutter app by using the following commands

$  cd flutter_bitbar && flutter run

 You will see the Flutter app running inside the simulator.

This application is a demo app created by the Flutter team, with a counter that increments every time you press the + button.

Testing the Flutter App

Flutter provides some amazing testing features to test the apps while they get developed. You can test Flutter apps at the unit, widget, and integration level. Widget testing is a new feature that Flutter provides to run UI tests as fast as unit tests. Flutter also has UI tests, known as integration tests, that run on the simulator or on real devices. The Flutter documentation on the testing covers all the types of testing layers in detail with confidence and speed matrix. In summary, Flutter has three major layers of the testing.

Unit Testing

Flutter unit testing covers a single unit or method works well with different conditions or parameters. The package flutter_test is required to run the unit test for the Flutter app.

Widget Testing

This is one of the new styles of testing introduced by Flutter to test Flutter widgets independently. You can launch a widget as part of the test and perform actions as real users do on the widget. The widget test asserts the state of the widget under test. The widget tests run as fast as unit tests, although it involves the user interface.

Integration Testing 

In the Flutter world, UI testing is known as integration testing. Flutter integration tests behave very similar to Xcode UI tests or Android Espresso tests which go through the UI to perform specific operations. Flutter integration tests run in a separate process and can be run on real devices or on simulators or emulators. Flutter provides a separate package, Flutter Driver, to drive these tests.

We will cover each testing method in detail in a later blog post, but for this tutorial, we will user Bitbar AI Testbot for testing the Flutter apps without any test configuration.

Artificial Intelligence for Testing Flutter Apps

Technologies like Artificial Intelligence and Machine Learning (AI/ML) has started to appear in software testing. Bitbar was the first in the industry to launch an AI testing solution. The AI Testbot can be used to perform the quick sanity and health check of mobile apps. You can read more about the role of AI in the existing mobile testing effort on our previous post here.

Prepare Flutter App for AI Testbot

In order to prepare the Flutter apps for the Bitbar AI Testbot, we need to first the build and APK or IPA file for Android and iOS respectively. For the purpose of this tutorial, we will create an Android APK file with a simple command:

$ flutter build apk

This command will build an APK for the demo app that we have just created.

This command will generate the APK file inside the "build/app/outputs/apk/" directory that we can use for the AI-based testing on Bitbar cloud. At this point, we got our Flutter app (Android) ready to be executed on the Bitbar cloud.

Bitbar AI Testbot

In order to test your Flutter app using the Bitbar AI Testbot, you need to have access to Bitbar platform, and you can sign up for free here. Once you get access to the Bitbar platform, you can launch a new AI Testbot run to test your own mobile apps. You can create a new test run from the Bitbar device cloud and select the AI Testbot option. Note that AI Testbot doesn't require any test file which means you don't have to write any tests to perform a quick sanity check of the app.

In the next steps, we can upload the APK that we have just created from the Flutter apps and select the devices to run the AI tests.

You can do additional test configuration like providing the test credential, but at this stage, our Flutter app doesn't need any test user credential. Let's just start the test run and wait for the results. After a few minutes, AI Testbot will report the results of the tests in the dashboard which includes the list of the steps, screenshots, and a video recording of the scenarios executed by the AI Testbot.

As shown above, AI Testbot has been performed the all the normal user journeys without writing any tests and reported back the results with all the details. You can drill down the reports and find the CPU and performance issues as well.

At this stage, we have performed a quick sanity test of the Flutter app using the AI Testbot, without writing any tests or without doing any test configuration.

What's Next

We have used an AI-based check to perform the sanity checks of the Flutter apps. There are integration tests that we can write for the Flutter apps and run them on the real devices. Using the Flutter Driver, we can easily do this by attaching the device or simulator to the local machine. However, Flutter integration testing is not compatible with any cloud-based vendors yet.

There's some work to do from the Flutter team itself to make the device testing on third-party vendors can be possible. You can read more about this on this thread explaining the compatibility of the few users. Once we figure out the way to set up your tests on the Bitbar Device Cloud, we will keep you posted.

Conclusion

You can test Flutter apps without doing any test-related setup using the AI-powered testing feature of Bitbar. Using the AI/ML technique on your next Flutter app can give you the confidence that your Flutter app can still work. Flutter developers can build solid test suites using the Flutter's testing features whilst Bitbar AI Testbot can help in the codeless testing of the Flutter apps.