Get things started

To show you what Suitest can do we will be using a demo application called WatchMe as our test automation subject. This is an HbbTV application that has been adapted to run in a browser environment. This application is configured in every Suitest account along with a bunch of test scenarios for your reference. You are welcome to experiment with it further.

For the quick showcase we will be using the browser as our testing "device" and later describe how to do the same on the real physical device.

So, let's start with the basics.

1. Get a free Suitest account

First, you will need a Suitest account. We have a Free plan to start you out. The Free plan comes with all the features which Suitest has, so you can take a look around. Go to get one now if you have not already.

2. Download SuitestDrive

SuitestDrive is our little helper app that helps Suitest take control of your browser, emulator or Xbox One. Heard of Selenium? Well, SuitestDrive works very similarly to a Selenium server (in fact, it uses some of the Selenium protocols internally) - Suitest talks to SuitestDrive which in turn talks to your browser/emulator/Xbox One and tells it what to do.

You can download SuitestDrive from your account profile page. To run SuitestDrive, you need to have Java installed. Chances are that you already have it, but if you don't, you can get it from the Oracle website.

SuitestDrive is a stand-alone application and it does not need administrative privileges to run on your computer, which is cool.

3. Add your browser to Suitest

Once you have downloaded SuitestDrive, it is time to tell it which browsers or emulators you would like to use it with. SuitestDrive currently supports:

  • Google Chrome
  • Mozilla Firefox
  • Vewd TV Emulator
  • Microsoft Edge

Once you have SuitestDrive open, pair it with your account and it will be added to your control units. Now press Add browser or device to add one of the supported devices. After the device has been successfully added you can minimize SuitestDrive as we will not be interacting with it anymore.

For more information about the pairing and device adding process refer to the SuitestDrive section.

4. Connect to your browser

Go to the Suitest website and log into your account. Then just hit Connect to device in the top right corner. This will pop up the window with available devices, which are currently only the browsers that you have added in the previous step. Then just press the Connect button:

Connect to your browser
Connect to your browser

Suitest will start a fresh browser instance and load a special Suitest start up page in it. After that page has been loaded Suitest maintains a live connection between itself and the browser which we call an interactive mode session. Now you are ready for your first test execution.

5. Run your first test

Every Suitest account has a pre-configured demo application called WatchMe. This application was developed to run on TVs and also adapted for some devices that are supported by SuitestDrive. We have created a bunch of test scenarios for this app and made them available in your account. To discover all the cool features in Suitest be sure to browse through them later.

For now, let's just watch how Suitest will execute the Welcome to Suitest test scenario on the WatchMe app. For this open the WatchMe app and select the "Welcome to Suitest" test from the left menu.

In Suitest you always run the test with a specific app configurations. The WatchMe app has a few of those, so you'll want to select the one called hbbtv from the run menu. Don't worry yourself about the purpose of other configurations for now. Now, hit Preview in the same menu and Suitest will start executing the "Welcome to Suitest" test in preview mode.

Run "Welcome to Suitest" in preview mode
Run "Welcome to Suitest" in preview mode

Switch over to the browser instance launched by Suitest to watch the test execution happening. In preview mode, Suitest displays an informational overlay on top of the application under test and also slows down the test execution just enough for a human to be able to follow along.

6. Record your first test

Alright, now we are ready for the big step. Let's create our very own test. Arrange your open windows so that you can comfortably see both the WatchMe app and the Suitest web app then hit new test on the main toolbar to create a new test and give it a nice name. Suitest will create the test and automatically add an Open App line to it, the action will look like this:

Creating a new test
Creating a new test

In test automation one always prefers the test scenarios to be short and focused on testing of a single specific aspect. A scenario can either test a feature or a user journey through the app. For example, the WatchMe app has a homepage, a gallery page and a video playback page. As a caring product owner, you want to make sure that your user can open a picture in the gallery mode. Basically, we should:

  1. Open the application.
  2. Wait until the app has loaded data.
  3. Navigate to the second picture in the first row (the one with cherries) and press OK.
  4. Check that the gallery has been loaded.

Let's see if we can get that down in Suitest. Step one is already done for us, so let's try and run it. Simply press the play button at the end of that line.

Playing the first line
Playing the first line

Suitest will open the app inside the browser you are connected to and the first line of your test will be marked with line passed

Before implementing the second step of our test we should make sure that the app has loaded content. Our cherries picture should be visible on the screen. When testing manually we would just wait for it to load. However in an automation scenario one has to be precise and write everything down. So let's add a test line establishing that.

For this purpose create a Wait until line which will wait for the cherries picture to be loaded. Click on the "Add line" button and select "Wait until", so that your line looks like this:

Wait Until line without any subject
Wait Until line without any subject

Now we need to specify what we are waiting for. Click on the subject placeholder and select element then select Cherries image from the drop-down (Or type it in the search to make it easier on you).

Now, this is an important part: The cherries image was in the drop-down, because we have already prepared it for you in the element repository of the WatchMe application. Element repository holds all of your view elements and when you need to check anything about any element on the page you have to first add that element to the element repository. When starting automation for a new app it is a good idea to first fill your repository with view elements and do it well before you start creating the actual automation scenarios.

Be sure to read up on the element repository later.

The completed line #2 will look like this:

Second line completed with element and action
Second line completed with element and action

Next we should be navigating over to our cherries picture. The shortest path to get there while using the regular remote control is two times right and then one time OK.

In Suitest the Press line is responsible for navigation. So click on the Add line button again and select Press then select the RIGHT key from the drop-down.

Repeat for the remaining keys until you get 3 lines like this:

Lines 3 to 5 with the navigation presses
Lines 3 to 5 with the navigation presses

Actually there is at least one more way to achieve the same effect faster. Suitest has a "Record navigation" function. When this is turned on Suitest will write down the key presses that you make on your remote control, so you don't have to do this manually. Be sure to try it at some point.

To make things more compact, we can merge our Press lines into one. Select the lines (by checking the boxes at the beginning of the line) and then click on Merge button in the main toolbar. The action looks like this:

Navigation merge
Navigation merge

So now we have reached the gallery page and all that remains is to verify that the gallery page did indeed open and has loaded a correct image.

To do this, let's again use the Wait until line with the element subject and select the Image (Cherries) in gallery element from the drop-down.

Your final test should look like this:

Test is finished
Test is finished

Note that the line #4 specifies a maximum waiting time of 2 seconds. Depending on the circumstances it may take more than 2 seconds to load the gallery page. To be sure that our test is resilient against slow devices/networks let's increase the timeout to 4 seconds.

Furthermore, the first rule of the test automation:

Document everything!

When you come back to this test a few weeks later you should be able to understand what it is doing at the first glance. So let's add a test description and a quick note about why we have increased the timeout on line #4.

Test description and note about timeout
Test description and note about timeout

With this last finishing touch we have successfully automated the scenario above and it is time to go get a well-deserved cup of coffee.

But wait! We are diligent so we will of course run the test from start to finish once, in order to make sure it really works. Now let's choose the Run test command from the "run test" drop-down so that the test executes at full speed.

7. Schedule a test pack

In Suitest the test packs are collections of test scenarios that can be scheduled for automatic unattended execution on a group of devices. Typically you would want to run a test pack after your application has been updated in some way to make sure that it works.

To try it out go to the Test packs section of the main menu and click on the Create a new test pack button. Specify the tests you want to run using tags, the app configuration and the devices where the tests should be executed. Then click on "Run test pack now".

Note: If you are still connected to the device, the test pack execution on that device will not start until you disconnect.

For production set up you may want to specify execution schedule or trigger test pack execution from our continuous integration API.

After the execution of a test scenario is complete the results will show up in the Results tab on the main toolbar.

8. Run scenarios on a real device

For running test on a real device you need to perform a one-time configuration of that device. Everything else works exactly the same as described above.

To configure a Smart TV or a set-top box you will need a CandyBox unit. If you already have it, take a look at Adding a device to Suitest.

Configuring Xbox One with Suitest is fairly similar to the process you have done with the browser. Check the Xbox One setup article for detailed instructions.

You are ready to automate!

Congratulations, if you have reached this point you are now well armed to create professional test automation scenarios with Suitest. There is a lot more that Suitest can do! Be sure to browse the sample scenarios in the WatchMe app and check out the following articles: