Suitest Test editor

In the Suitest Test editor you can create, edit and run automation scenarios without programming. You can also export the test cases to JavaScript files.

To create an automation test in the Test editor press New button on the main toolbar and then add operation lines to perform actions the scenario needs

Example test scenario in the test editor

The main toolbar

The main toolbar provides access to operations with test scenarios and their individual lines. All the buttons have a corresponding keyboard shortcut for quicker access. Check the keyboard shortcuts by pressing . (full stop).

Manipulating test cases and lines within a test case

Main toolbar available actions - basics

  1. New test - creates a new test in the specified folder.

  2. Save test - saves the test.

  3. More menu

    • Clean test - cleans up the results of the previous test execution.
    • Delete - deletes test.
    • Export - exports the test definition either in a JSON format or in JavaScript API format.
    • Import - imports the JSON test definition.

  4. Undo (Redo) - reverts (replays the reverted) edit operations.

  5. Paste, Copy, Cut, Delete, Duplicate - perform the standard editing operations on the selected test lines.

  6. Merge - merge compatible lines into one.

  7. Create snippet - creates a new snippet from selected lines and replaces the lines with a reference to that snippet.

Exporting and importing test definitions

You can export test definition in either JSON or JavaScript API format. The JSON format of the test is proprietary and intended for transferring the test between applications.

After exporting to JavaScript API format you can further edit the JavaScript code and run it with Suitest API launcher.

Please note, that currently the references to view elements are exported as is, meaning that elements referenced in the test are expected to exist (in element repository) before the definition will be imported again.

Creating new test lines

All test operations available for the currently selected configuration are presented on the main toolbar. Pressing a button will create a line template for that particular test operation that can be further customized.

Example line template for Assert operation in test editor

Default line actions on main toolbar

For HTML-based platforms the line actions are arranged on the main toolbar in the following way:

Main toolbar available actions

  1. Assert - inserts an assertion line.
  2. Press button - inserts a button press line.
  3. Poll URL - inserts a poll URL line.
  4. Open - inserts an open app line.
  5. Snippet inserts a reference to a test snippet.
  6. Sleep - inserts a sleep line.
  7. Execute - inserts an execute command (Only HTML-based platforms) line.
  8. Open URL - inserts an open URL line.
  9. Clear data - inserts a Clear application data line. (Not available for Apple TV and Roku)

Main toolbar line actions for websites

The Websites platform dedicated to testing traditional websites and web apps has the main toolbar line actions arranged in the following way:

Main toolbar available actions for the Websites platform

  1. Click on - inserts a click on line.
  2. Send text - inserts a send text line.
  3. Move to - inserts the Move to line.
  4. Set text - inserts a set text line.
  5. Browser command - inserts a line that sends a command to browser.

Interactive test recording and execution

The currently active test can be executed on a connected device directly from the Test editor. Editor can also record the interaction with the app and such recording can be used as a basis for a test.

Test editor run / schedule

  1. Add note - adds a note line, allowing to write comments into tests.

  2. Record Navigation - switch recording on or off.

  3. Schedule on devices - opens Test packs defined for this application.

  4. Run test interactively - drop-down with items:

    • Run test - runs test interactively on a connected device.
    • Run test with preview - runs test interactively with a preview turned on.
    • Other drop-down items are configurations created for the current app.

  5. Help - opens documentation pages.

  6. Actions - opens a window with all the actions available in test editor and their shortcuts.

Recording navigation

  • Android - WebView apps
  • HbbTV / Freeview Play
  • LG webOS
  • NextGen TV / ATSC 3.0
  • PlayStation 4/5
  • Samsung Tizen / Orsay
  • Sky
  • VIDAA
  • Vizio SmartCast
  • Xbox (One, Series X/S)
  • Xfinity / Xumo TV / XClass TV
  • Xumo (Entertainment OS)
  • Other Smart TVs and STBs

Suitest can record button presses and location changes that happen within the application. This recording can serve as a good basis for creating a test. You will only need to add your own assertions.

Not all buttons supported

Due to technical limitations we are not able to record all buttons on the platforms where recording navigation is supported.

Merging compatible lines

A test recording may become littered with many press button lines. You can combine similar press button lines by using the Merge lines operation on the main toolbar.

Merging press lines

You can also create lines with multiple presses without merging existing lines.

Press button line manipulations in Test editor

Selecting multiple lines

You can select multiple test lines inside of the Test editor by holding Shift key on your keyboard and clicking on the check-boxes.

Selecting multiple test lines using Shift key

The virtual remote control

Using the virtual remote control (VRC) you can control devices right within Suitest. Connect to a device, then press on the small VRC toggle on the right hand side to bring up the VRC.

TV remote and console controllers

The Suitest console

The Test editor has an integrated device console that displays application and network logs. It also allows you to interact with the application by sending commands like you would for example in the Chrome developer tools console (based on platform). It is located in the bottom of the Test editor.

Opening the console

Console in Test editor

The Suitest console also allows you to view network requests (including request and response headers) that are made by the application during test execution. From the log you can easily create assertions on the network requests.

Negative status codes

When using our Lite solution on one of the following platforms, Suitest is able to receive also negative status codes. Please check Chromium documation for a reference.

  • LG webOS - Lite
  • Samsung Tizen - Lite
  • Sky - Lite
  • VIDAA - Lite
  • Xfinity / Xumo TV / XClass TV - Lite
  • Xumo (Entertainment OS) - Lite

Network logs in Test editor

HTML-based

Not available with Lite solutions.

Most devices will support console API, for those devices that do not you can use the suitestConsole object to log messages to Suitest console. You can also send JavaScript commands to the device through the console's command line which will be executed in real-time.

The suitestConsole function is only available for apps that have included the Suitest instrumentation library. It is used like so:

suitestConsole.log('This will appear in the Suitest console');

Since you should not use the instrumentation library in your production code you should prepare your code accordingly.

For example, by using the following initialization snippet:

if (!window.console) {
    window.console = suitestConsole || {
        log : function(){/* own implementation*/},
        info : function(){/* own implementation*/},
        warn : function(){/* own implementation*/},
        error : function(){/* own implementation*/},
    }
}

Android

Once your application is instrumented you will be able to see the logcat coming from the device.

Roku

Once you are connected to the device, you will be able to see the BrightScript console output.

Xbox (One, Series X/S) Native

Once your application is instrumented you can use the Suitest console.

Suitest console supports the following:

await Suitest.Console.ErrorAsync("Your message...");
await Suitest.Console.WarnAsync("Your message...");
await Suitest.Console.InfoAsync("Your message...");
await Suitest.Console.LogAsync("Your message...");

Since you should not use the instrumentation library in your production code you should prepare your code accordingly.

If you followed the managing native app builds guide then you can wrap the console method calls like so:

#if SUITEST
await Suitest.Console.ErrorAsync("Your message...");
#endif

If you like to use the Suitest console in more often then it would make sense that you create a logger class and a special method in it that has the wrapper like the example above.

Beware that if you send empty messages with null or "" an exception System.ArgumentException will be thrown.

TraceSource

TraceSource is optional NuGet package for logging. Our instrumentation library adds the SuitestListener and forwards all the messages to the Suitest console.

Trace.Listeners.Add(new SuitestListener());

For example, when you call:

Trace.WriteLine("My message from Trace.");

The message will appear in the Suitest console. If you would like to keep this functionality do not remove the listener.

Getting test's unique ID

To use runTest function inside of our JavaScript API, you will need to get the ID from the Test editor. It can be found inside of test manipulation drop-down along side rename and delete.

Copy the test id