Suitest Test Launcher

Suitest Test Launcher is a CLI tool, designed to ease the configuration and usage of Suitest JavaScript API. You can still use JS API without the test launcher, but you'll have to handle the API bootstrapping process on your own. See Advanced usage for details.

NPM/Yarn Prefixes

Examples do not include yarn or npm / npx prefixes for brevity. Add the prefixes based on your installation method to examples. We recommend setting up those commands inside package.json to avoid using the prefixes every time.

What does the launcher do:

  • Makes it easier to submit configuration values to the API calls.
  • Manages simultaneous test execution on multiple devices.
  • Runs authentication requests and generates temporary tokens for the test runs.

Usage

CLI signature for the test launcher:

suitest [run mode] [run mode options] [test command] [test command options]
  • run mode - one of interactive or automated - sets the Suitest test run mode.
  • run mode options - modify the behavior of the execution.
  • test command - command to start the test run. For example node ./test.js.
  • test command options - options passed to the test command.

Launcher options

Suitest Test Launcher operates in two execution modes:

Interactive execution mode

suitest interactive reserves a single Suitest device for use in interactive mode and triggers execution of the test command. Interactive mode is intended only for test authoring and application debugging.

suitest interactive \
    -u <your@e.mail> \
    -p <your password> \
    -o <organization id> \
    -d <device id> \
    -c <app config id> \
  node ./test.js

Interactive mode options

In CLI In .suitestrc Mandatory Description
--app-config-id, -c appConfigId yes App configuration id to launch the app with.
--config-file configFile no Path to JSON config file to override .suitestrc config
--continue-on-fatal-error continueOnFatalError no, defaults to false If the Suitest server responds with a fatal error to any command made by Suitest API, Suitest API will by default attempt to kill the process it is running in. You can turn this off by passing --continue-on-fatal-error and handle the shutdown yourself. Please note, that after a fatal server error occurs, all subsequent Suitest commands will fail until the connection is reinitialized with startTest().
--default-timeout defaultTimeout no Default timeout for assertion line. Default set to 2000ms. Value set in ms.
--device-id, -d deviceId yes Device to which to connect interactively.
--disallow-crash-reports disallowCrashReports no, defaults to false If a crash occurs an automatic report will be submitted to Suitest. These reports help Suitest to improve and update faster.
--help, -h - no Show CLI usage help.
--log-dir logDir no If defined, logs will be stored inside specified folder.
--log-level logLevel no, defaults to normal Logger verbosity level. Values: silent, normal, verbose, debug
--org-id, -o orgId yes Id of the organization you want to login into.
--password, -p password yes Suitest password. Same that you're using to login to Suitest.
--timestamp timestamp no, defaults to YYYY.MM.DD HH:mm:ss format Console log timestamp in YYYY.MM.DD HH:mm:ss format unless other format is specified. Supported formats can be found in time-stamp package. Timestamp can be turned off by passing 'none'.
--username, -u username yes Suitest username. Same that you're using to login to Suitest.
--version, -v - no Show Suitest JavaScript API library version.

Automated execution mode

suitest automated executes a Test Pack on multiple devices in parallel. For every device involved in this Test Pack - a new instance of test command is started in a separate process. Automated mode is intended for periodic test executions (for example when used in a CI process).

suitest automated \
    -k <token key> \
    -p <token password> \
    -t <test pack id> \
  node ./tests

Automated mode options

In CLI In .suitestrc Mandatory Description
--concurrency concurrency no, defaults to 0 Amount of simultaneously running tests. I.e: how many devices may be simultaneously busy with running this Test Pack.
0 = unlimited. Normally you should not specify this parameter.
--config-file configFile no Path to JSON config file to override .suitestrc config
--continue-on-fatal-error continueOnFatalError no, defaults to false If the Suitest server responds with a fatal error to any command made by Suitest API, Suitest API will by default attempt to kill the process it is running in. You can turn this off by passing --continue-on-fatal-error and handle the shutdown yourself. Please note, that after a fatal server error occurs, all subsequent Suitest commands will fail until the connection is reinitialized with startTest().
--default-timeout defaultTimeout no Default timeout for assertion line. Default set to 2000ms. Value set in ms.
--disallow-crash-reports disallowCrashReports no, defaults to false If a crash occurs an automatic report will be submitted to Suitest. These reports help Suitest to improve and update faster.
--help, -h - no Show CLI usage info.
--log-dir logDir no If defined, logs will be stored inside specified folder.
--log-level logLevel no, defaults to normal Logger verbosity level. Values: silent, normal, verbose, debug
--test-pack-id, -t testPackId yes Id of the test pack to launch.
--timestamp timestamp no, defaults to YYYY.MM.DD HH:mm:ss format Console log timestamp in YYYY.MM.DD HH:mm:ss format unless other format is specified. Supported formats can be found in time-stamp package. Timestamp can be turned off by passing 'none'.
--token-key, -k tokenKey yes Suitest token key.
--token-password, -p tokenPassword yes Suitest token password.
--version, -v - no Show Suitest JavaScript API library version.
Concurrency - simultaneously running test limit

The amount of tests allowed to run simultaneously varies per subscription. You can check the current limit in your profile summary. Suitest Test Launcher would however launch as many processes as there are devices in the Test Pack (unless limited by the --concurrency flag). If beginning another parallel test would cause exceeding your subscription limits, the corresponding process will idly wait for a free execution slot.

Launcher child processes

Suitest Test Launcher will always execute the test command in a separate process (in both interactive and automated run modes).

In automated mode the running instances don't know about one another and run completely independently.

The environment variable used internally by the launcher and it's child processes:

  • SUITEST_CHILD_PROCESS

You should not rely on using this variable.

Enable E2E test debugging

To enable debugging in your End-To-End tests using the Test Launcher in interactive mode please refer to the E2E test debugging article.