Suitest Test Launcher

Suitest Test Launcher is a CLI tool, designed to ease the configuration and usage of Suitest JavaScript API. You can still use JS API without the test launcher, but you'll have to handle the API bootstrapping process on your own. See Advanced usage for details.

NPM/Yarn Prefixes

Examples do not include yarn or npm prefixes for brevity. Add the prefixes based on your installation method to examples. We recommend setting up those commands inside package.json to avoid using the prefixes every time.

What does the launcher do:

  • Makes it easier to submit configuration values to the API calls.
  • Manages simultaneous test execution on multiple devices.
  • Runs authentication requests and generates temporary tokens for the test runs.

Usage

CLI signature for the test launcher:

suitest [run mode] [run mode options] [test command] [test command options]
  • run mode - one of interactive or automated - sets the Suitest test run mode.
  • run mode options - modify the behavior of the execution.
  • test command - command to start the test run. For example node ./test.js.
  • test command options - options passed to the test command.

Launcher options

Suitest Test Launcher operates in two execution modes:

Interactive execution mode

suitest interactive reserves a single Suitest device for use in interactive mode and triggers execution of the test command. Interactive mode is intended only for test authoring and application debugging.

suitest interactive \
    -u <your@e.mail> \
    -p <your password> \
    -o <organization id> \
    -d <device id> \
    -c <app config id> \
  node ./test.js

Interactive mode options

In CLI In .suitestrc Mandatory Description
--username, -u username yes Suitest username. Same that you're using to login to Suitest.
--password, -p password yes Suitest password. Same that you're using to login to Suitest.
--org-id, -o orgId yes Id of the organization you want to login into.
--device-id, -d deviceId yes Device to which to connect interactively.
--app-config-id, -c appConfigId yes App configuration id to launch the app with.
--disallow-crash-reports disallowCrashReports no, defaults to false If a crash occurs an automatic report will be submitted to Suitest. These reports help Suitest to improve and update faster.
--log-dir logDir no If defined, logs will be stored inside specified folder.
--log-level logLevel no, defaults to normal Logger verbosity level. Values: silent, normal, verbose, debug
--continue-on-fatal-error continueOnFatalError no, defaults to false If the Suitest server responds with a fatal error to any command made by Suitest API, Suitest API will by default attempt to kill the process it is running in. You can turn this off by passing --continue-on-fatal-error and handle the shutdown yourself. Please note, that after a fatal server error occurs, all subsequent Suitest commands will fail until the connection is reinitialized with startTest().
--timestamp timestamp no, defaults to YYYY.MM.DD HH:mm:ss format Console log timestamp in YYYY.MM.DD HH:mm:ss format unless other format is specified. Supported formats can be found in time-stamp package. Timestamp can be turned off by passing 'none'.
--help - no Show CLI usage help.
--version - no Show Suitest JavaScript API library version.

Automated execution mode

suitest automated executes a Test Pack on multiple devices in parallel. For every device involved in this Test Pack - a new instance of test command is started in a separate process. Automated mode is intended for periodic test executions (for example when used in a CI process).

suitest automated \
    -k <token key> \
    -p <token password> \
    -t <test pack id> \
  node ./tests

Automated mode options

In CLI In .suitestrc Mandatory Description
--token-key, -k tokenKey yes Suitest token key.
--token-password, -p tokenPassword yes Suitest token password.
--test-pack-id, -t testPackId yes Id of the test pack to launch.
--config-override configOverride no A JSON object with config override.
--vcs-commit-hash vcsCommitHash no A VCS commit hash to associate this test run with.
--vcs-branch vcsBranch no A VCS branch name to associate this test run with.
--app-version appVersion no Version string of the tested application to associate this test run with.
--metadata metadata no Additional string data to associate this test run with.
--concurrency concurrency no, defaults to 0 Amount of simultaneously running tests. I.e: how many devices may be simultaneously busy with running this Test Pack.
0 = unlimited. Normally you should not specify this parameter.
--disallow-crash-reports disallowCrashReports no, defaults to false If a crash occurs an automatic report will be submitted to Suitest. These reports help Suitest to improve and update faster.
--log-dir logDir no If defined, logs will be stored inside specified folder.
--log-level logLevel no, defaults to normal Logger verbosity level. Values: silent, normal, verbose, debug
--continue-on-fatal-error continueOnFatalError no, defaults to false If the Suitest server responds with a fatal error to any command made by Suitest API, Suitest API will by default attempt to kill the process it is running in. You can turn this off by passing --continue-on-fatal-error and handle the shutdown yourself. Please note, that after a fatal server error occurs, all subsequent Suitest commands will fail until the connection is reinitialized with startTest().
--timestamp timestamp no, defaults to YYYY.MM.DD HH:mm:ss format Console log timestamp in YYYY.MM.DD HH:mm:ss format unless other format is specified. Supported formats can be found in time-stamp package. Timestamp can be turned off by passing 'none'.
--help - no Show CLI usage info.
--version - no Show Suitest JavaScript API library version.

The amount of tests allowed to run simultaneously varies per subscription. You can check the current limit in your profile summary. Suitest Test Launcher would however launch as many processes as there are devices in the Test Pack (unless limited by the --concurrency flag). If beginning another parallel test would cause exceeding your subscription limits, the corresponding process will idly wait for a free execution slot.

Launcher child processes

Suitest Test Launcher will always execute the test command in a separate process (in both interactive and automated run modes).

In automated mode the running instances don't know about one another and run completely independently.

The following environment variables are used internally by the launcher and it's child processes:

  • SUITEST_SESSION_TOKEN
  • SUITEST_SESSION_TYPE
  • SUITEST_DEVICE_ID
  • SUITEST_APP_CONFIG_ID

You should not rely on using these variables.

Enable E2E test debugging

To enable debugging in your End-To-End tests using the Test Launcher in interactive mode please refer to the E2E test debugging article.