Suitest Test Launcher

Suitest Test Launcher is a CLI tool, designed to ease the configuration and usage of Suitest JavaScript API. You can still use JavaScript API without the test launcher, but you'll have to handle the API bootstrapping process on your own. See Advanced usage for details.

What does the launcher do:

  • Makes it easier to submit configuration values to the API calls.
  • Manages simultaneous test execution on multiple devices.
  • Runs authentication requests and generates temporary tokens for the test runs.

Usage

CLI signature for the test launcher:

suitest run [options] [test command] [test command options]
  • options - modify the behavior of the execution.
  • test command - command to start the test run. For example node ./test.js.
  • test command options - options passed to the test command.

Launcher options

Example of test execution command:

suitest run \
    -k <your Token ID> \
    -p <your Token password> \
    -c <app config id> \
    -d <device id>
  node ./test.js
In CLI In the configuration file Mandatory Description
--app-config-id, -c appConfigId yes App configuration id to launch the app with.
--concurrency concurrency no Default: 0. Amount of simultaneously running tests. I.e: how many devices may be simultaneously busy with running this Test Pack.
0 = unlimited. Normally you should not specify this parameter.
--config-file configFile no Deprecated. Use --base-config-file or --override-config-file (overrideConfigFile). Path to config file to override the configuration file.
--base-config-file - no Path to config file to override the configuration file. Standard configuration file will be always ignored.
--override-config-file overrideConfigFile no Path to config file to override the configuration file config. Properties defined in the override config file have precedence over standard configuration file. If used with --base-config-file option, properties defined in override config file have precedence.
--default-timeout defaultTimeout no Default timeout for assertion line. Default set to 2000ms. Value set in ms.
--device-id, -d deviceId yes Device to which to connect interactively.
--disallow-crash-reports disallowCrashReports no, defaults to false If a crash occurs an automatic report will be submitted to Suitest. These reports help Suitest to improve and update faster.
--help, -h - no Show CLI usage help.
--include-changelist includeChangelist no If unapplied changes should be tested. Values: true, false. Default: false.
--inspect inspect no See debugging in Suitest JS API section.
--inspect-brk inspectBrk no See debugging in Suitest JS API section.
--log-dir logDir no If defined, logs will be stored inside specified folder.
--log-level logLevel no Logger verbosity level. Values: silent, normal, verbose, debug. Default: normal.
--preset preset no Preset for configuration and device. See presets documentation for more information.
--recording-option recordingOption no Session recording. Values: autostart, manualstart, none. Default: none.
--screenshot-dir screenshotDir no Location to store screenshots. Default: {screenshotDir}/{dateTime}-{currentFile}-l{currentLine}.png.
--timestamp timestamp no Default: YYYY.MM.DD HH:mm:ss format. Console log timestamp in YYYY.MM.DD HH:mm:ss format unless other format is specified. Supported formats can be found in time-stamp package. Timestamp can be turned off by passing 'none'.
--token-id, -k tokenId yes Your Suitest Token ID.
--token-password, -p tokenPassword yes Your Suitest Token password.
--token, -t token no One string instead of separate tokenId and tokenPassword. String format: tokenId:tokenPassword.
--version, -v - no Show Suitest JavaScript API library version.
--webhook-url webhookURL no Register a webhook which should be called when video recording is ready.
Concurrency - simultaneously running test limit

The amount of tests allowed to run simultaneously varies per subscription. You can check the current limit in your profile summary. Suitest Test Launcher would however launch as many processes as there are devices in the Test Pack (unless limited by the --concurrency flag). If beginning another parallel test would cause exceeding your subscription limits, the corresponding process will idly wait for a free execution slot.

Launcher child processes

Suitest Test Launcher will always execute the test command in a separate process.

The environment variable used internally by the launcher and its child processes:

  • SUITEST_CHILD_PROCESS

You should not rely on using this variable.

Available environment variables

name description
SUITEST_DEVICE_ID contains currently selected device id
SUITEST_PRESET_NAME if presets are used it contains preset name

Enable E2E test debugging

Debugging is available only for tests running on a single device.

To enable debugging in your End-To-End tests using the Test Launcher please refer to the E2E test debugging article.

Using presets

When testing on multiple platforms and devices, you do not want to always manually select the configuration and devices. Instead, you can use so called presets and select the right predefined one by one of the following ways:

  • Using option --preset <preset name> when executing a script (can use multiple presets):
suitest run --preset preset1 --preset preset2 node

{
  "tokenId": "yourTokenId",
  "tokenPassword": "yourTokenPassword",
  "logDir": "/path/to/logs",
  // ... other properties ...
  "preset": ["preset1"], // which presets you want to execute
  "presets": { // all presets definitions
    "preset1": {
      "device": "yourDeviceId",
      "config": "yourConfigurationId",
    },
    "preset2": {/*...*/},
  },
}