Skip to main content

Runner

A runner in WebdriverIO orchestrates how and where tests are being run when using the testrunner. WebdriverIO currently supports two different types of runner: local and browser runner.

Local Runner​

The Local Runner initiates your framework (e.g. Mocha, Jasmine or Cucumber) within worker a process and runs all your test files within your Node.js environment. Every test file is being run in a separate worker process per capability allowing for maximum concurrency. Every worker process uses a single browser instance and therefore runs its own browser session allowing for maximum isolation.

Given every test is run in its own isolated process, it is not possible to share data across test files. There are two ways to work around this:

If nothing else is defined in the wdio.conf.js the Local Runner is the default runner in WebdriverIO.

Install​

To use the Local Runner you can install it via:

npm install --save-dev @wdio/local-runner

Setup​

The Local Runner is the default runner in WebdriverIO so there is no need to define it within your wdio.conf.js. If you want to explicitly set it, you can define it as follows:

// wdio.conf.js
export const {
    // ...
    runner: 'local',
    // ...
}

Browser Runner​

As opposed to the Local Runner the Browser Runner initiates and executes the framework within the browser. This allows you to run unit tests or component tests in an actual browser rather than in a JSDOM like many other test frameworks.

While JSDOM is widely used for testing purposes, it is in the end not an actual browser nor can you emulate mobile environments with it. With this runner WebdriverIO enables you to easily run your tests in the browser and use WebDriver commands to interact with elements rendered on the page.

Here is an overview of running tests within JSDOM vs. WebdriverIOs Browser Runner

JSDOMWebdriverIO Browser Runner
1.Runs your tests within Node.js using a re-implementation of web standards, notably the WHATWG DOM and HTML StandardsExecutes your test in an actual browser and runs the code in an environment that your users use
2.Interactions with components can only be imitated via JavaScriptYou can use the WebdriverIO API to interact with elements through the WebDriver protocol
3.Canvas support requires additional dependencies and has limitationsYou have access to the real Canvas API
4.JSDOM has some caveats and unsupported Web APIsAll Web APIs are supported as test run in an actual browser
5.Impossible to detect errors cross browserSupport for all browsers including mobile browser
6.Can not test for element pseudo statesSupport for pseudo states such as :hover or :active

This runner uses Vite to compile your test code and load it in the browser. It comes with presets for the following component frameworks:

  • React
  • Preact
  • Vue.js
  • Svelte
  • SolidJS

Every test file / test file group runs within a single page which means that between each test the page is being reloaded to guarantee isolation between tests.

Install​

To use the Browser Runner you can install it via:

npm install --save-dev @wdio/browser-runner

Setup​

To use the Browser runner, you have to define a runner property within your wdio.conf.js file, e.g.:

// wdio.conf.js
export const {
    // ...
    runner: 'browser',
    // ...
}

Runner Options​

The Browser runner allows following configurations:

preset​

If you test components using one of the mentioned frameworks above, you can define a preset that ensures everything is configured out of the box. This option can't be used together with viteConfig.

Type: vue | svelte | solid | react | preact
Example:

wdio.conf.js
export const {
    // ...
    runner: ['browser', {
        preset: 'svelte'
    }],
    // ...
}

viteConfig​

Define your own Vite configuration. You can either pass in a custom object or import an existing vite.conf.ts file if you use Vite.js for development. Note that WebdriverIO keeps custom Vite configurations to set up the test harness.

Type: string or UserConfig or (env: ConfigEnv) => UserConfig | Promise<UserConfig>
Example:

wdio.conf.ts
import viteConfig from '../vite.config.ts'

export const {
    // ...
    runner: ['browser', { viteConfig }],
    // or just:
    runner: ['browser', { viteConfig: '../vites.config.ts' }],
    // or use a function if your vite config contains a lot of plugins
    // which you only want to resolve when value is read
    runner: ['browser', {
        viteConfig: () => ({
            // ...
        })
    }],
    // ...
}

headless​

If set to true the runner will update capabilities to run tests headless. By default this is enabled within CI environments where a CI environment variable is set to '1' or 'true'.

Type: boolean
Default: false, set to true if CI environment variable is set

rootDir​

Project root directory.

Type: string
Default: process.cwd()

coverage​

WebdriverIO supports test coverage reporting through istanbul. See Coverage Options for more details.

Type: object
Default: undefined

Coverage Options​

The following options allow to configure coverage reporting.

enabled​

Enables coverage collection.

Type: boolean
Default: false

include​

List of files included in coverage as glob patterns.

Type: string[]
Default: [**]

exclude​

List of files excluded in coverage as glob patterns.

Type: string[]
Default:

[
  'coverage/**',
  'dist/**',
  'packages/*/test{,s}/**',
  '**/*.d.ts',
  'cypress/**',
  'test{,s}/**',
  'test{,-*}.{js,cjs,mjs,ts,tsx,jsx}',
  '**/*{.,-}test.{js,cjs,mjs,ts,tsx,jsx}',
  '**/*{.,-}spec.{js,cjs,mjs,ts,tsx,jsx}',
  '**/__tests__/**',
  '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build}.config.*',
  '**/.{eslint,mocha,prettier}rc.{js,cjs,yml}',
]

extension​

List of file extensions the report should include.

Type: string | string[]
Default: ['.js', '.cjs', '.mjs', '.ts', '.mts', '.cts', '.tsx', '.jsx', '.vue', '.svelte']

reportsDirectory​

Directory to write coverage report to.

Type: string
Default: ./coverage

reporter​

Coverage reporters to use. See istanbul documentation for detailed list of all reporters.

Type: string[]
Default: ['text', 'html', 'clover', 'json-summary']

perFile​

Check thresholds per file. See lines, functions, branches and statements for the actual thresholds.

Type: boolean
Default: false

clean​

Clean coverage results before running tests.

Type: boolean
Default: true

lines​

Threshold for lines.

Type: number
Default: undefined

functions​

Threshold for functions.

Type: number
Default: undefined

branches​

Threshold for branches.

Type: number
Default: undefined

statements​

Threshold for statements.

Type: number
Default: undefined

Limitations​

When using the WebdriverIO browser runner, it's important to note that thread blocking dialogs like alert or confirm cannot be used natively. This is because they block the web page, which means WebdriverIO cannot continue communicating with the page, causing the execution to hang.

In such situations, WebdriverIO provides default mocks with default returned values for these APIs. This ensures that if the user accidentally uses synchronous popup web APIs, the execution would not hang. However, it's still recommended for the user to mock these web APIs for better experience. Read more in Mocking.

Examples​

Make sure to check out the docs around component testing and have a look into the example repository for examples using these and various other frameworks.