Tests API

Execute tests and get the status of individual test runs.

List all tests

Returns a list of test descriptions containing the test’s id, name, created timestamp and folders.

Note

very test in an account is associated with the all folder.

Request

GET https://api.reflect.run/v1/tests

Response

{
  "tests": [
    {
      "id": 3072323,
      "name": "Registration Flow",
      "created": 1570546969108,
      "folders": [
        "all",
        "regression-tests"
      ]
    }
  ]
}

Response Fields

tests object: A list of test descriptions, including the test's id, name and folders.

Run a test

Immediately schedules a test.

The browser field allows you to specify the browser to run the test with. Supported values are: Chrome, Edge, Firefox, and Safari - if no value is specified Chrome is used.

Note

Your account must have access to the requested browser - requesting an unsupported browser will return a 400 error code.

API Hub for Test identifies certain properties of the execution as “reserved”. These reserved fields can be overridden with values provided by the API request:

hostnames: Allows you to specify a replacement hostname value for a given target hostname. You can also optionally specify a new port by including it in the replacement (see below for an example). If your override specifies a port, note the following behavior:
- If original includes a port value, it will only be applied on URLs of the test that also use that same port. For example, an original value of example.com:1234 would have no effect on https://example.com or https://example.com:5678.
- If original does not include a port value, it will match every URL of the test that uses the same hostname, regardless of port. For example, an original value of example.com would match both https://example.com and https://example.com:1234.
- If a port was specified in the URL of the test, it will be preserved unless a new value is specified in replacement or original included the port and replacement did not - in that case no port would be included in the resulting URL.
parameters: Allows you to specify a URL parameter key/value pair that will be included (or overwritten) on the test’s initial URL.
cookies: Allows you to specify cookies that will be set when loading the test’s initial URL. A cookie consists of the following fields:
- name (required string): The name of the cookie to be set.
- value (required string): The cookie’s value.
- domain (optional string): Host to which the cookie will be sent. Defaults to the host of the test’s initial URL.
- expires (optional number): Epoch timestamp (in milliseconds) indicating when the cookie should be deleted. If unspecified, the cookie becomes a session cookie.
- httpOnly (optional boolean): Indicates the cookie should be inaccessible to JavaScript on the page and only sent to the server.
- maxAge (optional number): Number of milliseconds until the cookie expires. If both expires and maxAge are set, maxAge has precedence.
- path (optional string): A URL path that must exist in a request URL in order to include the cookie in a request.
- secure (optional boolean): Indicates that the cookie should only be sent to the server when a request is made with https:.
headers: Allows you to specify headers that will be sent when loading the test’s initial URL.
- name (required string): The name of the header.
- value (required string): The header value.
- persist (optional boolean): Specifies whether this header should only be set in the initial request or for every subsequent HTTP request (ex: when overriding the Accept-Language header). If set to true, the Authorization header is only set for requests that match the hostname of the test’s original URL. Defaults to false.
localStorage and sessionStorage: Allows you to specify local and session storage values to be set on the test’s initial URL.

The variables field allows you to specify a definition for a variable used in the test.

REQUEST

POST https://api.reflect.run/v1/tests/<test-id>/executions

{
  "browser": "Chrome",
  "overrides": {
    "hostnames": [{
      "original": "prod.myapp.com",
      "replacement": "staging.myapp.com:1234"
    }],
    "parameters": [{
      "key": "token",
      "value": "abcdef"
    }],
    "cookies": [{
      "name": "my-favorite-cookie",
      "value": "chocolate-chip",
      "domain": "myapp.com",
      "expires": 123456789,
      "httpOnly": false,
      "maxAge": 123,
      "path": "/",
      "secure": true
    }],
    "headers": [{
      "name": "X-Custom-Header",
      "value": "custom-value"
    }],
    "localStorage": [{
      "key": "my-local-key",
      "value": "local-value"
    }],
    "sessionStorage": [{
      "key": "my-session-key",
      "value": "session-value"
    }]
  },
  "variables": {
    "username1": "user+${alpha(8)}@example.com",
    "password": "acompletelyunguessablepassword"
  },
  "emailFailures": true
}

REQUEST FIELDS

overrides optional object: Overrides to apply to properties of the test being executed.

variables optional object: Collection of ('name', 'definition') variable overrides to apply to this scheduled execution.

emailFailures optiona boolean: Controls whether email notifications should be sent for a failed test run. The default is true.

RESPONSE

{
  "executionId": 8939
}

RESPONSE FIELDS

executionId number: An identifier for the scheduled execution created by the request.

Get execution status

Returns a list of test result objects. The test result object includes the test’s id and a status field describing whether the test is “queued”, “running”, “succeeded”, or “failed”. Additionally, it contains a run object describing the properties of the test run such as its starting and ending time, and its video URL.

REQUEST

GET https://api.reflect.run/v1/executions/<execution-id>

RESPONSE

{
  "executionId": 17671,
  "browser": "Edge",
  "tests": [
    {
      "testId": 1690,
      "status": "succeeded",
      "run": {
        "runId": 690,
        "status": "passed",
        "variables": {},
        "startTime": 1654632014295,
        "endTime": 1654632027297,
        "runTime": 13002,
        "videoUrl": "https://reflect-videos.s3.amazonaws.com/690_89f4.mp4"
      }
    }
  ]
}

Response Fields

executionId number: An identifier for the scheduled execution.

browser string: The browser that the tests in the scheduled execution are configured to run on.

tests object: List of test results, including the testId and status.

tests.testId number: An identifier for the test.

tests.status string: One of: queued, running, succeeded, or failed. Summarizes the outcome of the test instance from its runs.

tests.run object: The test run from this execution.

tests.run.runId number: An identifier for the test run.

tests.run.status string: The outcome of the test run. Either passed or failed.

tests.run.variables object: The variables that were used or created by the test run.

tests.run.startTime number: Timestamp of when the test run started.

tests.run.endTime number: Timestamp of when the test run stopped.

tests.run.runTime number: Total time that the test run spent running (endTime - startTime).

tests.run.videoUrl string: The URL of a video recording of the test run.

In this section:

Tests API

List all tests

Note

Run a test

Note

Get execution status

Search results