How LoadComplete Calculates Metric Values

How LoadComplete Calculates Metric Values

Which Requests, Scenarios and Pages LoadComplete Considers Successful

Why LoadComplete Reports Unknown Time

Incorrect or Incomplete Simulation

When processing the collected data, LoadComplete takes into account all the iterations of the appropriate requests, pages, transactions and scenarios.

When calculating the overall number of simulated requests, pages, transactions or scenarios, LoadComplete takes into account all their iterations, even those that were simulated with errors and warnings. For example, graphs that show the number of simulated scenarios will include the scenarios simulated with errors and the scenarios simulated incompletely.

When calculating the number of passed requests, pages, transactions and scenarios, LoadComplete considers only those iterations that passed successfully, without any errors or warnings. To learn which requests, pages and scenarios LoadComplete considers to be passed successfully, see below.

When calculating metrics that show duration and time values (for example, Page Load Time or Time to First Byte), LoadComplete takes into account only those iterations of requests, pages, transactions and scenarios that were simulated completely and correctly. It also takes into account requests, pages or transactions that violate SLA criteria (such requests, transactions and pages are considered successful).

When calculating metrics that show the overall number of errors and warning (for example, Errors and Warnings graphs on the Graphs page or Total Warnings and Total Errors metrics in the Report log), LoadComplete takes into account all the errors and warnings reported during the test run, including SLA violations.

• A request is simulated completely and correctly if –

  • LoadComplete sends a request to the server successfully and

  • Receives a response from the server successfully and

  • The response code is treated as OK or as a warning according to the Expected HTTP Responses list (you can find it in the Scenario editor).

• A scenario, a transaction or a page is simulated completely and correctly if all the requests belonging to this scenario or page are simulated correctly, that is –

  • All the requests are simulated and

  • All the responses are received and

  • There are no errors during the simulation.

• A scenario that calls other scenarios is simulated completely and correctly if all the scenarios it calls have been simulated completely and correctly.

Requests, pages, transactions and scenarios that do not match these conditions are considered to be simulated incorrectly or incompletely.

See below for information on the reasons that make LoadComplete “think” that a request or scenario was not simulated completely, or was simulated with an error.

Below is a list of situations that can cause incomplete simulation of requests, pages, transactions and scenarios:

• Errors that occur due to broken server connections. A scenario will be simulated partially if the connection to the server is broken and some requests are not sent.

• Errors that occur due to slow request processing or transferring, that is, a situation when LoadComplete cannot receive a response to a request due to slow request processing on the server or to a network collapse. This includes the situations when LoadComplete terminates the request simulation on a timeout.

• Incomplete simulation that occurs because the user terminates the test run. If you click Stop during the test execution, some requests will not be sent to the server.

• Incomplete simulation that occurs because a scenario is terminated by the Stop operation.

• Incomplete simulation that is caused by custom shape load or continuous load settings of your test (according to these settings, LoadComplete can terminate the virtual user simulation before the user scenario ends).

• Errors that occur because the response code is specified as “Error” in the Expected HTTP Responses table.

  Note:

  By default, LoadComplete treats erroneous HTTP response codes (4xx and 5xx) as “warnings”. Requests for which LoadComplete receives these codes are not excluded from processing.

• Incomplete simulation that is caused by authentication errors.

• Errors that occurred due to failed validation of the response contents.

• Errors caused by changes in the logic of the server work.

• Other errors, including errors posted by the Custom Message operation.