How LoadComplete Calculates Metric Values

Applies to LoadComplete 4.97, last modified on May 20, 2019

During the test run, LoadComplete gathers various data on simulated operations and displays the statistics on the Runtime > Graphs page and in the test results. This topic explains how LoadComplete calculates runtime graph values and summary values.

How LoadComplete Calculates Metric Values

When processing the collected data, LoadComplete takes into account all the iterations of the appropriate requests, pages, transactions and scenarios.

When calculating the overall number of simulated requests, pages, transactions or scenarios, LoadComplete takes into account all their iterations, even those that were simulated with errors and warnings. For example, graphs that show the number of simulated scenarios will include the scenarios simulated with errors and the scenarios simulated incompletely.

When calculating the number of passed requests, pages, transactions and scenarios, LoadComplete considers only those iterations that passed successfully, without any errors or warnings. To learn which requests, pages and scenarios LoadComplete considers to be passed successfully, see below.

When calculating metrics that show duration and time values (for example, Page Load Time or Time to First Byte), LoadComplete takes into account only those iterations of requests, pages, transactions and scenarios that were simulated completely and correctly. It also takes into account requests, pages or transactions that violate SLA criteria (such requests, transactions and pages are considered successful).

When calculating metrics that show the overall number of errors and warning (for example, Errors and Warnings graphs on the Graphs page or Total Warnings and Total Errors metrics in the Report log), LoadComplete takes into account all the errors and warnings reported during the test run, including SLA violations.

Which Requests, Scenarios and Pages LoadComplete Considers Successful

  • A request is simulated completely and correctly if –

    • LoadComplete sends a request to the server successfully and

    • Receives a response from the server successfully and

    • The response code is treated as OK or as a warning according to the Expected HTTP Responses list (you can find it in the Scenario editor).

  • A scenario, a transaction or a page is simulated completely and correctly if all the requests belonging to this scenario or page are simulated correctly, that is –

    • All the requests are simulated and

    • All the responses are received and

    • There are no errors during the simulation.

  • A scenario that calls other scenarios is simulated completely and correctly if all the scenarios it calls have been simulated completely and correctly.

Requests, pages, transactions and scenarios that do not match these conditions are considered to be simulated incorrectly or incompletely.

See below for information on the reasons that make LoadComplete “think” that a request or scenario was not simulated completely, or was simulated with an error.

Why LoadComplete Reports Unknown Time

Unknown "Time to First Byte", "Time to Last Byte" and "Percent Response Time"

If a request is simulated incorrectly or incompletely during the test run, it is excluded from the calculation (requests violating SLA are considered to be passed and are not excluded). It is possible that no requests were simulated successfully and LoadComplete has no data to calculate time to first byte and time to last byte. In this case, it will report the unknown “Time to First Byte”,  “Time to Last Byte” and “Percent Response Time” values on the Summary log page.

Unknown "Page Load Time"

If a page contains at least one incomplete or incorrect request that was excluded from the calculation, the page is also excluded from the page load time metric calculation. If no page is considered to be simulated successfully, LoadComplete reports unknown page load time values on the Summary log page.

Unknown "Scenario Completion Time"

A scenario is excluded from calculation if:

  • It contains at least one incomplete or incorrect request.

– or –

  • It contains at least one error message posted by the Custom Message operation.

– or –

  • It was executed incompletely. This can happen if you terminated the scenario execution manually or if it was terminated by the Stop operation.

If a scenario called from another scenario is excluded from calculation, the scenario from which it is called is excluded from the calculation as well.

It is possible that all the scenarios of a test were executed incorrectly or incompletely and were excluded from calculation. In this case, LoadComplete reports an unknown scenario completion time.

Unknown "Transaction Completion Time"

LoadComplete reports an unknown transaction completion time in one of the following cases:

  • Test scenarios do not have any transactions defined.

  • All the transactions of the test were executed incorrectly or incompletely and were excluded from calculation. A transaction is excluded from calculation if:

    • It contains at least one incomplete or incorrect request.

    – or –

    • It contains at least one error message posted by the Custom Message operation.

    – or –

    • It was executed incompletely. This can happen if you terminated the execution manually, or if the execution was terminated by the Stop operation before the transaction was finished.

Incorrect or Incomplete Simulation

Below is a list of situations that can cause incomplete simulation of requests, pages, transactions and scenarios:

  • Errors that occur due to broken server connections. A scenario will be simulated partially if the connection to the server is broken and some requests are not sent.

  • Errors that occur due to slow request processing or transferring, that is, a situation when LoadComplete cannot receive a response to a request due to slow request processing on the server or to a network collapse. This includes the situations when LoadComplete terminates the request simulation on a timeout.

  • Incomplete simulation that occurs because the user terminates the test run. If you click Stop during the test execution, some requests will not be sent to the server.

  • Incomplete simulation that occurs because a scenario is terminated by the Stop operation.

  • Incomplete simulation that is caused by custom shape load or continuous load settings of your test (according to these settings, LoadComplete can terminate the virtual user simulation before the user scenario ends).

  • Errors that occur because the response code is specified as “Error” in the Expected HTTP Responses table.

    Note: By default, LoadComplete treats erroneous HTTP response codes (4xx and 5xx) as “warnings”. Requests for which LoadComplete receives these codes are not excluded from processing.
  • Incomplete simulation that is caused by authentication errors.

  • Errors that occurred due to failed validation of the response contents.

  • Errors caused by changes in the logic of the server work.

  • Other errors, including errors posted by the Custom Message operation.

See Also

Summary Page of the Report Panel
About Report Panel
Resolving Errors
About Test Results
Unknown Warnings Percentage and Errors Percentage

Highlight search results