Setting Reasonable Goals for Service Level Agreement

One of key factors for a web application success is the load time speed. Keeping the website load time to a minimum will satisfy your users. Once the website load time starts increasing, you may lose visitors.

With LoadComplete, you can monitor your website page load time, time to first byte and other performance metrics during the test run (see About Server Monitoring). In addition, you can define service level agreement (SLA) thresholds for your website and check if the performance of your website exceeds these thresholds.

SLA thresholds include the maximum acceptable page load time, time to first byte, and duration metrics. You can set common thresholds for all pages, requests, and WebSocket messages in your load test, as well as custom thresholds for individual pages, requests, and other operations. For more information, see Setting Service Level Agreement (SLA) Criteria and SLA Settings.

The rest of this topic contains tips on defining SLA thresholds for your website load tests.

Setting the correct threshold gives less false-positive and more real signals about an unacceptable level of response time. To specify the reasonable threshold level, you need to know the average load time for your web site. To measure your website load time with LoadComplete:

1. Record a user scenario.

2. Play back the recorded scenario using one virtual user. During the scenario run, LoadComplete will measure the website page load time and time to first byte.

3. In the load test results, open the Report > Response Time page. Here you can see graphs that show how the web site page load time and time to first byte values changed during the test run. Use these values as the basis when specifying SLA thresholds.

   To get the most probable values, you can play back the recorded scenario several times and then calculate the average page load time and time to first byte values.

The key principle you must follow when specifying the SLA thresholds is using the website page load time measured by LoadComplete as the basic time.

You may ask why you cannot measure the time spent loading the page manually, just by noting down the time spent loading the page in a web browser, and use this time as the basis for the threshold. This approach is not suitable for the following reasons:

• When measuring the time spent on loading a page in a web browser manually, one may consider the page fully loaded although the browser may still continue loading some resources, such as images, scripts, and stylesheets, which will result in incorrect measurements.

• The time required to load a page in the browser can differ from the page load time that LoadComplete measures. LoadComplete may need more or less time to load the page. This depends on the following:

  • In contrast to a web browser, LoadComplete neither renders the page nor parses it. So, it takes LoadComplete less time to download the resources.

  • LoadComplete requires additional time to handle dynamic parameters. This time is included in the response time.

  • The web browser uses cache data to load web pages faster while LoadComplete fully loads all of the web page’s resources each time.

  • During a massive load on the server, for example, 100 virtual users, the time the browser required to load a web page can be less than the time that LoadComplete measures. The fact is that the real behavior of users differs from the behavior LoadComplete simulates. Although LoadComplete simulates requests sequentially, it simulates virtual users quite synchronously. So, the load on the server at a certain point of time can be higher than in a real situation, and the load time can increase.

  So, LoadComplete measures the page load time in a way different from that of the browser. If you use the time spent loading a page in the browser as a threshold, it is most probable that real-time values that LoadComplete measures will exceed the threshold and your test will fail.