Applies to TestLeft 4.20, last modified on September 11, 2019

Possible Issues With Web Testing

Microsoft Edge

  • Starting with October 2018 Update of Windows 10, the Edge web browser process is not terminated when you close the browser, it remains in the system. As a result, the following issues may occur in your tests:

    • When your test accesses the current web browser in your system, it may return the suspended Edge instance.

    • The Edge browser may fail to navigate to the needed web page and will show an empty page instead.

    To avoid possible issues, we recommend that you close all existing instances of Edge before running your web tests. To do this, you can use the following code:

    C#

    using SmartBear.TestLeft;
    using SmartBear.TestLeft.TestObjects;
    using SmartBear.TestLeft.TestObjects.Web;


    public void Test()
    {
      IWebBrowser edgeBrowser;
      Driver.TryFind<IWebBrowser>(new WebBrowserPattern()
      {
        ObjectIdentifier = "edge"
      }, 3, out edgeBrowser);
      if (edgeBrowser != null)
      {
        edgeBrowser.Close();
      }
    }

    Visual Basic .NET

    Imports SmartBear.TestLeft
    Imports SmartBear.TestLeft.TestObjects
    Imports SmartBear.TestLeft.TestObjects.Web


    Public Sub Test()
      Dim edgeBrowser As IWebBrowser
      Driver.TryFind(Of IWebBrowser)(New WebBrowserPattern() With {
          .ObjectIdentifier = "edge"
        }, 3, edgeBrowser)
      If edgeBrowser IsNot Nothing Then
        edgeBrowser.Close()
      End If
    End Sub

    Java

    import com.smartbear.testleft.*;
    import com.smartbear.testleft.testobjects.*;


    public void Test() throws Exception{

      WebBrowser edgeBrowser = (WebBrowser) driver.tryFind(WebBrowser.class, new WebBrowserPattern(){{
        ObjectIdentifier = "edge";
      }}, 3).get();
      if (edgeBrowser != null)
      {
        edgeBrowser.close();
      }
    }

  • When simulating clicks over control items by using the ClickItem method, TestLeft sometimes sets the selected item through the property value instead of simulating a click.

  • Sometimes, TestLeft fails to simulate clicking items of jQuery controls. To make sure a click is simulated, check that the needed item is selected, and if it is not, simulate the click again.

  • TestLeft may fail to simulate the first click action after the browser is restored from the minimized state. To make sure the click is simulated, check the click result and simulate the click again, if needed.

  • TestLeft cannot get page contents if the Content-Security-Policy response header is set to unsafe-inline. To avoid this problem, do not use the Content-Security-Policy header in your tested application or use another browser for testing.

Google Chrome

  • Starting from version 69, the ThirdPartyBlockingEnabled security policy prevents any third-party modules, including TestLeft testing modules, from injecting into Chrome processes. As a result, you may face issues when simulating user actions on processes launched from Google Chrome (for instance, on the Open File and Save File dialogs).

    To avoid possible issues, you can disable the policy:

    1. On your computer, open the Registry editor. To do this, click the Start or Windows button, click Run, type regedit, and press Enter.

    2. In the Registry editor, locate the following key:

      Software\Policies\Google\Chrome

      If this key does not exist, create it.

    3. Make sure the ThirdPartyBlockingEnabled value of the key is set to false (0).

      If the value does not exist, create the DWORD value ThirdPartyBlockingEnabled and set it to 0.

    If your Google Chrome browser settings are managed via group policies, ask your system administrator to update the policies to disable the ThirdPartyBlockingEnabled policy.

    To learn more about Chrome security policies, see chromium.org/administrators/policy-list-3.

  • If Chrome cannot verify an SSL certificate provided by a tested website, it shows the "Your connection is not private" error message page. Due to specifics of this message implementation in Chrome, TestLeft cannot interact with this message page. It cannot access internal objects of the message and cannot simulate any user actions over it.

  • Sometimes, TestLeft Pick Object and Point and Fix tools are unable to select the needed target object from screen.

  • Searching for objects by using XPath may work too slowly in Chrome. In this case, use the Find or QuerySelector method to find web objects.

  • After you change the options of the SmartBear Test Extension, restart your Chrome web browser. Otherwise, TestLeft will not be able to access web pages that are open in Chrome.

Chromium Embedded Framework

TestLeft cannot simulate clicking items of list box controls in applications based on CEF3 builds 1650 and 1750 by using the ClickItem method. This behavior is caused by an issue in the Chromium engine and was fixed in CEF build 1916. As a workaround, you can select items of list box controls by calling the Keys method:

C#

using SmartBear.TestLeft;
using SmartBear.TestLeft.TestObjects;
using SmartBear.TestLeft.TestObjects.Web;


public void Test()
{
  IComboBox listbox = Driver.Find<IProcess>(new ProcessPattern()
  {
    ProcessName = "CEF_App",
    Index = 1
  }).Find<IComboBox>(new WebElementPattern()
  {
    ObjectType = "Select"
  }, 10);

  listbox.Keys("[Down][Down]"); // Selects the second item in the list
  listbox.Keys("myItem"); // Selects the needed item
}

Visual Basic .NET

Imports SmartBear.TestLeft
Imports SmartBear.TestLeft.TestObjects
Imports SmartBear.TestLeft.TestObjects.Web


Public Sub Test()
  Dim listbox As IComboBox = Driver.Find(Of IProcess)(New ProcessPattern() With {
          .ProcessName = "CEF_App",
          .Index = 1
    }).Find(Of IComboBox)(New WebElementPattern() With {
          .ObjectType = "Select"
    }, 10)

  listbox.Keys("[Down][Down]") ' Selects the second item in the list
  listbox.Keys("myItem") ' Selects the needed item
End Sub

Java

import com.smartbear.testleft.*;
import com.smartbear.testleft.testobjects.*;


public void Test() throws Exception{

  ComboBox listbox = driver.find(TestProcess.class, new ProcessPattern() {{
    ProcessName = "CEF_App";
    Index = 1;
  }}).find(ComboBox.class, new WebElementPattern() {{
    ObjectType = "Select";
  }}, 10);

  listbox.keys("[Down][Down]"); // Selects the second item in the list
  listbox.keys("myItem"); // Selects the needed item
}

Possible Issues With Desktop Applications

Visual C++ Applications

TestLeft cannot get access to the application’s internal members

If TestLeft cannot access the internals of the Visual C++ application under test, check whether the tested application is compiled with debug information and whether generation of runtime type information (RTTI) is enabled.

  • To access public, protected and private properties and methods of your Visual C++ application from your tests, compile your application with debug information.
  • To access public members of MFC classes, enable generation of runtime type information.

For a detailed description of how to configure your compiler, see Preparing Visual C++ Applications for Testing.

Reading an application’s debug information takes a lot of time

The delay occurs when TestLeft is reading debug information for the first time. The amount of time TestLeft spends on reading the debug information depends on the size of the debug information. Large applications have a lot of debug info, and reading it can significantly slow down the test run.

If you do not use protected and private methods and fields in your tests, compile your application without debug information to speed up the tests.

Delphi and C++Builder Applications

TestLeft cannot get access to applications’ internal members

If TestLeft cannot access the internals of a Delphi or C++Builder application under test:

  • Check whether the tested application is compiled with debug information.

    To access public, protected and private properties and methods of your application from tests, compile it with debug information:

    Preparing C++Builder Applications for Testing

    Preparing Delphi Applications for Testing

    If your application is compiled without debug information, TestLeft can access only published properties of the application.

  • Make sure that the application is not compiled with a tool that may change the debug information format.

    TestLeft is incompatible with tools that change the debug information format. An example of such a tool is EurekaLog 7. We recommend that you disable EurekaLog when compiling an application that is aimed for automated testing with TestLeft.

  • Check whether the tested object of the application is visible.

    The methods and properties provided by TestLeft can be applied to objects (specifically to VCL objects) which are currently visible. If your tested object is invisible in your application, you will not be able to call the properties and methods provided by TestLeft. As a workaround, you can –

    • Call the object’s native properties and methods instead.

    • Make sure the object is visible before callings its properties and methods provided by TestLeft. (You may need to simulate user actions over your application to make the object visible.)

TestLeft currently does not support access to public, protected and private members of VCL controls in 64-bit C++Builder applications.

Java and JavaFX Applications

TestLeft cannot get access to Java or JavaFX application’s internals

TestLeft cannot get access to internal methods and properties of Java and JavaFX objects if the tested application was launched with the -verbose or -verbose:class command-line argument.

To resolve this issue, launch your application without these arguments.

Tests run with errors when using the Windows look and feel

If your Java application uses the Java Swing ComboBox, Java Swing List or Java Swing Tree control and the com.sun.java.swing.plaf.windows.WindowsLookAndFeel look and feel, launch your application with the swing.disablevistaanimation command-line parameter set to True, or specify the parameter in the application code. Otherwise, errors may occur when playing back your tests.

Here is an example of how you can specify this parameter:

java.exe -Dswing.disablevistaanimation=true -cp MyApp.jar MyApp.Main
An exception occurs when debugging a Java or JavaFX application

When you are debugging a Java or a JavaFX application and TestLeft is running, an exception may occur in the application. When getting access to internal methods and fields, TestLeft may load classes to memory and the debugger may fail to find the appropriate class reference. To resolve this issue, do not debug your application when TestLeft is running, or do not run your application under a debugger.

The "Could not create the Java virtual machine" error message is displayed

During automated testing, when you start a Java application with JDK (or JRE) v. 1.6_18, the "Could not create the Java virtual machine." error message may be displayed. The problem is caused by insufficient memory allocation. 256 Mb of memory is sufficient for a Java application, but the Java 1.6_18 build apparently requires more memory.

Since TestLeft loads its helper modules to the address space of the tested application, the application’s memory consumption increases. So, loading additional modules to the application sometimes can lead to exceeding the 256 Mb bound.

To avoid this problem, it is recommended to update the Java console with the -Xmx512M parameter to increase memory allocation to 512 Mb.

Qt Applications

See Also

Troubleshooting
Windows Settings

Highlight search results