Possible Issues With Web Testing
Microsoft Edge
-
The Edge web browser process is not terminated when you close the browser, it remains in the system. As a result, the following issues may occur in your tests:
-
When your test accesses the current web browser in your system, it may return the suspended Edge instance.
-
The Edge browser may fail to navigate to the needed web page and will show an empty page instead.
To avoid possible issues, we recommend that you close all existing instances of Edge before running your web tests. To do this, you can use the following code:
C#
using SmartBear.TestLeft;
using SmartBear.TestLeft.TestObjects;
using SmartBear.TestLeft.TestObjects.Web;
…
public void Test()
{
IWebBrowser edgeBrowser;
Driver.TryFind<IWebBrowser>(new WebBrowserPattern()
{
ObjectIdentifier = "edge"
}, 3, out edgeBrowser);
if (edgeBrowser != null)
{
edgeBrowser.Close();
}
}Visual Basic .NET
Imports SmartBear.TestLeft
Imports SmartBear.TestLeft.TestObjects
Imports SmartBear.TestLeft.TestObjects.Web
…
Public Sub Test()
Dim edgeBrowser As IWebBrowser
Driver.TryFind(Of IWebBrowser)(New WebBrowserPattern() With {
.ObjectIdentifier = "edge"
}, 3, edgeBrowser)
If edgeBrowser IsNot Nothing Then
edgeBrowser.Close()
End If
End SubJava
import com.smartbear.testleft.*;
import com.smartbear.testleft.testobjects.*;
…
public void Test() throws Exception{
WebBrowser edgeBrowser = (WebBrowser) driver.tryFind(WebBrowser.class, new WebBrowserPattern(){{
ObjectIdentifier = "edge";
}}, 3).get();
if (edgeBrowser != null)
{
edgeBrowser.close();
}
} -
-
When simulating clicks over control items by using the
ClickItem
method, TestLeft sometimes sets the selected item through the property value instead of simulating a click. -
Sometimes, TestLeft fails to simulate clicking items of jQuery controls. To make sure a click is simulated, check that the needed item is selected, and if it is not, simulate the click again.
-
TestLeft may fail to simulate the first click action after the browser is restored from the minimized state. To make sure the click is simulated, check the click result and simulate the click again, if needed.
-
TestLeft cannot get page contents if the Content-Security-Policy response header is set to unsafe-inline. To avoid this problem, do not use the Content-Security-Policy header in your tested application or use another browser for testing.
-
TestLeft cannot simulate keyboard input into Input Method Editors (IMEs) in Edge.
Google Chrome
-
Starting from version 69, the
ThirdPartyBlockingEnabled
security policy prevents any third-party modules, including TestLeft testing modules, from injecting into Chrome processes. As a result, you may face issues when simulating user actions on processes launched from Google Chrome (for instance, on the Open File and Save File dialogs).To avoid possible issues, you can disable the policy:
-
On your computer, open the Registry editor. To do this, click the Start or Windows button, click Run, type regedit, and press Enter.
-
In the Registry editor, locate the following key:
Software\Policies\Google\Chrome
If this key does not exist, create it.
-
Make sure the
ThirdPartyBlockingEnabled
value of the key is set to false (0).If the value does not exist, create the DWORD value
ThirdPartyBlockingEnabled
and set it to 0.
If your Google Chrome browser settings are managed via group policies, ask your system administrator to update the policies to disable the
ThirdPartyBlockingEnabled
policy.To learn more about Chrome security policies, see chromium.org/administrators/policy-list-3.
-
-
If Chrome cannot verify an SSL certificate provided by a tested website, it shows the "Your connection is not private" error message page. Due to specifics of this message implementation in Chrome, TestLeft cannot interact with this message page. It cannot access internal objects of the message and cannot simulate any user actions over it.
-
Sometimes, TestLeft Pick Object and Point and Fix tools are unable to select the needed target object from screen.
-
Searching for objects by using XPath may work too slowly in Chrome. In this case, use the
Find
orQuerySelector
method to find web objects. -
After you change the options of the SmartBear Test Extension, restart your Chrome web browser. Otherwise, TestLeft will not be able to access web pages that are open in Chrome.
-
If you face playback performance problems when testing web pages with a large number of objects, use the --disable-renderer-accessibility command-line argument. It can improve web testing performance because it disables creating MSAA/UIA web page objects.
Chromium Embedded Framework
TestLeft cannot simulate clicking items of list box controls in applications based on CEF3 builds 1650 and 1750 by using the ClickItem
method. This behavior is caused by an issue in the Chromium engine and was fixed in CEF build 1916. As a workaround, you can select items of list box controls by calling the Keys
method:
C#
using SmartBear.TestLeft;
using SmartBear.TestLeft.TestObjects;
using SmartBear.TestLeft.TestObjects.Web;
…
public void Test()
{
IComboBox listbox = Driver.Find<IProcess>(new ProcessPattern()
{
ProcessName = "CEF_App",
Index = 1
}).Find<IComboBox>(new WebElementPattern()
{
ObjectType = "Select"
}, 10);
listbox.Keys("[Down][Down]"); // Selects the second item in the list
listbox.Keys("myItem"); // Selects the needed item
}
Visual Basic .NET
Imports SmartBear.TestLeft
Imports SmartBear.TestLeft.TestObjects
Imports SmartBear.TestLeft.TestObjects.Web
…
Public Sub Test()
Dim listbox As IComboBox = Driver.Find(Of IProcess)(New ProcessPattern() With {
.ProcessName = "CEF_App",
.Index = 1
}).Find(Of IComboBox)(New WebElementPattern() With {
.ObjectType = "Select"
}, 10)
listbox.Keys("[Down][Down]") ' Selects the second item in the list
listbox.Keys("myItem") ' Selects the needed item
End Sub
Java
import com.smartbear.testleft.*;
import com.smartbear.testleft.testobjects.*;
…
public void Test() throws Exception{
ComboBox listbox = driver.find(TestProcess.class, new ProcessPattern() {{
ProcessName = "CEF_App";
Index = 1;
}}).find(ComboBox.class, new WebElementPattern() {{
ObjectType = "Select";
}}, 10);
listbox.keys("[Down][Down]"); // Selects the second item in the list
listbox.keys("myItem"); // Selects the needed item
}
Possible Issues With Desktop Applications
Visual C++ Applications
TestLeft cannot get access to the application’s internal members
If TestLeft cannot access the internals of the Visual C++ application under test, check whether the tested application is compiled with debug information and whether generation of runtime type information (RTTI) is enabled.
- To access public, protected and private properties and methods of your Visual C++ application from your tests, compile your application with debug information.
- To access public members of MFC classes, enable generation of runtime type information.
For a detailed description of how to configure your compiler, see Preparing Visual C++ Applications for Testing.
Reading an application’s debug information takes a lot of time
The delay occurs when TestLeft is reading debug information for the first time. The amount of time TestLeft spends on reading the debug information depends on the size of the debug information. Large applications have a lot of debug info, and reading it can significantly slow down the test run.
If you do not use protected and private methods and fields in your tests, compile your application without debug information to speed up the tests.
Delphi and C++Builder Applications
TestLeft cannot get access to applications’ internal members
If TestLeft cannot access the internals of a Delphi or C++Builder application under test:
-
Check whether the tested application is compiled with debug information.
To access public, protected and private properties and methods of your application from tests, compile it with debug information:
Preparing C++Builder Applications for Testing
Preparing Delphi Applications for Testing
If your application is compiled without debug information, TestLeft can access only published properties of the application.
-
Make sure that the application is not compiled with a tool that may change the debug information format.
TestLeft is incompatible with tools that change the debug information format. An example of such a tool is EurekaLog 7. We recommend that you disable EurekaLog when compiling an application that is aimed for automated testing with TestLeft.
- Check whether the tested object of the application is visible.
The methods and properties provided by TestLeft can be applied to objects (specifically to VCL objects) which are currently visible. If your tested object is invisible in your application, you will not be able to call the properties and methods provided by TestLeft. As a workaround, you can –
-
Call the object’s native properties and methods instead.
-
Make sure the object is visible before callings its properties and methods provided by TestLeft. (You may need to simulate user actions over your application to make the object visible.)
-
TestLeft currently does not support access to public, protected and private members of VCL controls in 64-bit C++Builder applications. |
Java and JavaFX Applications
TestLeft cannot get access to Java or JavaFX application’s internals
TestLeft cannot get access to internal methods and properties of Java and JavaFX objects if the tested application was launched with the -verbose or -verbose:class command-line argument.
To resolve this issue, launch your application without these arguments.
Tests run with errors when using the Windows look and feel
If your Java application uses the Java Swing ComboBox, Java Swing List or Java Swing Tree control and the com.sun.java.swing.plaf.windows.WindowsLookAndFeel
look and feel, launch your application with the swing.disablevistaanimation
command-line parameter set to True, or specify the parameter in the application code. Otherwise, errors may occur when playing back your tests.
Here is an example of how you can specify this parameter:
An exception occurs when debugging a Java or JavaFX application
When you are debugging a Java or a JavaFX application and TestLeft is running, an exception may occur in the application. When getting access to internal methods and fields, TestLeft may load classes to memory and the debugger may fail to find the appropriate class reference. To resolve this issue, do not debug your application when TestLeft is running, or do not run your application under a debugger.
The "Could not create the Java virtual machine" error message is displayed
During automated testing, when you start a Java application with JDK (or JRE) v. 1.6_18, the "Could not create the Java virtual machine." error message may be displayed. The problem is caused by insufficient memory allocation. 256 Mb of memory is sufficient for a Java application, but the Java 1.6_18 build apparently requires more memory.
Since TestLeft loads its helper modules to the address space of the tested application, the application’s memory consumption increases. So, loading additional modules to the application sometimes can lead to exceeding the 256 Mb bound.
To avoid this problem, it is recommended to update the Java console with the -Xmx512M parameter to increase memory allocation to 512 Mb.
Qt Applications
The context menu items and their subitems may be recognized as siblings
This behavior is caused by the following reason: invoked subitems are not child items of the control’s context menu items to which they correspond (their Parent
is an empty object).
For tests to run correctly, the tested application’s developer must associate the context menu items with the appropriate subitems in the application.
A control’s popup menu may be recognized incorrectly.
Due to the specifics of Qt applications, it is impossible to determine which menu is a control’s popup menu. Because of this, TestLeft considers the first child menu of a control as its popup menu. If the control has more than one child menu, TestLeft may recognize the popup menu incorrectly.
To avoid this, you can use one of the following solutions:
-
Make sure the popup menu is the only child menu of the control
-
Use the
QtObject
method to get to the popup menu. The method is available through theIObject
interface in .NET, and theTestObject
interface in Java. To learn how to call such methods, see Methods and Properties Common for All Test Objects.To avoid wrong recognition, specify the menu name in the
windowsTitle
property in your Qt application:C++
pMenuObj->setWindowTitle("MyWndPopupMenuName");
In this case, you can get the popup menu by using the following code:
C#
using SmartBear.TestLeft;
using SmartBear.TestLeft.TestObjects;
using SmartBear.TestLeft.TestObjects.Qt;
…
public void Test()
{
IProcess qtTestApp = Driver.Find<IProcess>(new ProcessPattern()
{
ProcessName = "QtSample"
});
IMenu menu = qtTestApp.CallMethod<IQtMenu>("QtObject", "QMenu", "MyWndPopupMenuName").QtMenu;
…
}Visual Basic .NET
Imports SmartBear.TestLeft
Imports SmartBear.TestLeft.TestObjects
Imports SmartBear.TestLeft.TestObjects.Qt
…
Public Sub Test()
Dim qtSampleApp As IProcess = Driver.Find(Of IProcess)(New ProcessPattern() With {
.ProcessName = "QtSample"
})
Dim menu As IMenu = qtSampleApp.CallMethod(Of IQtMenu)("QtObject", "QMenu", "MyWndPopupMenuName").QtMenu
…
End SubJava
import com.smartbear.testleft.*;
import com.smartbear.testleft.testobjects.*;
import com.smartbear.testleft.testobjects.qt.*;
…
public void Test() throws Exception{
TestProcess qtSampleApp = driver.find(TestProcess.class, new ProcessPattern() {{
ProcessName = "QtSample";
}});
Menu menu = qtSampleApp.callMethod(QtMenu.class, "QtObject", "QMenu", "MyWndPopupMenuName");
…
}