Applies to TestLeft 4.10, last modified on May 28, 2019

General Limitations

  • The computer where the tested application is located must have either TestLeft, TestExecute or TestComplete installed. See How It Works.

  • TestLeft turns off the Caps Lock key at the beginning of the test run to make sure text input is simulated correctly.

  • TestLeft tests that simulate user actions over a graphical user interface cannot run on locked computers. See Running Tests on Locked Computers.

Limitations for Web Browsers and Web Applications

Microsoft Edge

  • To test Microsoft Edge, TestLeft must be installed into a subfolder of the <Program Files> folder (or <Program Files(x86)> on 64-bit operating systems).

  • Microsoft Edge does not support the ActiveX technology, which means that Microsoft Silverlight is not supported either.

    To test Silverlight applications, use other browsers.

  • TestLeft does not support MSAA for testing Flash and Flex applications running in Microsoft Edge.

  • Standard dialogs (for example, Open File, Save File, Select Folder, and others) you call in the Edge browser are children of a separate PickerHost helper process, not the Edge process. To create a cross-browser test that simulates actions against these dialogs, you need to address those dialogs in the test properly.

  • The Authorization dialog is implemented as a separate page, not as a browser dialog.

  • Due to security restrictions, the Edge browser prohibits certain actions when local web pages are open in it (that is, web pages stored on your local computer). It does not display linked images, nor does it load data for framesets that display local files.

Internet Explorer

  • The Modern UI (Metro) versions of Internet Explorer 10 and 11 are not supported.

  • TestLeft does not support MSAA for testing Flash and Flex applications running in Internet Explorer 11 on Windows 8 and later.

  • After closing a tab in Internet Explorer 11 or navigating a tab to another page, the old Page object may remain in the browser’s object hierarchy for some time. This is the actual lifetime of browser tab objects in Internet Explorer.

  • TestLeft may be unable to access the browser’s Login dialog if it is invoked by a domain other than the domain to which the current web page belongs. To work around this limitation, before accessing protected resources, navigate to any web page on that domain that does not require user authentication.

Chrome

  • Testing Silverlight applications in Chrome is not currently supported.

  • Chrome running in Modern UI (Metro) mode is not supported.

  • Page hierarchy is not exposed if JavaScript is disabled in Chrome. The SmartBear Test Extension embeds its scripts into the page, so, scripting must be enabled in the browser. JavaScript can be enabled (globally or for the tested site) on the Content Settings page in Chrome (chrome://settings/content).

  • If the script injection is restricted by Content Security Policy (CSP) in your tested web application, TestLeft will not be able to access internal objects of the application.

  • Due to limitations of Chrome’s native messaging protocol, the size of parameters passed to web page scripts cannot exceed 1 megabyte.

  • While performing long-term automated tests, Chrome’s hang monitor can consider the SmartBear Test Extension to be stalled and re-launch it.

    To prevent such behavior, TestLeft disables the hang monitor when Chrome is run using the following methods:

    .NET: Driver.Applications.RunBrowser.

    Java: driver.getApplications().runBrowser.

    If you launch Chrome in some other way, we recommend disabling the monitor by appending the --disable-hang-monitor command-line switch to the Chrome executable (chrome.exe).

  • For security considerations Chrome refuses all extensions to inject code into its internal pages (about:*, chrome:*). Because of that, internal pages, like the new tab page, the extension manager, the download manager, the history page and so on, are not accessible to TestLeft.

Embedded Browsers

Common for WebBrowser and CEF-Based Applications
  • Web tests created for applications with embedded browsers are not cross-browser tests (as they depend on the position of the embedded browser control within the object hierarchy). That is, for the same web page displayed in a standard and in an embedded browser, the path to the Page object varies, while the child objects are the same.

Specific to CEF-Based Applications
  • Testing of Silverlight applications in CEF-based applications is not currently supported.

  • .NET applications created with CefGlue are not supported.

  • The windowless (offscreen) rendering mode is supported only for WPF controls of CefSharp applications. The other applications that use offscreeen rendering are not supported.

Flash and Flex Applications

  • Testing Flash applications in Microsoft Edge is currently not supported.

  • Testing Flex applications under 32-bit Windows in Microsoft Edge is currently not supported.

Native Object Properties, Fields and Methods That Are Unavailable to TestLeft

If an application is compiled with debug information, almost every object, property and method (including private ones) are available to TestLeft, and you can view them in the TestLeft UI Spy panel.

However, some methods and properties may remain inaccessible to TestLeft for some reasons:

In All Applications

  • TestLeft cannot access properties and methods for which it cannot find type information. The presence or absence of this information depends on the application type. Some applications, for instance, Java or .NET, include type information about their objects, methods and properties. Other applications (for instance, Delphi or C++Builder) do not contain this information and must be compiled in a special way provide it.

  • TestLeft cannot recognize routines in tested applications if —

    • A routine returns non-object types that are not OLE-compatible (Int64, Extended or ShortString).

      This limitation is not always applied to .NET and Java applications. If a routine in a .NET application returns a value of the Int64 type or includes a parameter of this type, TestLeft can recognize this routine. Similarly, if a routine in a Java application returns a value of the long type or includes a parameter of this type, TestLeft can recognize this routine. The same concerns properties and fields of the Int64 (.NET) and long (Java) type.
    • A routine returns arrays (except for strings), pointers to pointers, untyped pointers, pointers to functions, unions (variant records in Pascal) and bit fields.

      This restriction is not fully applied to .NET and Java applications. If a routine in a .NET or Java application returns an array or uses arrays as parameters, TestLeft can recognize this routine. The same concerns properties and fields that return arrays.

    Also, TestLeft does not recognize constructors, destructors, safecall functions (safecall procedures are available), or functions that return structures (records).

  • TestLeft reads boolean properties and fields as integers: False is 0, True is any other integer value, for example, 1 or -2.

  • Any method removed by the development tool’s Smart Linker since it is inaccessible.

  • TestLeft recognizes properties by detecting their read or write methods. It does not detect a property if the property —

    • Only has abstract virtual read and write methods (directly or by inheritance).

    • Is never read or written to, since its access methods were eliminated by the Smart Linker.

    Although an inherited abstract property remains abstract for TestLeft, and invisible to it, if a child class implements a read or write method for the property, that method becomes visible to TestLeft and can be called directly.

  • Since debug information does not differentiate the Char and Byte types, TestLeft does not distinguish them. Char properties are always considered as Byte type. To set these properties from tests, use numeric values.

  • Methods can use parameters that are pointers to one- or two-byte values, for instance, short int *, wchar_t * or bool * in C++, PSmallInt, PWordBool in Delphi. Debug information can report that these parameters are pointers to strings, so, by default, when calling an object’s method from a test, TestLeft will attempt to convert the specified parameter value to a string and the call will fail.

  • Functions can return values through a reference parameter. Usually, these are functions that fill buffers with some data: from a file, a window, memory, etc. TestLeft does not support reference parameters used to return values from functions. If a function writes a value to a parameter passed by a reference, the value is not copied to the variable passed as a parameter and cannot be retrieved in a test.

In .NET Applications

  • The Decimal data type is supported as Double.

  • TestLeft supports indexed properties by adding the Item property to the object that contains an indexed property.

  • TestLeft does not show property values for objects that are inherited from the System.Windows.Forms.Control class and whose IsHandleCreated property returns False. Instead of property values, TestLeft displays the “Cannot get the value of this property.” string. Nevertheless, you can work with properties of these objects in your tests.

  • Extension methods are not supported.

  • TestLeft cannot get and set values of .NET object fields that hold COM Interop object references.

In Microsoft Visual C++ Applications

Wrapper class limitations:
  • To be visible, variables must be one of the following types:

    • Ordinary OLE-compatible types.

    • CObject class or descendant, and not template-generated.

    • Pointers to ordinal types (e.g. int, char, int *, char *) and pointers to Win32 or MFC structures. Also, MFC classes that are not inherited from CObject.

    Pointers to pointers are excluded.

    To be visible, methods must only use parameters and return values of the above types permitted for variables.

    Note: In addition to these supported types, there are wrappers for structures and classes which are not inherited from CObject, but are used in methods of MFC classes. These methods are also supported.
  • Overloaded routines are not supported. Overloaded routines are methods with the same names that belong to the same class, but have a different set of parameters. Overloaded operators are supported.

Debug Information restrictions:
  • TestLeft supports “ordinary” types: int, char, long, unsigned char, unsigned int, and pointers to ordinary types, structures or classes. Pointers to pointers, unions, pointers to procedures, bit fields and arrays are not supported.

    • The char * type is treated as a pointer to an unsigned char.

    • The int * type is treated as a pointer to an integer.

  • C++ virtual base classes cannot be recognized.

  • When multiple inheritances are present, some fields of base classes may be inaccessible to TestLeft. For example, suppose we have the following class hierarchy:

    C++

    class C1
    { int x;}

    class C2 : public C1
    { int x2;}

    class C3 : public C1
    { int x3;}

    class C4 : public C2, public C3
    { int x4;}

    Class C4 has two x members, one inherited from C2 and the other from C3. In the UI Spy, you can only see one C4::x member and C2::x. Field C3::x is unavailable to TestLeft.

  • An ATL window object must be inherited from the CWindowImplBaseT and CMessageMap classes or from the CAxWindowT class. MFC window objects must be inherited from CWnd, otherwise TestLeft cannot obtain the pointer for these objects.

  • TestLeft cannot use debug information in the DBG format, since it does not supply enough type information for methods and properties.

In Microsoft Visual Basic 6 Applications

Due to certain limitations in Visual Basic 6, TestLeft UI Spy does not show user-defined methods, properties and classes of Visual Basic applications.

In Delphi Applications

  • Delphi objects can have default properties, which are only used when the object is specified alone. When creating tests, do not skip default property names. Debug information does not specify which property is the default, so all properties must be named in tests.

  • For Delphi applications created with CodeGear Delphi 2009 for Win32, TestLeft can expose methods and properties of those objects that have at least one (not inherited) method implemented.

  • TestLeft is not able to access public, protected and private methods and properties of Delphi applications compiled with EurekaLog 7 modules. To avoid the issue, we recommend that you disable EurekaLog when compiling applications aimed for automated testing with TestLeft.

In C++Builder Applications

  • TestLeft supports “ordinary” types:

    • int

    • char

    • long

    • unsigned char

    • unsigned int

    • pointers to ordinary types, structures or classes

    Pointers to pointers, unions, pointers to procedures, bit fields and arrays are not supported.

    • The char * type is always treated as pointers to an unsigned char.

    • The int * type is treated as pointers to an integer.

  • TestLeft does not support applications created with Borland C++Builder v. 3.0.

  • TestLeft does not provide access to public, protected and private properties and methods of VCL controls in 64-bit C++Builder applications.

  • C++Builder objects can have default properties that are only used when the object is specified alone. When creating tests, do not skip default property names. Debug information does not specify which property is the default, so all properties must be named in tests.

  • TestLeft can only expose methods and properties of objects inherited from TObject. C++ virtual base classes are not recognized.

  • Functions with the __msfastcall and __msreturn conventions cannot be called correctly.

  • When multiple inheritances are present, some fields of base classes may be inaccessible to TestLeft. For example, suppose we have the following class hierarchy:

    C++

    class C1
    { int x;}

    class C2 : public C1
    { int x2;}

    class C3 : public C1
    { int x3;}

    class C4 : public C2, public C3
    { int x4;}

    Class C4 has two x members, one inherited from C2 and the other from C3. However, in the UI Spy, you can only see one C4::x member and C2::x. Field C3::x is unavailable to TestLeft.

  • TestLeft is not able to access public, protected and private methods and properties of C++Builder applications compiled with EurekaLog 7 modules. To avoid the issue, we recommend that you disable EurekaLog when compiling applications aimed for automated testing with TestLeft.

In Qt Applications

  • TestLeft has access only to native Qt object properties that meet the following conditions:

    • Defined using the Q_PROPERTY macro,

    • Have standard data types, such as int, double, bool and others, or types handled by QVariant out of the box.

  • Dynamic properties defined using the setProperty method are unavailable to TestLeft.

  • Properties that have custom (user-defined) data types are unavailable.

See Also

Troubleshooting
Possible Issues
Windows Settings

Highlight search results