TestEngine Version History

• SLM authentication migrated to use Auth0 for improved security.

• Improved the shutdown behavior so that the call to the unlock endpoint of the SLM server is sent during product shutdown, ensuring the license is correctly unlocked.

• Created a function to perform and parse CURL and CLI requests from the command line to TestEngine.

• After updating httpclient, ReadyAPI changed the way of building request URLs. TestEngine was modified to support this new behavior, ensuring duplicate '/' symbols are not merged.

• Set the disableSniHostCheck flag to true by default to avoid issues with HTTPS communication.

• Fixed an error related to unmatched closing Regex Patterns when running test cases in RAPI 3.48. TestEngine was updated to support this fix.

• When the "Use Access for Everyone" checkbox is selected, the access key input window will be cleared and not used to install the license.

• TestEngine is updated to support ReadyAPI 3.53.

TestEngine 1.31.0 includes newer versions of previously used libraries.

• We resolved an issue where an incorrect license was presented in TestEngine when the SLM SaaS license was installed in ReadyAPI. Additionally, addressed the inability to uninstall this license. The license was showing as a "File-based license."

• The command-line interface (CLI) and console now feature updated error messages in case the test project fails to refresh the access token.

• TestEngine now provides compatibility for running gRPC tests that were created and updated in ReadyAPI.

• We added a checkbox to simplify the license installation process when working within private networks.

• TestEngine no longer removes users; instead, it deactivates them. We have redesigned the user interface to indicate that users are deactivated.

• In the test job result report, there is an additional "Success Ratio" column with the format <passed-count>/<total-count> next to the status.

• We added a new column in both the overview and history tables, showing "Run Time," specifically dedicated to displaying the "Test Run Time." This addition aims to provide users with a clear distinction between the run time while a test suite is being executed and the total time required to complete the test suite.

• We provided installers for Mac X64 and ARM.

• TestEngine CLI 1.9.0 is released.

TestEngine 1.30.0 includes newer versions of previously used libraries.

Important Updates for customers

• When installing and starting TestEngine, there's a conflict with the Guice framework that prevents TestEngine from starting if a user has ReadyAPI installed and running on the same machine while being logged in. To resolve this, log out from ReadyAPI, start TestEngine, and then log back into ReadyAPI to install your license.

• 1.29.1 introduced an unintended issue that will be fixed in a future release. When installing an ID-Based License through the on-premise workflow with an SLM license server configured for Access for Everyone, there is a validation setting in the Access Key field. It requires a character entry to activate the Install button.

• From October 2023, we won't provide file-based licensing. Instead, we will replace it with SmartBear ID-based licensing - With TestEngine 1.29.0, we extend support for SmartBear ID-based licenses hosted on SmartBear licensing servers. You can now specify your Access Key in the license installation process and request a license, once is it made available and assigned on manage.smartbear.com. As we approach the October milestone, we're actively encouraging customers to transition over. Now is the time to start planning that transition. You should immediately engage with your SmartBear Account Manager to understand the steps involved in transitioning and getting started. Please read here to learn more.

• We released TestEngine CLI in Version 1.8.0.

• In 1.29.1, we fixed the problem with test steps in ReadyAPI projects that required a Pro license. Now they work as expected in TestEngine for SLM licenses.

• From October 2023, we won't provide file-based licensing. Instead, we will replace it with SmartBear ID-based licensing - SmartBear ID-based licensing (also known as SmartBear License Management or SLM), is a new license management option that we started using in ReadyAPI 3.8.1. Support for SmartBear ID-based licenses is now being added to TestEngine 1.28. It is more reliable and convenient than the license type used in the previous product versions. As we approach the October milestone, we're actively encouraging customers to transition over. Now is the time to start planning that transition. You should immediately engage with your SmartBear Account Manager to understand the steps involved in transitioning and getting started. Please read here to learn more.

  Note:

  Support for SmartBear ID-based licenses is starting with offline (Customer on-premise) deployment in Q2 2023 (where the customer hosts the license server on-premise). We will add support for online (SmartBear hosts the license servers) later in 2023.

• We upgraded to Java 17.0.6.

• We updated a large list of libraries and vulnerabilities.

• We updated CLI to 1.7.0 (adding support to manage ID-based licenses).

• We bumped OpenSSL3 to latest version in Docker in response to CVE-2022-3602, CVE-2022-3786.

• Please Note - Windows, Linux and Platform Independent versions remain at 1.27.0 as these are not impacted.

• We upgraded to Java 17.0.4.

• TestEngine now uses ReadyAPI 3.41 to execute tests. At this time, it does not support gRPC tests. This will be added in the future release.

• We updated commons-text to 1.10 in response to CVE-2022-42889.

• Upgraded spring libraries to 5.3.18.

• Excluded log4j 1.2.17 that was present in the docker image.

• Upgraded log4j2 to version 2.17.1.

• TestEngine now uses ReadyAPI 3.10.1 to execute tests.

• Upgraded log4j2 to version 2.15.0 and added flag -Dlog4j2.formatMsgNoLookups=true to mitigate vulnerability CVE-2021-44228.

• TestEngine now uses ReadyAPI 3.9.2 to execute tests. This includes support for Smart Assertions and other new ReadyAPI features.

• Support for API recipe versions is added.

• Web reports now available in the UI.

• Testengine's version is now visible in the user menu.

• Diagnostics report can now be downloaded by an admin from the user menu.

• Now only testsuites/cases that are going to be run are validated.

• Support for the vmoptions file is now added to the .sh and .bat script in addition to the generated launcher.

• Hid recipes behind a feature-flag. This is start of a revamp of recipes in testengine. Any customer that uses recipes should specify -Dtestengine.preview.recipes=true in vmoptions to enable them.

• TestEngine now uses ReadyAPI 3.8.0 to execute tests.

• Docker version support graceful shutdown and will return floating license on shutdown.

• Docker version can now be run as non root users.

• Bug fix: Sometimes the with the testjob provided client certificate couldn't be read on Windows.

• API call added to get a server diagnostics report.

• Groovy scripts are now allowed get information about proxy.

API Definition v.1.10.0 at SwaggerHub

• Bug fix: The values of parameters clientCertFileName and clientCertPassword were lost when a job was queued.

• TestEngine is stopped and a message is printed if the database is locked after a failed schema upgrade.

• TestEngine now uses ReadyAPI 3.6.0 to execute tests.

• Added actions to the "Recently Completed", similar to "Run History".

• Moved User Management, License Management and Audit Log to the profile dropdown.

• Bug fixes:

  1. A grammar fix in the wording for the jdbc security setting.

  2. NPE fix when running securityTest with custom scripts.

  3. Testsuite/testcase are not validated when they are disabled.

• Moved User Management, License Management and Audit Log to the profile dropdown.

• Only the user running TestEngine has access to the temporary project files.

• Bug fix: Canceling a queued testjob is quicker now. No reports are generated and the UI shows a disabled download-button for such jobs.

• Run history page is added.

• Replaced with Recently completed jobs table in Overview page.

• Added Running and in queued table in Overview page.

• Admin tab is renamed to Server.

• Bug fix: It should be possible to add allowed file paths making it possible to write to the root of the file-system.

• Better validation for empty and faulty projects.

• Added functionality to delete a Test Job on TestEngine UI.

• Added global search bar on all the tables.

• It is also possible to search for job with testjobId in testjobs table.

• Column filters are always visible in testjobs table.

• If a test step use property expansion, the file path validation is not done when the project is submitted but instead during runtime. This to solve errors with wrongly failing validations.

• GET /testjobs/{testjobId}/transactions now returns http status 204 (no data) instead of 500 (internal server error) if no transactions are found.

API Definition v.1.8.0 at SwaggerHub

• JDBC connections in test steps are now not allowed until an admin user enabled them in the settings file, via the settings endpoint or in the user interface. This will make all (new and already present) tests using JDBC fail unless this setting is changed.

• References to the SoapUI brand name have been removed.

• TestEngine now uses Ready API version 3.4.0.

• This version of TestEngine require Protection License Server 5.1.5 or later.

API Definition v.1.4.0 at SwaggerHub

• Improved error messages for file access check.

• Added field passedAssertions to json result report.

• Fixed bug where Properties test step would complain about blank paths.

API Definition v.1.5.0 at SwaggerHub

• TestEngine can now reconnect to the floating license server if the license server gets restarted.

• Fixed bug where the environment query parameter would be lost if a project gets queued.

• TestEngine now allows the user to change the priority for testjobs.

• The TestEngine UI now shows priority jobs as part of the job table.

• Security fixes.

API Definition v.1.4.0 at SwaggerHub

• Security fixes.

API Definition v.1.3.0 at SwaggerHub

• Support for submitting prioritized jobs.

• Tags can now be used for both test cases and test suites.

• Bump ReadyAPI dependency to 3.2.5.

API Definition v.1.3.0 at SwaggerHub

• Support for running Security Tests.

API Definition v.1.2.0 at SwaggerHub

• The bundled JRE for the Linux installer has been updated.

API Definition v.1.1.0 at SwaggerHub

• Back end:

  • Update the ReadyAPI dependency to 3.1.0.

  • Update the Java version to 11. If the zip distribution is used, a new Java version needs to be set up (Java 11 or later).

  • Stop using "testengine" as the default password. Instead, enter the password in the installer or as startup parameters. If no password is provided, a random password will be generated and printed in the console during the first startup. Old installations are not affected by this when upgrading.

  • The security fix of WebSocket tokens.

API Definition v.1.1.0 at SwaggerHub

• Front end:

  • Minor UI fixes.

• Back end:

  • It is possible to get a token and use it as a bearer token instead of using basic authorization.

API Definition v.1.1.0 at SwaggerHub

• Front end:

  • Minor UI fixes.

• Back end:

  • It is possible to install TestEngine as a Windows service.

API Definition v.1.0.4 at SwaggerHub

• Front end:

  • Minor UI fixes.

API Definition v.1.0.4 at SwaggerHub

• Front end:

  • Minor UI fixes.

• Back end:

  • Reports are in the PDF format.

  • Tuning of groovy script security.

  • The possibility of using a full URL in a transaction log.

API Definition v.1.0.4 at SwaggerHub

• Front end:

  • The third-party libraries have been upgraded to be up to date.

  • Minor UI fixes.

API Definition v.1.0.3 at SwaggerHub

• Back end:

  • The third-party libraries have been upgraded to be up to date.

• Front end:

  • Part of third-party libraries has been upgraded to be up to date.

  • Minor UI fixes.

API Definition v.1.0.3 at SwaggerHub

• Back end:

  • It is possible to specify a timeout for a job.

• Front end:

  • A better tooltip in the test jobs graph.

  • Redirecting when opening the application with no license present has been improved.

API Definition v.1.0.3 at SwaggerHub

• Back end:

  • Using ReadyAPI 2.8.2 internally.

  • A floating license can be installed even if the server is not listed in the Allowed connection targets.

• Front end:

  • The license section has moved to a separate tab.

  • Expose settings for database cleanup on the settings page.

API Definition v.1.0.3 at SwaggerHub

• Back end:

  • Automatic cleanup of test jobs.

  • Negative performance timing has been fixed.

• Front end:

  • Update the Settings UI.

  • Get a test job ID from the UI.

API Definition v.1.0.2 at SwaggerHub

• Back end:

  • Using ReadyAPI 2.8.0 internally.

  • The bug when installing a floating license after a trial one has been fixed.

  • In JUnit reports, canceled steps have been marked as failed.

  • The TestEngine version is now shown on startup and in /api/v1/version.

  • The API to clean up test results has been improved.

• Front end:

  • Save Server Settings on the Enter key.

  • Redirect to log in even from the 404 page when logged out.

API Definition v.1.0.1 at SwaggerHub

The initial version.

API Definition v.1.0.0 at SwaggerHub