You can use AlertSite Monitor Runs dashboard to view detailed results of your API monitors and troubleshoot failed API calls. Troubleshooting works best if the monitor has the Capture Level option set – when this is set, AlertSite stores SoapUI logs for failed test steps or all steps.
Start by selecting a monitor and time frame, then find a run of interest. Expand the run to see the test steps performed and identify the steps that failed.
Click the image to enlarge it.
Step details
Click a specific step to see the request URL for that API call and the response status (such as 200 OK or 400 Bad Request). Steps marked </> also have SoapUI step logs that contain full request and response data for that step. These logs are displayed under Step Report in the step details. Here, you can find the exact URL, request headers, parameters, server response, and the results of failed assertions, if any. Which steps have SoapUI logs – all steps or just the failed ones – depends on the monitor’s Capture Level option.
Click the image to enlarge it.
The step log contains these sections:
-
Messages – If the step has assertions configured and some of them failed, the results of failed assertions appear here. Assertion results include expected and actual values.
- Properties – Test step properties that were specified in SoapUI.
-
Request – Request URL and headers.
- Response – Full response (headers and body).
Run logs
If SoapUI step logs are not available, you can review the Run Log or Run Log 2 – logs for the entire test. The second log is available if the first run failed and the monitor has local retry enabled. These logs contain request URLs, response status codes, and assertion results. All error messages are consolidated in the “TestCaseRunner Summary” section of the log for a quick reference.
SoapUI Run Log – Example
SoapUI Run Log – Example
SoapUI Run Log – Example
This log shows that all API calls were successful (HTTP status 200 OK), but the API call for “Update Pet” returned different data than expected.
...
10:21:35,430 INFO [SoapUIProTestCaseRunner] Running TestCase [Petstore Test]
10:21:35,445 INFO [SoapUIProTestCaseRunner] running step [Add Pet]
10:21:35,573 DEBUG [HttpClientSupport$SoapUIHttpClient] Attempt 1 to execute request
10:21:35,574 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Sending request: POST /v2/pet HTTP/1.1
10:21:35,649 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Receiving response: HTTP/1.1 200 OK
10:21:35,661 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Connection shut down
10:21:36,245 INFO [SoapUIProTestCaseRunner] Assertion [Match content of [status]] has status VALID
10:21:36,251 INFO [SoapUIProTestCaseRunner] running step [Update Pet]
10:21:36,416 DEBUG [HttpClientSupport$SoapUIHttpClient] Attempt 1 to execute request
10:21:36,417 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Sending request: PUT /v2/pet HTTP/1.1
10:21:36,490 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Receiving response: HTTP/1.1 200 OK
10:21:36,491 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Connection shut down
10:21:36,496 INFO [SoapUIProTestCaseRunner] Assertion [Match content of [status]] has status FAILED
10:21:36,496 ERROR [SoapUIProTestCaseRunner] ASSERTION FAILED -> Comparison failed for path [$.status], expecting [sold], actual was [available]
10:21:36,496 ERROR [SoapUIProTestCaseRunner] Update Pet failed, exporting to [logs/soapui/74-127854666/logs/try1/Petstore_TestSuite-Petstore_Test-Update_Pet-0-FAILED.txt]
10:21:36,497 INFO [SoapUIProTestCaseRunner] running step [Delete Pet]
10:21:36,581 DEBUG [HttpClientSupport$SoapUIHttpClient] Attempt 1 to execute request
10:21:36,581 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Sending request: DELETE /v2/pet/1459415734823 HTTP/1.1
10:21:36,655 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Receiving response: HTTP/1.1 200 OK
10:21:36,655 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Connection shut down
10:21:36,657 INFO [SoapUIProTestCaseRunner] Assertion [Valid HTTP Status Codes] has status VALID
10:21:36,665 INFO [SoapUIProTestCaseRunner] Finished running TestCase [Petstore Test], time taken: 468ms, status: FAILED
10:21:36,665 INFO [SoapUIProTestCaseRunner] TestCase [Petstore Test] finished with status [FAILED] in 468ms
Ready! API 1.1.0 TestCaseRunner Summary
-----------------------------
Time Taken: 2483ms
Total TestSuites: 0
Total TestCases: 1 (1 failed)
Total TestSteps: 3
Total Request Assertions: 3
Total Failed Assertions: 1
Total Exported Results: 3
10:21:36,666 ERROR [SoapUIProTestCaseRunner] java.lang.Exception: Match content of [status] in [Update Pet] failed;
[Comparison failed for path [$.status], expecting [sold], actual was [available]]
Status: FAILED
Time Taken: 147
Size: 66
Timestamp: Thu Mar 31 10:21:36 UTC 2016
TestStep: Update Pet
Example
The above images show a failed test run with status 6060 SoapUI-specific error. The test has 3 steps: Add Pet, Update Pet, and Delete Pet. The error occurred in the Update Pet step, which makes a PUT request to http://petstore.swagger.io/v2/pet.
SoapUI logs show that this step has an assertion that checks the value of the status field in the API response. The expected value is sold, but the actual value is available. This may indicate an error in the API. Or, if the API has changed recently, the SoapUI project in AlertSite may need to be updated to match the current behavior of the API (see Managing the SoapUI Project).
See Also
Create API Monitor From SoapUI Test
Monitor Runs Dashboard
AlertSite Status Codes
Verifying Monitors: Test on Demand
SoapUI API Monitor Settings