Perfecto Mobile Monitoring with AlertSite

Last modified on May 14, 2021

Perfecto Mobile Cloud offers automated testing on real mobile devices, connected via Wi-Fi or cellular carriers to the Internet, spread in different locations around the world. AlertSite integrates with Perfecto Mobile Cloud, allowing you to schedule and monitor test runs on real devices. This way you can continuously monitor the key transactions on mobile devices and get proactively notified of any availability or performance issues.

How AlertSite and Perfecto Mobile Work Together

Perfecto Mobile provides automated testing capabilities and real devices for monitoring, and AlertSite provides scheduling, alerting, and reporting. The following image shows the integration flow between the two systems.

AlertSite and Perfecto integration

Click the image to enlarge it.

  1. User uploads an app and creates tests in Perfecto MobileCloud.

  2. User schedules tests in AlertSite.

  3. Tests run on real devices around the world in the Perfecto cloud.

  4. Test results are sent to AlertSite.

  5. AlertSite sends alerts if a test fails. User can also view test reports online in AlertSite.

Purchasing Mobile Monitoring Plan

Perfecto Mobile monitoring is an optional extra service to AlertSite Enterprise customers. Please contact your AlertSite Sales Manager for details. To view the contact information, select > AlertSite 1.0 from the top menu, then go to Support > Support Center.

Configuring Perfecto

Configure Monitoring Devices

Perfecto MobileCloud provides multiple copies of the same device model for redundancy. You use some devices for script development and debugging, and others for AlertSite monitoring. For example, if your Perfecto cloud includes three iPhone 5 devices, you use two of them for monitoring, and another one for scripting.

Monitoring devices must be paired, with each pair including two identical devices with the same model, OS version, network carrier, and geographical location. A pair of devices is used as a logical monitoring device, and they appear as a single device in AlertSite. A monitoring script runs automatically on both paired devices, one after another (in random order based on device availability). This improves fault tolerance: if a script fails on one device but succeeds on another, the run is reported as successful. Only errors detected on both devices during the same run may indicate an actual problem.

To designate devices for monitoring, you need to pick pairs of twin devices and specify a unique description for each device pair.

  1. In Perfecto MobileCloud, select More > System Administration from the top menu.

  2. Switch to the Devices tab.

  3. Find two devices with the same model and OS version to use as a logical monitoring device.

    Perfecto device pairs

    Click the image to enlarge it.

  4. Select one device and click Properties on the toolbar.

  5. Enter the description for the device in the Description edit box. Use something that reflects the device’s business function or characteristics, such as model, carrier network, or location. For example, Amazon-Monitor-iPhone5, Android-ATT-Boston, and so on.

    Perfecto Mobile device description

    Click the image to enlarge it.

  6. Click OK.

  7. Add the same description for another paired device.

Repeat the procedure to specify descriptions for all pairs of your monitoring devices.

Create Monitoring Scripts

You create and maintain the monitoring scripts in the Perfecto MobileCloud IDE. Monitoring scripts usually contain these elements:

  • Device preparation and cleanup – application restart, returning the device to the Home screen, and so on.

  • User actions – button presses, link clicks, text input, and so on.

  • Transactions – operations whose availability and performance you want to measure, like the specific screens in your application. Transaction results will be sent to AlertSite. The next section discusses transactions in more details.

If you are targeting multiple platforms and devices, follow the Perfecto ScriptOnce methodology to create a device-agnostic script.

Save your monitoring scripts inside the Public or My (private) folder. Do not use group folders for monitoring scripts (group folders are named after the user group you are member of).

Define Transactions to Monitor

Perfecto monitoring scripts must include transactions to mark the actual things you want to monitor. A transaction is a group of commands with a transaction ID assigned. It succeeds if all of its commands succeed, and fails if at least one command fails.

Usually a transaction is a text checkpoint that measures your application’s response time and status (OK/error) in reaction to the preceding user action. This way you can monitor the application launch time, or the load time of specific screens in your application. To learn more about configuring Perfecto scripts to measure application responsiveness, see Measuring User Experience Timing in Perfecto documentation.

You can have up to 5 transactions in a Perfecto script.

For example, the following script includes two transactions, Site Load and What's new. Each transaction is a web page checkpoint that waits for a page to load, and verifies that is loaded successfully. The transaction results sent to AlertSite are the page load success/failure and the page load time.

Transactions in a Perfecto script

To define a transaction:

  1. Open your script in the Perfecto IDE.

  2. To create a new group, select a command or several consecutive commands, then click the  Group icon on the toolbar.

  3. Double-click a group to open its properties.

  4. Change the group name to something more descriptive. For example, Home page, Log in, Book a hotel, and so on. This name will appear in AlertSite reports.

  5. Select the Transaction ID, from trans1 to trans5. The IDs define the order in which the script’s transactions will appear in AlertSite reports. It is best to assign the IDs sequentially from top to bottom of your script.

    Perfecto transaction properties
  6. Click Close to save the changes.

Repeat the procedure to define other transactions, up to 5 in total.

Note: You can have as many groups as you wish, but only those with the transaction ID assigned will be reported to AlertSite.

Configure Error Policies

Commands in Perfecto scripts have error policies that specify how to handle failed commands, for example, whether to continue or stop the script. If later transactions in your script depend on earlier transactions, you may want to stop the script on error. In this case, use the Abort, Break, or Continue policy (see below). If the transactions are independent, you can use Ignore to ignore the errors and continue the script.

To specify the error policy for a command:

  1. Open the script in the Perfecto IDE.

  2. Select a command in the script. You can select several commands using SHIFT-clicks.

  3. Click Edit on the toolbar.

  4. Switch to the On error tab.

  5. Select the error policy and click OK.

    Perfecto error policy

The error policies work as follows:

  • Ignore (default) – Continue the script. The transaction is marked failed.

  • Abort – Stop the script. The transaction is marked failed.

  • Break – Outside of a loop, works as Abort. When used inside a loop, exits the loop and continues from the next command following the loop. The transaction is marked as failed.

  • Continue – Outside of a loop, works as Abort. When used inside a loop, jumps to the next loop iteration. The transaction is marked as failed.

  • Catch – Use this in commands directly before a condition. Any error is ignored, and does not affect the transaction status.

For more information, see Error Policy in Perfecto documentation.

Using AlertSite

Create AlertSite Monitors From Perfecto Scripts

  1. On the AlertSite Dashboard, hover over + Add New and select Monitor.

  2. Select Mobile and click Real Device Monitor.

  3. Specify your Perfecto account details and basic monitor information:

    • Enter the monitor name.

    • Select your mobile monitoring plan.

    • Fill in the Hostname (for example,, User ID, and Password fields.

      Perfecto monitor configuration

      Click the image to enlarge it.

    • Run Interval and Monitor Offset – The monitoring schedule: how often to run the script and which minute of an hour to start the run. For example, with the 60-minute interval and 10-minute offset, the script runs every hour at 8:10, 9:10, 10:10, and so on. With the 20-minute interval and 5-minute offset, the script runs every hour at 5, 25, and 45 minutes past hour. For 120-minute monitors, offsets 1-60 mean even hours (2 PM, 4 PM, and so on), offsets 61-120 – odd hours (3 PM, 5 PM, and so on)

      Devices in Perfecto MobileCloud can run only one script at a time. Configure the schedule to spread the monitoring over an hour between the various devices you have. Make sure that no two monitors try to access the same device at the same time.
    • Collection/Notification Mode – If the script will run on multiple devices, choose whether to run on all devices at the same time, or on one device at a time, rotating through the selected devices.

  4. Click Start Monitoring Now.

    Note: This completes basic monitor configuration, but the monitor does not actually run yet.
  5. Click Add/Edit Script. Here you can select the script to run, and the mobile devices for monitoring.

    Selecting a Perfecto script and devices

    Click the image to enlarge it.

  6. Click Add Script and select the script to run.

  7. Click Add a Device, select one or more devices to run this script, and click Continue.

  8. To enable monitoring, select the Enabled check box on the Main tab of the monitor settings.

  9. Click Save to save the monitor configuration.

AlertSite will now run the selected script in your Perfecto MobileCloud according to the schedule and collect the results.

Set Up Alerts

AlertSite can send automated alerts when your mobile transactions fail, or the response time exceeds the configured limit. You need to enable alerts in the monitor settings and configure the alert recipients.

To enable alerts for a monitor:

  1. Open the monitor settings. To do this, click on the monitor tile on the Dashboard and select Edit Configurations.

  2. Click Monitor Alerts.

  3. Select which alert types the monitor will send:

    Perfecto monitor alerts

    Click the image to enlarge it.

    Availability alerts – Sent on an error condition.

    Performance alerts – Sent when the response time exceeds a threshold. You can set thresholds for the entire script or individual transactions.

    To configure the threshold:

    • Set Measurement Alert Type to Response Time.

    • Select the threshold type – Static (a fixed value) or Dynamic (compared to the historical performance).

    • Select the time period for analysis. Use a larger period to look for trends rather than single events.

    • Enter the error and warning thresholds.

    • Select the number of devices that should be in error or warning to trigger the alert.

      Performance alert settings

      Click the image to enlarge it.

    To learn more about configuring performance alerts, see Performance Alerts.

  4. Click in the top right to save the changes.

Now, configure recipients for alerts:

  1. From the top menu, select Alerts > Alert Recipients.

  2. To create a new recipient, click New Recipient and specify the contact information.

  3. Select the recipient in the list and click Edit Recipient.

  4. To receive alerts about errors, switch to the Availability Alerts tab and enable these alerts.

    For email recipients, also select Attach server response to e-mail alerts. This will include Perfecto execution reports in email alerts.
    Recipient configuration for availability alerts

    Click the image to enlarge it.

  5. To receive response time alerts, switch to the Performance Alerts tab and enable these alerts.

  6. Configure other recipient settings as needed.

  7. Click Submit.

Note: If you have many recipients in AlertSite, but you want to send Perfecto monitor alerts only to specific recipients, you need to create a recipient group for the monitor and its alert recipients. For instructions, see Recipient Groups.

View Monitoring Status in AlertSite

You can see the status of your monitoring scripts on the AlertSite dashboard. The response time displayed is the total UX time of all transactions in the script. If a monitor discovers an error, it turns red and is moved to the top of the dashboard.

Perfecto monitor on the AlertSite dashboard

Click the image to enlarge it.

To view detailed results, you can either click the charts on the tiles or manually navigate the following dashboards:

  • Monitors > Availability – Shows the monitor status (OK/error) over time, as a total and per individual mobile devices.

  • Monitors > Performance – Shows the response time over time, as an average or per individual mobile devices.

  • Monitors > Runs – Shows all monitor runs and results as a list.

See Common Status Codes for status code descriptions and troubleshooting tips.

View Perfecto Execution Reports

You can receive Perfecto reports attached to email alerts when your monitoring script completes with an error. Email recipients should have the Attach server response to e-mail alerts option selected to use this feature (see above).

You can also access Perfecto reports from AlertSite. First, go to Monitors > Runs, select your monitor and choose a time period at the top. You will see all the test results for that period. You can filter the runs by status (OK/error) or sort them by execution time to find the needed results. Expand the runs to see the results of individual transactions.

Perfecto run results in AlertSite

Click the image to enlarge it.

In the expanded run, click  View Perfecto Report to view the detailed execution results with device screenshots and videos.

Perfecto report

Click the image to enlarge it.

Failed runs with status 5300 may not have an associated Perfecto report. In this case, click Request Log on the Monitor Runs dashboard to view the AlertSite error log. The first error in the log indicates the reason why the test failed. See Common Status Codes below for some typical errors.

Error log for a Perfecto monitor

Click the image to enlarge it.

AlertSite Reports

AlertSite provides a set of reports that you can download, print, or email.

To create a cumulative report:

  • Go to Reports > Performance Reports.

  • Select the report type, the monitor, and the report period.

  • Click Run Report.

Once the report is created you can view it, download, email, or schedule it as a recurring email report. You can also access all preciously created reports in Reports > Document Manager.

To view AlertSite report for a specific test run, do any of the following:

  • In Monitors > Runs, expand a run and click  Detail Report.

  • In Monitors > Summary > Availability History, click a time range, then click a specific run. In the tooltip, click View Detail Report.

This opens the Detail report for that run, with the status and response time of all transactions in your Perfecto script.

Reports contain test results as a table. You can expand a test to see the transactions performed in that test, their status and response time. Click the camera icon next to a test to view the Perfecto report. If the Perfecto report is not available, click to view the AlertSite error log.

Detail report for a Perfecto monitor

Click the image to enlarge it.

Common Status Codes

This table contains some common error codes used by Perfecto monitors in AlertSite. For other status codes, please see AlertSite Status Codes.

Status Description Possible Causes and Solutions
0 Site is OK now The test completed successfully.
2 Test timed out Status 2 is a failed time checkpoint. This usually means your application is busy and experiencing a slowdown. If the response time looks normal, review the timers in your script and increase the timeouts.
5 Validation failed Status 5 is a failed checkpoint, meaning the expected text or image was not found in your application. You can see the checkpoint condition and results in the Perfecto execution report. If everything looks OK, review the checkpoint conditions in the script.
95 Unable to monitor Unable to run the script in Perfecto MobileCloud.
  • The Perfecto host name, user ID or password is incorrect
  • The specified script is not found (for example, it has been moved or deleted)
  • The specified device is not found (for example, it is no longer available for monitoring)
  • A temporary out-of-resource state. This condition should clear up shortly, but if it persists, please contact SmartBear Support
Note: Status 95 “Unable to monitor” appears only on the main Dashboard, in Monitor Summary and Monitor Availability. No new entries are added to the monitor run history, because this status usually indicates a temporary condition.
96 Invalid XML Unable to fetch script results from Perfecto MobileCloud. Please report this error to SmartBear Support.
5090 Processing exception Status 5090 is a legacy status code that was used prior to October 23, 2015. It means no device was available to run the script. You may need to adjust the monitoring schedule (run interval and monitor offset) to avoid overlaps with other monitors.
5300 Service execution error

There was a problem running your Perfecto script. To view the error message, go to Monitors > Runs, expand the failed run and click Request Log. Below are some common causes and solutions.


Handset unavailable, or Cannot open device <device ID> for user <>, device is in use

No devices are available to run the script. To avoid the problem, you can:

  • Change the monitor run interval and offset to avoid overlaps with your other Perfecto monitors.

  • Make sure the monitoring devices are not used for scripting, debugging, or any other purposes other than production monitoring.

  • Allocate more devices for monitoring.


Transaction results missing from Perfecto

The script does not have any transactions defined. Transactions mark the operations to monitor, so no transactions means no performance data sent to AlertSite. To learn how to configure transactions in your Perfecto script, see Define Transactions to Monitor.


<Command name> сommand failed. Reason: handset server: <error message>

A failed script command means a problem with the script or the monitored application. You can review Perfecto report to identify the failed command and troubleshoot the script in the Perfecto IDE to fix the problem.


Empty variable or parameter: <variable name>

Review the script in the Perfecto IDE and make sure all command parameters are specified.

If your script includes variables, make sure all variables except DUT (device under test) have default values and are marked as Runtime. The default values will be used when the script is run from AlertSite.


Response time is 0 seconds in AlertSite

This happens if your Perfecto script does not contain any transactions – groups with the transaction ID assigned. AlertSite monitors transactions only rather than the entire script. To learn how to configure transactions, see Define Transactions to Monitor.

Some Perfecto scripts do not show up in AlertSite

AlertSite can monitor only public and private scripts from your Perfecto repository, but not group scripts (scripts shared between members of a group). Please save your monitoring scripts to the Public or My (private) folder or subfolders.

Error when creating a Perfecto monitor: "Your primary home location does not support Mobile Monitoring"

This happens if the default primary location specified in your AlertSite account preferences is other than Fort Lauderdale or Atlanta. To avoid the issue, you need to temporarily change your default primary location:

  1. From the top menu, select  > Settings.

  2. Switch to the Public Locations tab.

  3. Select Fort Lauderdale or Atlanta as the Primary location.

  4. Click Save.

Then, create your Perfecto monitor.

After that, you can revert to your previous location settings.

What’s Next

  • Scheduling Monitor Blackouts – Define maintenance windows to prevent script runs during device maintenance periods
  • Alerts – learn more about the AlertSite notification system
  • Reporting – Schedule daily, weekly, or monthly reports
  • SLA Monitoring – define your availability and performance requirements, and monitor SLA compliance

See Also

Perfecto Monitor Settings
Perfecto Mobile Monitoring

Highlight search results