Perfecto Mobile Monitoring with AlertSite

Perfecto MobileCloud provides multiple copies of the same device model for redundancy. You use some devices for script development and debugging, and others for AlertSite monitoring. For example, if your Perfecto cloud includes three iPhone 5 devices, you use two of them for monitoring, and another one for scripting.

Monitoring devices must be paired, with each pair including two identical devices with the same model, OS version, network carrier, and geographical location. A pair of devices is used as a logical monitoring device, and they appear as a single device in AlertSite. A monitoring script runs automatically on both paired devices, one after another (in random order based on device availability). This improves fault tolerance: if a script fails on one device but succeeds on another, the run is reported as successful. Only errors detected on both devices during the same run may indicate an actual problem.

To designate devices for monitoring, you need to pick pairs of twin devices and specify a unique description for each device pair.

1. In Perfecto MobileCloud, select More > System Administration from the top menu.

2. Switch to the Devices tab.

3. Find two devices with the same model and OS version to use as a logical monitoring device.

   

   Click the image to enlarge it.

4. Select one device and click Properties on the toolbar.

5. Enter the description for the device in the Description edit box. Use something that reflects the device’s business function or characteristics, such as model, carrier network, or location. For example, Amazon-Monitor-iPhone5, Android-ATT-Boston, and so on.

   

   Click the image to enlarge it.

6. Click OK.

7. Add the same description for another paired device.

Repeat the procedure to specify descriptions for all pairs of your monitoring devices.

You create and maintain the monitoring scripts in the Perfecto MobileCloud IDE. Monitoring scripts usually contain these elements:

• Device preparation and cleanup – application restart, returning the device to the Home screen, and so on.

• User actions – button presses, link clicks, text input, and so on.

• Transactions – operations whose availability and performance you want to measure, like the specific screens in your application. Transaction results will be sent to AlertSite. The next section discusses transactions in more details.

If you are targeting multiple platforms and devices, follow the Perfecto ScriptOnce methodology to create a device-agnostic script.

Save your monitoring scripts inside the Public or My (private) folder. Do not use group folders for monitoring scripts (group folders are named after the user group you are member of).

Perfecto monitoring scripts must include transactions to mark the actual things you want to monitor. A transaction is a group of commands with a transaction ID assigned. It succeeds if all of its commands succeed, and fails if at least one command fails.

Usually a transaction is a text checkpoint that measures your application’s response time and status (OK/error) in reaction to the preceding user action. This way you can monitor the application launch time, or the load time of specific screens in your application. To learn more about configuring Perfecto scripts to measure application responsiveness, see Measuring User Experience Timing in Perfecto documentation.

You can have up to 5 transactions in a Perfecto script.

For example, the following script includes two transactions, Site Load and What's new. Each transaction is a web page checkpoint that waits for a page to load, and verifies that is loaded successfully. The transaction results sent to AlertSite are the page load success/failure and the page load time.

To define a transaction:

1. Open your script in the Perfecto IDE.

2. To create a new group, select a command or several consecutive commands, then click the  Group icon on the toolbar.

3. Double-click a group to open its properties.

4. Change the group name to something more descriptive. For example, Home page, Log in, Book a hotel, and so on. This name will appear in AlertSite reports.

5. Select the Transaction ID, from trans1 to trans5. The IDs define the order in which the script’s transactions will appear in AlertSite reports. It is best to assign the IDs sequentially from top to bottom of your script.

   

6. Click Close to save the changes.

Repeat the procedure to define other transactions, up to 5 in total.

Commands in Perfecto scripts have error policies that specify how to handle failed commands, for example, whether to continue or stop the script. If later transactions in your script depend on earlier transactions, you may want to stop the script on error. In this case, use the Abort, Break, or Continue policy (see below). If the transactions are independent, you can use Ignore to ignore the errors and continue the script.

To specify the error policy for a command:

1. Open the script in the Perfecto IDE.

2. Select a command in the script. You can select several commands using SHIFT-clicks.

3. Click Edit on the toolbar.

4. Switch to the On error tab.

5. Select the error policy and click OK.

   

The error policies work as follows:

• Ignore (default) – Continue the script. The transaction is marked failed.

• Abort – Stop the script. The transaction is marked failed.

• Break – Outside of a loop, works as Abort. When used inside a loop, exits the loop and continues from the next command following the loop. The transaction is marked as failed.

• Continue – Outside of a loop, works as Abort. When used inside a loop, jumps to the next loop iteration. The transaction is marked as failed.

• Catch – Use this in commands directly before a condition. Any error is ignored, and does not affect the transaction status.

For more information, see Error Policies in Perfecto documentation.

1. On the AlertSite Dashboard, hover over + Add New and select Monitor.

2. Select Mobile and click Real Device Monitor.

3. Specify your Perfecto account details and basic monitor information:

   • Enter the monitor name.

   • Select your mobile monitoring plan.

   • Fill in the Hostname (for example, mylab.perfectomobile.com), User ID, and Password fields.

     

     Click the image to enlarge it.

   • Run Interval and Monitor Offset – The monitoring schedule: how often to run the script and which minute of an hour to start the run. For example, with the 60-minute interval and 10-minute offset, the script runs every hour at 8:10, 9:10, 10:10, and so on. With the 20-minute interval and 5-minute offset, the script runs every hour at 5, 25, and 45 minutes past hour. For 120-minute monitors, offsets 1-60 mean even hours (2 PM, 4 PM, and so on), offsets 61-120 – odd hours (3 PM, 5 PM, and so on)

     Devices in Perfecto MobileCloud can run only one script at a time. Configure the schedule to spread the monitoring over an hour between the various devices you have. Make sure that no two monitors try to access the same device at the same time.

   • Collection/Notification Mode – If the script will run on multiple devices, choose whether to run on all devices at the same time, or on one device at a time, rotating through the selected devices.

4. Click Start Monitoring Now.

   Note:

   This completes basic monitor configuration, but the monitor does not actually run yet.

5. Click Add/Edit Script. Here you can select the script to run, and the mobile devices for monitoring.

   

   Click the image to enlarge it.

6. Click Add Script and select the script to run.

7. Click Add a Device, select one or more devices to run this script, and click Continue.

8. To enable monitoring, select the Enabled check box on the Main tab of the monitor settings.

9. Click Save to save the monitor configuration.

AlertSite will now run the selected script in your Perfecto MobileCloud according to the schedule and collect the results.

AlertSite can send automated alerts when your mobile transactions fail, or the response time exceeds the configured limit. You need to enable alerts in the monitor settings and configure the alert recipients.

To enable alerts for a monitor:

1. Open the monitor settings. To do this, click on the monitor tile on the Dashboard and select Edit Configurations.

2. Click Monitor Alerts.

3. Select which alert types the monitor will send:

   

   Click the image to enlarge it.

   Availability alerts – Sent on an error condition.

   Performance alerts – Sent when the response time exceeds a threshold. You can set thresholds for the entire script or individual transactions.

   To configure the threshold:

   • Set Measurement Alert Type to Response Time.

   • Select the threshold type – Static (a fixed value) or Dynamic (compared to the historical performance).

   • Select the time period for analysis. Use a larger period to look for trends rather than single events.

   • Enter the error and warning thresholds.

   • Select the number of devices that should be in error or warning to trigger the alert.

     

     Click the image to enlarge it.

   To learn more about configuring performance alerts, see Performance Alerts.

4. Click in the top right to save the changes.

Now, configure recipients for alerts:

1. From the top menu, select Alerts > Alert Recipients.

2. To create a new recipient, click New Recipient and specify the contact information.

3. Select the recipient in the list and click Edit Recipient.

4. To receive alerts about errors, switch to the Availability Alerts tab and enable these alerts.

   For email recipients, also select Attach server response to e-mail alerts. This will include Perfecto execution reports in email alerts.

   

   Click the image to enlarge it.

5. To receive response time alerts, switch to the Performance Alerts tab and enable these alerts.

6. Configure other recipient settings as needed.

7. Click Submit.

You can see the status of your monitoring scripts on the AlertSite dashboard. The response time displayed is the total UX time of all transactions in the script. If a monitor discovers an error, it turns red and is moved to the top of the dashboard.

Click the image to enlarge it.

To view detailed results, you can either click the charts on the tiles or manually navigate the following dashboards:

• Monitors > Availability – Shows the monitor status (OK/error) over time, as a total and per individual mobile devices.

• Monitors > Performance – Shows the response time over time, as an average or per individual mobile devices.

• Monitors > Runs – Shows all monitor runs and results as a list.

See Common Status Codes for status code descriptions and troubleshooting tips.

You can receive Perfecto reports attached to email alerts when your monitoring script completes with an error. Email recipients should have the Attach server response to e-mail alerts option selected to use this feature (see above).

You can also access Perfecto reports from AlertSite. First, go to Monitors > Runs, select your monitor and choose a time period at the top. You will see all the test results for that period. You can filter the runs by status (OK/error) or sort them by execution time to find the needed results. Expand the runs to see the results of individual transactions.

Click the image to enlarge it.

In the expanded run, click  View Perfecto Report to view the detailed execution results with device screenshots and videos.

Click the image to enlarge it.

Failed runs with status 5300 may not have an associated Perfecto report. In this case, click Request Log on the Monitor Runs dashboard to view the AlertSite error log. The first error in the log indicates the reason why the test failed. See Common Status Codes below for some typical errors.

Click the image to enlarge it.

AlertSite provides a set of reports that you can download, print, or email.

To create a cumulative report:

• Go to Reports > Performance Reports.

• Select the report type, the monitor, and the report period.

• Click Run Report.

Once the report is created you can view it, download, email, or schedule it as a recurring email report. You can also access all preciously created reports in Reports > Document Manager.

To view AlertSite report for a specific test run, do any of the following:

• In Monitors > Runs, expand a run and click  Detail Report.

• In Monitors > Summary > Availability History, click a time range, then click a specific run. In the tooltip, click View Detail Report.

This opens the Detail report for that run, with the status and response time of all transactions in your Perfecto script.

Reports contain test results as a table. You can expand a test to see the transactions performed in that test, their status and response time. Click the camera icon next to a test to view the Perfecto report. If the Perfecto report is not available, click to view the AlertSite error log.

Click the image to enlarge it.

This happens if your Perfecto script does not contain any transactions – groups with the transaction ID assigned. AlertSite monitors transactions only rather than the entire script. To learn how to configure transactions, see Define Transactions to Monitor.

AlertSite can monitor only public and private scripts from your Perfecto repository, but not group scripts (scripts shared between members of a group). Please save your monitoring scripts to the Public or My (private) folder or subfolders.

This happens if the default primary location specified in your AlertSite account preferences is other than Fort Lauderdale or Atlanta. To avoid the issue, you need to temporarily change your default primary location:

1. From the top menu, select > Settings.

2. Switch to the Public Locations tab.

3. Select Fort Lauderdale or Atlanta as the Primary location.

4. Click Save.

Then, create your Perfecto monitor.

After that, you can revert to your previous location settings.