Splunk Integration

• Splunk integration is available on AlertSite Enterprise (Usage-Based Monitoring) plans. You must be an AlertSite Admin, Co-Admin, or Power User to configure the integration.

• If you use self-hosted Splunk:

  • Your Splunk server must be accessible from the Internet.

  • If Splunk is behind a firewall, the firewall must allow traffic on the Splunk management port (default is 8089). AlertSite locations send data to Splunk through this port.

• Private nodes (Private Node Server and InSite) support Splunk alerts starting from version 2.0.

AlertSite needs a user account to send data to Splunk. The account you can use depends on your Splunk edition.

• If you use Splunk Cloud (paid subscription), contact Splunk Support and ask for the REST API credentials. This is the account you will need to specify in AlertSite.

• If you use Splunk Light Free, it supports a single administrator account and does not support additional user accounts. You will need to specify the admin account in AlertSite.

• In other Splunk editions, you will need a user account with the edit_tcp capability. By default, only Splunk administrators have this capability, and regular users do not. For security reasons you may want to create a separate user with just the edit_tcp capability, so that it can only be used to post data to Splunk.

  Create a user via Splunk web interface

  Create a user via Splunk web interfaceCreate a user via Splunk web interface

  1. Log into Splunk as administrator.

  2. Go to Settings > Access controls > Users.

  3. Click New.

  4. Enter the user name (for example alertsite) and password.

  5. Clear all Selected roles and select Create a role for this user.

     

     Click the image to enlarge it.

  6. Click Save.

  7. Go to Settings > Access controls > Roles.

  8. Click the role named user-<username> (for example, user-alertsite).

  9. Under Capabilities, add edit_tcp to the selected capabilities.

     

  10. Click Save.

  11. Go to Settings > Access controls > Authentication method and click Reload authentication configuration.

  Create a user via Splunk configuration files

  Create a user via Splunk configuration filesCreate a user via Splunk configuration files

  In self-hosted Splunk, you can add a user as follows:

  1. Edit the authorize.conf file in $SPLUNK_HOME/etc/system/local/ and add this section:

     [role_user-alertsite]
     edit_tcp = enabled

  2. Add the alertsite user:

     splunk add user alertsite -password passw0rd -role user-alertsite -auth admin:changeme

     Note:

     Passwords with special characters that would be interpreted by the shell (for example, '$' or '!') must be either escaped or single-quoted.

  3. Reload authentication configuration to apply the changes:

     splunk reload auth

You need to create a new source type in Splunk for AlertSite data so Splunk can properly parse AlertSite alerts and extract timestamps. You can add source types in Splunk web interface, or by editing the props.conf file.

Add source type in Splunk web interface

Add source type in Splunk web interfaceAdd source type in Splunk web interface

1. Log into Splunk as administrator.

2. Go to the Source types configuration screen (for example, Settings > Source types in Splunk Enterprise, or menu button > Data > Source types in Splunk Light).

3. Click New Source Type.

4. Specify the following settings:

   

   Click the image to enlarge it.

   Setting	Value
   Name	alertsite
   Indexed Extractions	json
   Timestamp > Extraction	Advanced
   Time zone	Select the same time zone as specified in your AlertSite settings in  > Settings > Preferences > Time Settings. For example, if your AlertSite time zone is "GMT-05 Eastern Time (USA)", select the GMT-5 time zone in Splunk too.
   Timestamp format	%Y-%m-%d %H:%M:%S
   Timestamp fields	details.timestamp,details.tested at,details.fixed at
   Advanced	Click New setting and add these settings: KV_MODE = none AUTO_KV_JSON = false

5. Click Save.

6. Restart Splunk to complete the configuration.

Add source type via props.conf file

Add source type via props.conf fileAdd source type via props.conf file

In self-hosted Splunk, you can edit the props.conf file in $SPLUNK_HOME/etc/system/local/ and add the following section:

[alertsite]
INDEXED_EXTRACTIONS = json
KV_MODE = none
AUTO_KV_JSON = false
TIMESTAMP_FIELDS = details.timestamp,details.tested at,details.fixed at
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TZ = America/New_York

Set TZ to the same time zone as specified in your AlertSite settings in  > Settings > Preferences > Time Settings. For possible TZ values, see the tz database.

Restart Splunk to reload the props.conf file:

splunk restart

1. Log in to AlertSite UXM.

2. From the top menu, select  > Manage Integrations.

3. Click in the integrations list.

4. In the dialog that opens, click New Recipient.

5. Enter your Splunk server information.

   

   Click the image to enlarge it.

   Setting	Description
   Name	An optional display name for your Splunk server.
   Splunk URL	Your Splunk server URL, including the prefix (https:// or http://) and the Splunk management port (default is 8089). For example: https://splunk.myserver.com:8089 Note: The management port differs from the web port used to access Splunk in web browsers. See Network ports in Splunk documentation.
   User and Password	A user account on your Splunk server that will be used to post data to Splunk. This can be an admin account or any account with the edit_tcp capability (see above). LDAP and SAML accounts are not supported.

6. Click next to the created recipient.

7. In the Availability Alerts column, select how many consecutive alerts to send to Splunk, and whether to send a “clear” notification. For a description of available settings, see Recipient Properties.

   

8. To receive performance alerts, enable them in the Performance Alerts column. Select which alert types to receive and whether to repeat successive performance alerts.

   

9. Click Save.

That’s it! Splunk will now receive alerts from your AlertSite monitors.

Tips:

• To receive alerts from specific monitors rather than all monitors, you can create a recipient group containing Splunk and the needed monitors.

• If you use several Splunk servers, you can add them as individual recipients.

• Splunk recipients also appear in your global recipient list in Alerts > Alert Recipients, and you can manage them from there.

You can view and analyze AlertSite monitor alerts using Splunk’s search engine. For a description of fields included in alerts, see Alert Data Fields. You can find sample alerts below.

Default Splunk fields are set to:

• sourcetype – alertsite.
• source – the monitor ID, or www for test alerts.

• host – the IP address of the monitoring location that sent the error. You can see our IP addresses here.

  Note:

  host is not necessarily the same as the location that found the error (details.location). For example, in the Global Verify mode, alerts are always sent from the primary location even if the error was found at another location.

Sample Queries

Sample Alerts