Web API plugin

The Web API plugin allows you to query data from any HTTP API that returns JSON, and then visualize that data. This is particularly useful if there isn't a SquaredUp plugin available for your data source.

For more information about what this plugin does and the data streams it retrieves, see:

Web API

To add a data source click on the + next to Data Sources on the left-hand menu in SquaredUp. Search for the data source and click on it to open the Configure data source page. Adding from data sources page

Before you start

The Web API plugin is a "hybrid" plugin, meaning it is available in SquaredUp as both a cloud and an on-prem plugin.

• Use the cloud plugin if your API is available on the internet. You do not need to configure a relay agent.
• Use the on-prem plugin to access an API on your internal network. You will need to configure a relay agent before you configure the Web API on-prem plugin.
  What is an on-prem data source?

  An on-prem data source uses a relay agent to connect SquaredUp to a data source running on your internal network.

  A relay agent is installed on a server on your internal network, and has access to your data source.

  Using a relay agent means that you don't need to open your firewall to allow access.

Configuring and deploying an agent

For an on-prem plugin you will need a relay agent that can access the server hosting your on-prem data source. You do not need a relay agent for cloud plugins.

If you have already created a relay agent in SquaredUp that can access this data source, then you can skip this step and choose the agent group you want to use while Configuring the data source.

You can install an agent on either Windows or Linux:

• Configuring an agent (Windows platforms)
• Configuring an agent (Linux platforms)

Configuring the data source

For testing you could use the JSON Placeholder API. This is a JSON REST API that will show you some sample data: http://jsonplaceholder.typicode.com

1. Display name:
   Enter a name for your data source. This helps you to identify this data source in the list of your data sources.

   
   For example: JSON Placeholder

2. Agent group:
   Select the Agent Group that contains the agent(s) you want to use. Only for on-prem plugins

3. Base URL:
   Enter the base URL of the API to be used for requests.
   For example you could use http://jsonplaceholder.typicode.com For this sample API the provider needs no further configuration so just click Add and then go to Show data on a tile.
4. Query arguments:
   Optionally, add any parameters that should be added to the Base URL. You may prefer to add these in the tile configuration so you can use different queries on different tiles.
5. Authentication:
   • None: No authentication required.
   • Basic: You must enter a Username and Password.
   • OAuth 2: Token-based authentication according to the OAuth 2.0 standard. Many APIs use OAuth 2.0 for authorization, and will require an OAuth provider to include the additional information about how to authorize against the service. If this option is selected, follow the steps in Configure OAuth 2.0 below.
     Configure OAuth 2.0

     Specify the details for your required OAuth 2.0 flow. For more detailed information see OAuth 2.0 Configuration.

     1. Token URL:
        Enter the URL of the third-party authentication server you are using. For example, https://oauth2.googleapis.com/token.
     2. Client ID:
        Enter the ID provided by your client application.
     3. Client Secret:
        Enter the secret provided by your client application.
     4. Authorization Scope:
        Enter your required access scopes for the third-party application server you are using. For example, if you are using the Web API plugin to access the https://www.googleapis.com/ API and you wish to use the drive/v3/files endpoint, you can use the authorization scope https://www.googleapis.com/auth/drive.readonly.
        If you require more than one scope, you add them all here separated by spaces.
        Offline access

        The Web API plugin requires offline access to be requested, which requires the correct scope to be entered, dependent on the authentication server being used:

        • Atlassian: Request the offline_access scope.
        • Google APIs: Offline access isn't requested via a scope at all. Instead, you must add the access_type = offline and include_granted_scopes = true query arguments to the Authorization URL.

     5. How to send credentials to the Token URL:
        Select where you want the client credentials you entered above to be included in the OAuth 2.0 request.
     6. Grant Type:
        Select the OAuth 2.0 flow you need to use. Choose from:
        • Authorization Code: Enter the Authorization URL of the authorization server you are using (for example, https://www.googleapis.com/) and then click the Sign In button below. You are redirected to the authorization sign in page.
          Upon returning to SquaredUp , if the request was successful, the Sign In button shows you as logged in.
        • Client Credentials: No additional details are required.
        • Password: You must enter a Username and Password.
     7. If you need to send the authorization token in the query URL, select Send authorization with query.
6. Headers can be added if required.

7. Ignore certificate errors:
   If you activate this checkbox the data source will ignore certificate errors when accessing the server. This is useful if you have self-signed certificates.

8. Test endpoint:
   Optionally, you can tick Test endpoint to run a test request and see an example payload.
   Test endpoint

   Enter the details of an endpoint you'd like to run a test against to see what is returned. The information entered here is only used for the test.

   1. Endpoint path to test:
      Enter an endpoint path.
   2. Additional headers for the test:
      Enter any additional header names and values to be used for the test.
   3. HTTP method for the test:
      GET or POST
   4. Query arguments for the test GET:
      If you chose GET you can optionally add any query arguments to be used for the test.
   5. Body for test POST:
      If you chose POST you can optionally enter a JSON string representing the body of the POST request.
   6. Click Send.
   7. The Result box will show the resulting payload.

9. Optional: Restrict access to this data source:
   You can enable this option if you only want certain users or groups to have access to the data source, or the permission to link it to new workspaces. See data source access control for more information.

   Restrict access to this data source

   The term data source here really means data source instance. For example, a user may configure two instances of the AWS data source, one for their development environment and one for production. In that case, each data source instance has its own access control settings.

   By default, Restrict access to this data source is set to off. The data source can be viewed, edited and administered by anyone. If you would like to control who has access to this data source, switch Restrict access to this data source to on.

   Use the Restrict access to this data source dropdown to control who has access to the workspace:

   • By default, the user setting the permissions for the data source will be given Full Control and the Everyone group will be given Link to workspace permissions.
   • Tailor access to the data source, as required, by selecting individual users or user groups from the dropdown and giving them Link to workspace or Full Control permissions.
   • If the user is not available from the dropdown, you are able to invite them to the data source by typing in their email address and then clicking Add. The new user will then receive an email inviting them to create an account on SquaredUp. Once the account has been created, they will gain access to the organization.
   • At least one user or group must be given Full Control.
   • Admin users can edit the configuration, modify the Access Control List (ACL) and delete the data source, regardless of the ACL chosen.

   Access level	Permissions
   Link to workspace	User can link the data source to any workspace they have at least Editor permissions for. Data from the data source can then be viewed by anyone with any access to the workspace. User can share the data source data with anyone they want. User cannot configure the data source in any way, or delete it.
   Full Control	User can change the data source configuration, ACL, and delete the data source.

   See Access control for more information.

10. Click Add.

Next steps

Showing data on a tile

1. On a dashboard click + and select Data to open the tile editor, then configure the following tabs.
2. Data Stream tab:
   1. Select HTTP Request.

      If HTTP Request is not listed, it's possible that the data source was added for the organization but not the workspace. To check, select the Data Source menu > Add new data source and look through the listed data sources.

      If it still does not display, then it may not exist at the organization level, so instead click the link at the bottom of the page to add a new data source.

   2. If you only have one Web API data source configured then the object is automatically selected and you can continue to the Parameters tab. Otherwise, select the Objects tab.
3. Objects tab:
   1. Select the object with the same name that you gave your Data Source. If you are using the JSON Placeholder API for sample data then this might be JSON placeholder.
   2. Select the Parameters tab.
4. Parameters tab:
   The parameters tab is composed of several sub-tabs responsible for configuring each part of the HTTP request. Complete each of them as follows.
   1. Basics:
      Use this sub-tab to configure the request method and endpoint URL of the request. The Base URL you configured in the Data Source setup is automatically populated in the address (unless you are using a Collection with more than one base URL). Depending on your API, you may see data at this point, or you may need to carry out some further configuration.
      1. Method:
         Select GET or POST from the dropdown to the left of the URL.
      2. Enter a path:
         Optionally, enter an endpoint path after the URL. For the JSON Placeholder API you might enter /posts or /todos.
         Supports timeframe mustache parameters

         A mustache parameter is a dynamic value, the actual value will be inserted to replace the field in curly braces. For example, {{timeframe.start}} will insert the start time based on the timeframe configured within the tile, or {{name}} will insert the name of the object(s) in scope.

         This data stream supports timeframe parameters:

         Parameter	Replacement value	Type
         {{timeframe.start}}	2022-03-13T19:45:00.000Z	string
         {{timeframe.unixStart}}	1647200700	number
         {{timeframe.end}}	2022-03-14T19:45:00.000Z	string
         {{timeframe.unixEnd}}	1647287100	number
         {{timeframe.enum}}	last24hours	string
         {{timeframe.interval}}	PT15M	string
         {{timeframe.durationSeconds}}	86400	number
         {{timeframe.durationMinutes}}	1440	number
         {{timeframe.durationHours}}	24	number

         For the JSON placeholder API a suitable visualization is shown at this point, and further configuration is optional.

      3. Click Apply to update the URL and resend the request.
   2. Body:
      If you chose the POST method, optionally enter a JSON string representing the body of the POST request.
      Supports timeframe mustache parameters

      A mustache parameter is a dynamic value, the actual value will be inserted to replace the field in curly braces. For example, {{timeframe.start}} will insert the start time based on the timeframe configured within the tile, or {{name}} will insert the name of the object(s) in scope.

      This data stream supports timeframe parameters:

      Parameter	Replacement value	Type
      {{timeframe.start}}	2022-03-13T19:45:00.000Z	string
      {{timeframe.unixStart}}	1647200700	number
      {{timeframe.end}}	2022-03-14T19:45:00.000Z	string
      {{timeframe.unixEnd}}	1647287100	number
      {{timeframe.enum}}	last24hours	string
      {{timeframe.interval}}	PT15M	string
      {{timeframe.durationSeconds}}	86400	number
      {{timeframe.durationMinutes}}	1440	number
      {{timeframe.durationHours}}	24	number

   3. Parameters:
      Optionally, add any query arguments to be used.
      
      Supports timeframe mustache parameters (in values field)

      A mustache parameter is a dynamic value, the actual value will be inserted to replace the field in curly braces. For example, {{timeframe.start}} will insert the start time based on the timeframe configured within the tile, or {{name}} will insert the name of the object(s) in scope.

      This data stream supports timeframe parameters:

      Parameter	Replacement value	Type
      {{timeframe.start}}	2022-03-13T19:45:00.000Z	string
      {{timeframe.unixStart}}	1647200700	number
      {{timeframe.end}}	2022-03-14T19:45:00.000Z	string
      {{timeframe.unixEnd}}	1647287100	number
      {{timeframe.enum}}	last24hours	string
      {{timeframe.interval}}	PT15M	string
      {{timeframe.durationSeconds}}	86400	number
      {{timeframe.durationMinutes}}	1440	number
      {{timeframe.durationHours}}	24	number

   4. Headers:
      Optionally, click Add new header to enter any additional header names and values to be used. You can choose to encrypt a Header Value by clicking the encrypt icon
      . This turns the field into a password field, so that the value is hidden and the data is stored as encrypted.
      Supports timeframe mustache parameters (in values field)

      A mustache parameter is a dynamic value, the actual value will be inserted to replace the field in curly braces. For example, {{timeframe.start}} will insert the start time based on the timeframe configured within the tile, or {{name}} will insert the name of the object(s) in scope.

      This data stream supports timeframe parameters:

      Parameter	Replacement value	Type
      {{timeframe.start}}	2022-03-13T19:45:00.000Z	string
      {{timeframe.unixStart}}	1647200700	number
      {{timeframe.end}}	2022-03-14T19:45:00.000Z	string
      {{timeframe.unixEnd}}	1647287100	number
      {{timeframe.enum}}	last24hours	string
      {{timeframe.interval}}	PT15M	string
      {{timeframe.durationSeconds}}	86400	number
      {{timeframe.durationMinutes}}	1440	number
      {{timeframe.durationHours}}	24	number

   5. Options:
      Use this tab to configure how to path to the data you want from the URL.
      1. Path to data:
         Enter the location of the results set that is returned. You can click Send on the Response sub-tab for the location of the data returned, and use that in the Path to data.
         

      2. Expand inner objects:
         Objects inside the requested data path will be used to make extra columns of data, for example if you have columns that show [object Object].
         What exactly does Expand inner objects do?

         Let's keep this example payload in mind:

         Copy

         const payload = {
             a1: 'a1',
             a2: 111,
             a3: true,
             arr: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
             obj: {
                 b1: 'b1',
                 b2: 222,
                 b3: true,
                 sub1Arr: [
                     { x1: 'x1', x2: 6661, x3: { y1: 'y11', y2: 'y21' } },
                     { x1: 'x2', x2: 6662, x3: { y1: 'y12', y2: 'y22' } }
                 ],
                 subObj: {
                     c1: 'c1',
                     c2: 333,
                     c3: false
                 }
             }
         };

         If Expand inner objects is unchecked (off)

         If Path to data points to an array, so values like arr, obj.sub1Arr, the data stream is going to return one row for each element (with no limit of 10 involved here).

         arr will return rows with a single column called value with all the integers from 1 to 12 (incl.)

         Copy

         value
            1
            2
         ...
           11
           12

         obj.sub1Arr will return two rows and three columns, like:

         Copy

         x1     x2           x3
         "x1"  6661    "[object Object]"
         "x2"  6662    "[object Object]"

         If Path to data points to a scalar, the data stream will return a single row with a single column called result and the value will be the scalar, for example, obj.b2 will return:

         Copy

         result
         222

         If Path to data points to an object, the data stream will return a single row with columns for each of the properties, for example, obj will return:

         Copy

         b1   b2   b3        sub1Arr           subObj
         "b1" 222 true   "[object Array]"  "[object Object]"

         If Expand inner objects is checked (on)

         Behavior differs wherever one of the [object XXX] placeholders appeared before.

         For example,

         obj.sub1Arr will return two rows and four columns, like:

         Copy

         x1     x2    x3.y1    x3.y2
         "x1"  6661    "y11"    "y21"
         "x2"  6662    "y12"    "y22"

         Also, nested arrays will be expanded (but only to a maximum of ten elements.

   6. Response:
      Click Send to fire off the HTTP request. The returned payload displays in the Response box (unless you are using a Collection with more than one base URL).
   7. Select the Timeframe tab.
5. Timeframe tab:
   Optionally, you can specify a timeframe here, but you may have specified a set timeframe in the query or endpoint path.
6. At this point you might like to configure the visualization, see Visualization Settings.
   For the JSON placeholder sample data you could choose Donut.
   The Shaping and Columns sections can help you configure the visualization as you need.
7. Shaping tab:
   Shaping allows you to perform filtering, grouping and sorting operations on the data you retrieve, although you may also do this using your query.
   
   Filtering

   Data can be filtered according to whether data in a column meets or does not meet specified text or numerical value conditions.

   Multiple filters

   You are able to add multiple filter conditions using the following operators:

   • AND: All conditions must be satisfied (e.g. Status-Equals-Closed ANDType-Equals-Question).
   • OR: Any condition can be satisfied (e.g. Status-Equals-Pending ORStatus-Equals-Closed).

   Available filters

   The following options are available when filtering data, which ones display depends on the column type.

   Option	Description
   Equals	Checks if the value of a field is the same as the specified value. For example,a Status of Active will return results where the status is Active.
   Not equals	Checks if the value of a field is not equal to the specified value. It returns true if the values are different. For example, a status of Active would return results where the category is notActive.
   Contains	Returns data if the specified value exists within the field value. For example, example: URL Contains projects will return results where the URL includes the word "projects" anywhere in the string.
   Doesn't contain	Returns data if the specified value doesn't exist within the field value. For example, example: URL Doesn't contain projects will return results where the URL doesn't include the word "projects" anywhere in the string.
   Less than	Checks if the value of a field is below the specified value. It is used for numerical or date values. For example, IncidentsLess than50 would return results where the number of incidents is below 50.
   Greater than	Checks if the value of a field is over a specified value. It is used for numerical or date values. For example, Incidents Greater than 50 would return results where the number of incidents is over 50.
   Is more than	Available when working with a date / time column. Checks if the value of a field is older than a given time period. You must additionally specify a time quantity and period, and whether to measure ago or from now. For example, Due Is more than 100 days from now will return results where the due date is later than the current day + 100 days. Similarly, Due Is more than 100 days ago will return results where the submitted date is earlier than the current day.
   Within last	Filter records that fall within a specific time range from before the current date and time. You must additionally specify a time quantity and period. For example, Submitted Within last 100 days will return results where the submitted date is between the current day and 100 days ago.
   Within next	Filter records that are within a specified time range after the current date and time. You must additionally specify a time quantity and period. For example, Event date Within next 7 days would return results where the event date is within the next week.
   Is empty	Returns all data without a date value. Useful for identifying records where data is missing.
   Is not empty	Returns all data with a date value.

   Grouping

   Use the grouping section to group and aggregate data columns.

   For example, for AWS cost data you might configure the following settings to display a table or bar chart of cost per label:

   • Group by:label
   • Aggregation type: Total
   • Aggregation column: Amount

   Which columns are available depends on the data stream you chose.

   Configuring grouping enables different visualizations to be displayed, such as bar chart and donut. For example, grouping tickets by channel allows you to show a donut of how many tickets were logged by email vs web form.

   Bucket by

   If you group by a time column, and further grouping is possible, the Bucket by dropdown appears. Use this field to control how the time data is grouped, for example by hour, day, month etc.

   Aggregation

   To aggregate your data, you must select an Aggregation type and a target Aggregation column. For example, if creating bar chart for an Azure Resource Group cost, you could configure the following settings:

   • Group by: Timestamp
   • Bucket by: Day
   • Aggregation type: Total
   • Aggregate column: Cost

   The following lists the available aggregation types.

   Aggregate	Description
   Total	Sums up all numerical values in a dataset, providing the overall total
   Average	Calculates the mean by dividing the total sum by the number of values
   Count	Determines the number of entries in a dataset, including duplicates
   Distinct Count	Counts only unique values, ignoring duplicates
   Max	Identifies the highest value in a dataset
   Median	Finds the middle value when data is sorted in ascending order
   Min	Identifies the lowest value in a dataset
   Mode	Determines the most frequently occurring value in a dataset

   Sorting

   The Sort section allows you to select one or more columns to sort your date by, in either ascending or descending order.

   While this sets the default sort order of data, but you can always click on a column heading to sort the data table on the fly.

   To sort by multiple columns, click Add sort by to add a new row of sort fields to the list. This allows you perform more complex sorts, such as sorting data by the data it was created, then sorting those results alphabetically.

   Enabling the Top toggle allows you to specify the top n rows of data to display.

   
   For a JSON placeholder sample donut you could use: Filter: Id Less than 30
8. Columns tab:
   Column configuration

   SquaredUp automatically defines the metadata retrieved from data streams so the data is assigned the correct data type, however in some circumstances you may want to override this.

   For example, when retrieving data using the Web API plugin, scripting, or custom query data streams (such as Splunk Enterprise plugin), the assigned data type may not be quite correct or as you expect.

   Formatting columns

   Use the following options to format your columns.

   Option	Description
   Name	To rename a column, click the current Name value and enter a new one. Columns that can be renamed display the Rename column icon when hovered over.
   Type	Select an option from the Type dropdown to change the data type of the column. If any additional options are available to configure, the dropdown is expanded below.
   Value	Displays the original value of the column.
   Formatted	Displays the formatted value of the column.
   Add a copy of this column	Click to duplicate a column. The copied field displays Copy of [field name] above its Name.
   Remove this column	Click to delete a cloned column.

   Comparison columns

   Comparison columns are used to compare two values, for example you may want to compare the number of tickets raised this month to the number of tickets raised last month. You can choose to show the value as an absolute change (for example, 12 more tickets) or as a percentage change (for example, a 28% increase).

   When a column has a Type of Number, the Add comparison

   button displays at the end of the row, which you can click to open the Add comparison window.

   From this window, if you have multiple columns with a Type of Number, you can create a comparison column by doing the following:

   1. Column A:
      Select the first column to compare against. Automatically populated with the column of which you clicked Add comparison
      .
   2. Column B:
      Select the second column to compare against.
   3. Output:
      Select how to display the comparison value. This value is displayed in the Preview field. Choose from:
      • Absolute: Show the numerical value of Column A - Column B.
      • Percentage: Show the ratio of Column B to Column A as a percent.
   4. Click Add to create the comparison.

   Additional options

   Some data types have advanced settings that can be configured in the options section, which is displayed whenever you change the data Type or by clicking expand

   next to the column Name.

   Option	Description
   Output Format	Enter a custom format to display date values as a string. Any specified output format is supported. For example, dd/mm/yy, dd/mm/yyyy or d/M/Y. By default, dates and times are displayed in your local timezone to ensure the data makes sense to you.
   Input Format	Enter the format that corresponds to the inputted date. For example, if your data has values such as 05/27/24 01:44 PM, then the Input Format should be set to dd/mm/yy hh:mm aa. Any input format is supported, however if you have a custom input format that is missing any time zone information, the input is always assumed to be UTC. This field is required if the data string for the column are not ISO-8601 formatted. For example, 2024-09-09T13:52:25.281Z.
   Currency	Select the currency to display the value in. This does not convert the currency value.
   Decimal Places	Formats the number of decimal places for a supported data type. Enter a value between 0 and 20.
   Link Text	Specify the text of the URL links in the column.
   Format as duration	Toggle between displaying the time value in minutes and seconds or seconds.
   Map Values to States	You can map the states you're getting back from your data to the states SquaredUp expects. SquaredUp expects the following values for states to be able to show Status Blocks in the correct matching color: success green warning yellow error red unknown gray If your data uses different values for states, you can map them to the values SquaredUp expects. Tip: Any state value that SquaredUp doesn't recognize gets automatically set to unknown. You can usually just leave out the unknown state from your mapping and just specify the other three states.

9. Optionally, configure Monitoring and KPIs.
10. Click Save when you have finished.

Advanced users

{
	"name": "scope example",
	"matches": "all",
	"dataSourceConfig": {
		"endpointPath": "post",
		"httpMethod": "post",
		"idSeparator": "], [",
		"headers": [
			{
			"key": "header1",
			"value": "header1Val"
			},
			{
			"key": "sourceIds",
			"value": "[{{sourceIds}}]"
			}
		],
		"expandInnerObjects": true
	},
	"rowPath": []
}

//If you want to specify only one value of an object property//
"matches": {
	"ObjectProperty": {
		"type": "equals",
		"value": "ValueOfTheObjectProperty"
	}
},
 
//If you want to specify multiple values for an object property//
"matches": {
	"ObjectProperty": {
 		"type": "oneOf",
		"values": ["ValueOfTheObjectProperty1", "ValueOfTheObjectProperty2", "ValueOfTheObjectProperty3"]
	}
},

"matches": {
    "sourceName": {
      "type": "equals",
      "value": "AppDynamics"
    },
    "type": {
      "type": "equals",
      "value": "app"
    }
  },

"matches": [
    {
        "sourceName": {
            "type": "equals",
            "value": "AppDynamics"
        }
    },
    {
        "type": {
            "type": "equals",
            "value": "app"
        }
    }
]

{
    "generalInfo": "some info",
    "results": [
        {
            "name": "object 1",
            "tags": [
                "tag 1",
                "tag 2",
                "tag 3"
            ]
        },
        {
            "name": "object 2",
            "tags": [
                "tag 1",
                "tag 4"
            ]
        }
    ]
}

Was this article helpful?