Elasticsearch plugin

Monitor any metrics from your Elasticsearch environment using custom QueryDSL, Lucene queries and API calls.

For more information about what this plugin does and the data streams it retrieves, see:

To add a data source click on the + next to Data Sources on the left-hand menu in SquaredUp. Search for the data source and click on it to open the Configure data source page.

You can also add a data source by clicking Add data source on the Settings > Data Sources page, but pre-built dashboards are not added when using this method.

Before you start

The Elasticsearch plugin is a "hybrid" plugin, meaning it is available in SquaredUp as both a cloud and an on-prem plugin.

Use the cloud plugin if your Elasticsearch instance is available on the internet. You do not need to configure a relay agent.
Use the on-prem plugin to access a Elasticsearch instance on a server on-prem. You will need to configure a relay agent before you configure the Elasticsearch on-prem plugin.
An on-prem data source uses a relay agent to connect SquaredUp to a data source running on your internal network.
A relay agent is installed on a server on your internal network, and has access to your data source.
Using a relay agent means that you don't need to open your firewall to allow access.

Configuring and deploying an agent

For an on-prem plugin you will need a relay agent that can access the server hosting your on-prem data source. You do not need a relay agent for cloud plugins.

If you have already created a relay agent in SquaredUp that can access this data source, then you can skip this step and choose the agent group you want to use while Configuring the data source.

See one of the following, depending on your platform type:

Configuring the data source

Display Name:
Enter a name for your data source. This helps you to identify this data source in the list of your data sources.
Agent Group:
Select the Agent Group that contains the agent(s) you want to use.
This field will only appear if you are adding the on-prem plugin.
Instance URL:
Enter the URL for your Elasticsearch environment.
Where to find the Instance URL:
The URL depends on your Elasticsearch environment:
1. If you are using the Elastic cloud service, you need to enter the endpoint URL, which has the following format:
  https://CLUSTER_ID.REGION.CLOUD_PLATFORM.DOMAIN:PORT
2. If you are running Elasticsearch in a different environment, ask your Elastic administrator for the instance URL and port number.
Enter the username and password for your Elasticsearch environment.
Ignore Certificate errors:
If you activate this checkbox the data source will ignore certificate errors when accessing the server. This is useful if you have self-signed certificates.

Restrict access to this data source:
You can enable this option if you only want certain users or groups to have access to the data source, or the permission to link it to new workspaces. See data source access control for more information.

The term data source here really means data source instance. For example, a user may configure two instances of the AWS data source, one for their development environment and one for production. In that case, each data source instance has its own access control settings.

By default, Restrict access to this data source is set to off. The data source can be viewed, edited and administered by anyone. If you would like to control who has access to this data source, switch Restrict access to this data source to on.

Use the Restrict access to this data source dropdown to control who has access to the workspace:

By default, the user setting the permissions for the data source will be given Full Control and the Everyone group will be given Link to workspace permissions.
Tailor access to the data source, as required, by selecting individual users or user groups from the dropdown and giving them Link to workspace or Full Control permissions.
If the user is not available from the dropdown, you are able to invite them to the data source by typing in their email address and then clicking Add. The new user will then receive an email inviting them to create an account on SquaredUp. Once the account has been created, they will gain access to the organization.
At least one user or group must be given Full Control.
Admin users can edit the configuration, modify the Access Control List (ACL) and delete the data source, regardless of the ACL chosen.

Access level	Permissions
Link to workspace	User can link the data source to any workspace they have at least Editor permissions for. Data from the data source can then be viewed by anyone with any access to the workspace. User can share the data source data with anyone they want. User cannot configure the data source in any way, or delete it.
Full Control	User can change the data source configuration, ACL, and delete the data source.

See Access control for more information.

Click Add.

Next steps

Data streams

Data streams standardize data from all the different shapes and formats your tools use into a straightforward tabular format.

While creating a tile you can tweak data streams by grouping or aggregating specific columns.

Depending on the kind of data, SquaredUp will automatically suggest how to visualize the result, for example as a table or line graph.

Data streams can be either global or scoped:

Global data streams are unscoped and return information of a general nature (e.g. "Get the current number of unused hosts").
A scoped data stream gets information relevant to the specific set objects supplied in the tile scope (e.g. "Get the current session count for these hosts").

See Data Streams for more information.

The following data streams are installed with this plugin.

Data stream	Description
Indices	Retrieve a list of your Elasticsearch indexes.

Configurable data streams

The following data streams have configurable Parameters.

Data stream	Description	Parameters
Query DSL	Use a query Domain Specific Language based on JSON to define queries	Query Body:Optionally, enter a JSON query. For example: `{ "query": { "query_string": { "query": "managedEntityDisplayName:\"Microsoft Windows Server 2016 Datacenter\"" } } }` This data stream supplies scoped objects individually for mustache parameters. When there are multiple objects in scope this data source will send the query multiple times, once for each object. The results are then displayed together, for example in a single table. You can use properties of objects and write them in between curly braces e.g `{{name}}` to use them as mustache parameters. Whenever you use mustache parameters, you need to use a scope of objects that contain the property you're referencing. For example, if objects of type "host" have a property called `name`, you can use `{{name}}`. This will resolve `{{name}}` to the value of the name property of the different "host" objects used in the scope. Query Parameters: Optionally, specify a set of JSON parameters for your Query Body. for example: `{ "q": "managedEntityDisplayName:\"Microsoft Windows Server 2016 Datacenter\"" }` This data stream supplies scoped objects individually for mustache parameters. When there are multiple objects in scope this data source will send the query multiple times, once for each object. The results are then displayed together, for example in a single table. You can use properties of objects and write them in between curly braces e.g `{{name}}` to use them as mustache parameters. Whenever you use mustache parameters, you need to use a scope of objects that contain the property you're referencing. For example, if objects of type "host" have a property called `name`, you can use `{{name}}`. This will resolve `{{name}}` to the value of the name property of the different "host" objects used in the scope.
Lucene Query	Use a Lucene query to fetch data.	Lucene Query: Enter a Lucene Query, for example: `managedEntityDisplayName:"Microsoft Windows Server 2016 Datacenter"`
Custom API Call	Build a custom API call to fetch data.	HTTP Method Select the method to use for the API call. The next three fields, Elasticsearch API Endpoint, JSON Headers and JSON Parameters, are all optional, but you must enter details for at least one. Elasticsearch API Endpoint: Specify an Elasticsearch API Endpoint to send the call to, for example: `{{name}}/_search` This data stream supplies scoped objects individually for mustache parameters. When there are multiple objects in scope this data source will send the query multiple times, once for each object. The results are then displayed together, for example in a single table. You can use properties of objects and write them in between curly braces e.g `{{name}}` to use them as mustache parameters. Whenever you use mustache parameters, you need to use a scope of objects that contain the property you're referencing. For example, if objects of type "host" have a property called `name`, you can use `{{name}}`. This will resolve `{{name}}` to the value of the name property of the different "host" objects used in the scope. JSON Headers: Optionally, enter a set of JSON Headers for the API call. JSON Parameters: Optionally, enter a set of JSON Parameters for the API call. This data stream supplies scoped objects individually for mustache parameters. When there are multiple objects in scope this data source will send the query multiple times, once for each object. The results are then displayed together, for example in a single table. You can use properties of objects and write them in between curly braces e.g `{{name}}` to use them as mustache parameters. Whenever you use mustache parameters, you need to use a scope of objects that contain the property you're referencing. For example, if objects of type "host" have a property called `name`, you can use `{{name}}`. This will resolve `{{name}}` to the value of the name property of the different "host" objects used in the scope.

Was this article helpful?

Have more questions or facing an issue?

Submit a ticket