Google Cloud Platform (GCP) plugin

Click the following link for additional content such as blogs, videos, use cases, and more:

Google Cloud Platform

Monitor your GCP environment, including GKE, Hosts and more.

Adding a data source

To add a data source, open the relevant workspace and then click on the + next to Data Sources on the left-hand menu in SquaredUp. Search for the data source and click on it to open the Add data source page.

Before you start

  1. Enable the APIs
  2. Configure the GCP Service Account
  3. Configure the data source

Enable the APIs

In the GCP console select your project and check that the following are enabled:

To view cost information in SquaredUp using the Cost data stream check the following:

Configure the GCP Service Account

  1. Create a new Service Account, or edit an existing account.
    See GCP - Creating and managing service accounts
  2. Ensure the account has the role Viewer by adding the role Basic > Viewer.
    To view cost information, the account also needs to have BigQuery Data Viewer role.
    See GCP - Grant a single role
  3. Create a new key for the Service Account using the key type JSON. Download the JSON file as you will need to copy information from this JSON file when adding the data source next.
    See GCP - Creating service account keys
    Important

    Make sure to store the key file securely, because it can be used to authenticate as your service account.

Configuring the data source

  1. Open the JSON file that you downloaded when creating the key.
  2. Copy and paste the clientEmail from the JSON file into the data source form.
  3. Copy and paste the private_key from the JSON file into the data source form (everything between the quotes).
  4. Copy and paste the project_ids from the JSON file into the data source form.
  5. Optionally, to use the Cost data stream, copy and paste in the billingProjectId, billingDataSetName and billingTableName.

  6. Restrict access to this data source:
    Optionally, enable this toggle if you only want certain users/groups to have access to the data source, or those with the permission to link it to new workspaces. See data source access control for more information.

    Note

    The term data source here really means data source instance. For example, a user may configure two instances of the AWS data source, one for their development environment and one for production. In that case, each data source instance has its own access control settings.

    By default, Restrict access to this data source is set to off. The data source can be viewed, edited and administered by anyone. If you would like to control who has access to this data source, switch Restrict access to this data source to on.

    Use the Restrict access to this data source dropdown to control who has access to the workspace:

    • By default, the user setting the permissions for the data source will be given Full Control and the Everyone group will be given Link to workspace permissions.
    • Tailor access to the data source, as required, by selecting individual users or user groups from the dropdown and giving them Link to workspace or Full Control permissions.
    • If the user is not available from the dropdown, you are able to invite them to the data source by typing in their email address and then clicking Add. The new user will then receive an email inviting them to create an account on SquaredUp. Once the account has been created, they will gain access to the organization.
    • At least one user or group must be given Full Control.
    • Admin users can edit the configuration, modify the Access Control List (ACL) and delete the data source, regardless of the ACL chosen.

    Access level
    Permissions
    Link to workspace
    • User can link the data source to any workspace they have at least Editor permissions for.
    • Data from the data source can then be viewed by anyone with any access to the workspace.
    • User can share the data source data with anyone they want.
    • User cannot configure the data source in any way, or delete it.
    Full Control
    • User can change the data source configuration, ACL, and delete the data source.

    See Access control for more information.

  7. Click Test and add to validate the data source configuration. SquaredUp will now attempt to connect using the provided authentication method.

    • Testing passed – a success message will be displayed and then the configuration will be saved.
    • Testing passed with warnings – warnings will be listed and potential fixes suggested. You can still use the data source with warnings. Select Save with warnings if you believe that you can still use the data source as required with the warnings listed. Alternatively, address the issues listed and then select Rerun tests to validate the data source configuration again. If the validation now passes, click Save.
    • Testing failed – errors will be listed and potential fixes suggested. You cannot use the data source with errors. You are able to select Save with errors if you believe that a system outside of SquaredUp is causing the error that you need to fix. Alternatively, address the issues listed and then select Rerun tests to validate the data source configuration again. If the validation now passes, click Save.

    You can edit data source configurations at any time from Settings > Data Sources.

Testing and troubleshooting

If you encounter an error, refer to the guidance or contact our support team in-app or via SquaredUp Support

For errors on dashboard tiles see Troubleshooting tiles.

Error
What to do
Unable to communicate with GCP API. Check client email.
Check the Client email is correct. Copy the email address without the quotes from the client_email field of the downloaded JSON file.
Unable to communicate with GCP API. Check private key.
Check the Private key is correct. Copy everything between the quotes from the private_key field in the downloaded JSON file.
Unable to retrieve information about project(s): <project IDs>.
Check that the project IDs are correct and that the service account has permission to view those projects.
Unable to retrieve VMs for project(s): <project IDs>.
This is a warning. Check that the Compute Engine API is enabled and that the service account can view VM instances in those projects.
Unable to retrieve GKE clusters for project(s): <project IDs>
This is a warning. Check that the Kubernetes Engine API is enabled and that the service account can view clusters in those projects.
Unable to retrieve node pools for cluster(s): <cluster names>
This is a warning. Check that the service account can view node pools for those clusters.
Unable to retrieve cloud billing account information. Check client email and private key.
This is a warning. It appears when billing details are configured. Check that the service account can read the billing project, dataset and table.

Next steps

Data streams

You can use these data streams to create new tiles to show data, or if there are preconfigured dashboards installed you can copy or edit those.

Data streams are named queries targeted at a data source, typically mapped to a specific API endpoint. They define how SquaredUp retrieves live data. By standardizing information from diverse formats into a simple table, data streams provide a consistent starting point regardless of the original source system.

Each plugin includes its own set of data streams. When editing a tile or exploring data, the data stream you choose acts as the entry point to the objects and records within that data source.

Scoped data streams allow you to specify which objects you want to pull data for. Global data streams return general information that is not tied to objects.

Some data streams are configurable, meaning you can configure additional settings on the Parameters tab of the tile editor to create a bespoke query.

When you're creating a tile, you can tweak data streams by grouping or aggregating specific columns.

See Data Streams for more information.

The following data streams are installed with this plugin.

Region Estimates

Estimated annual sustainability metrics per cloud region, trended from provider-disclosed data

Parameters
Region Metadata

Provider-disclosed annual sustainability metrics per cloud region, one row per region per year

Parameters

Tips for using the GCP Cost data stream

The Cost data stream is part of the GCP data source and uses the GCP BigQuery API. The GCP data source needs some additional configuration to enable the Cost data stream see How to add a Google Cloud Platform data source

  • The GCP Cost data stream shows daily data.
  • By default the Cost data stream splits costs by service.
  • We recommend scopes used with the GCP Cost data stream contain only one type of object, for example, only accounts, projects or hosts. Mixed scopes may include duplicated data in the results, because a query is sent for each type of object in the scope, and the results are combined. For example, resulting data would show the cost of a host, as well as the cost of a project, even if the host is part of that project. It is useful to create a scope for your account(s), a scope of project(s) and a scope for hosts.

Use Cases for the GCP Cost data stream

Create a new tile on a dashboard with the following settings:

Data stream:

Cost

Objects:

Account(s) or Project(s)

Visualization:

Line Graph is good for showing data for dashboard timeframes of 7 or 30 days. A line will be shown for each service.

Note

The GCP Cost data stream shows daily data, so timeframes of less than 7 days will not display a line.

The dashboard timeframe setting, located on the toolbar at the top of a dashboard, lets you easily toggle the time span that a dashboard displays data for.

Selecting a new timeframe immediately updates the data for every tile on the dashboard that is configured to inherit that dashboard's timeframe.

When the dashboard timeframe covers more than one day the first column of a table shows the date. To only show each service once for the selected dashboard timeframe see Use Case: How much does each GCP service cost for a specific timeframe?.

Create a new tile on a dashboard with the following settings:

Data stream:

Cost

Objects:

Account(s) or Project(s) or Hosts

Shaping:

The following settings will show how much each service costs you in total for the account(s) or project(s) in scope for the whole timeframe selected:

Group by
Service (For each service...)
Aggregate column
Amount (...show me the combined amount...)
Aggregation type
Total (... in total.)

Visualization:

Table, Donut or Bar Chart. Each bar or segment will show a service.

After you're done creating the tile, you can specify the timeframe by changing the dashboard timeframe to 7 or 30 days.

The dashboard timeframe setting, located on the toolbar at the top of a dashboard, lets you easily toggle the time span that a dashboard displays data for.

Selecting a new timeframe immediately updates the data for every tile on the dashboard that is configured to inherit that dashboard's timeframe.

Create a new tile on a dashboard with the following settings:

Data stream:

Cost

Objects:

Hosts

Shaping:

The following settings will show how much each service costs you in total for the account(s) or project(s) in scope for the whole timeframe selected:

Group by
Object (For each host...)
Aggregate column
Amount (...show me the combined amount...)
Aggregation type
Total (... in total.)

Visualization:

Table, Donut or Bar Chart. If, for example, you've scoped to Hosts, then each bar will be the cost for one host.

Note

When you choose to 'Group by' Object this means it will group by whatever you have scoped to. In this case it will group by Host, but if you've scoped to Project(s) it will group by projects and each segment or bar would show a project.

After you're done creating the tile, you can specify the timeframe by changing the dashboard timeframe to 7 or 30 days.

The dashboard timeframe setting, located on the toolbar at the top of a dashboard, lets you easily toggle the time span that a dashboard displays data for.

Selecting a new timeframe immediately updates the data for every tile on the dashboard that is configured to inherit that dashboard's timeframe.

Create a new tile on a dashboard with the following settings:

Data stream:

Cost

Objects:

Account(s)

Shaping:

The following settings will show how much all the services cost you in total for the accounts in scope for the whole timeframe selected:

Group by
Object (For the account(s) in scope...)
Aggregate column
Amount (...show me the combined amount...)
Aggregation type
Total (... in total.)
Note

When you choose to 'Group by' Object this means it will group by whatever you have scoped to. In this case it will group by Account, but if you've scoped to Project(s) it will group by projects.

Visualization:

If you only have one account in the scope you can choose Scalar, as you only want to see one number (total cost for all services).

If you have several accounts in the scope you can choose Table, to see the cost per account.

After you're done creating the tile, you can specify the timeframe by changing the dashboard timeframe to 7 or 30 days.

The dashboard timeframe setting, located on the toolbar at the top of a dashboard, lets you easily toggle the time span that a dashboard displays data for.

Selecting a new timeframe immediately updates the data for every tile on the dashboard that is configured to inherit that dashboard's timeframe.

Was this article helpful?

Have more questions or facing an issue?
Submit a ticket