Azure DevOps plugin

Monitor the Builds and Releases from your Azure DevOps Server or Azure DevOps Services (Cloud) environment.

Click the following link for additional content such as blogs, videos, use cases, and more:

Azure DevOps

Hybrid plugin

The Azure DevOps plugin is a "hybrid" plugin, meaning it is available in SquaredUp as both a cloud and an on-prem plugin. Which plugin you use depends on whether your Azure DevOps instance is publicly accessible, rather then whether you're using Azure DevOps Server or Azure DevOps Services (also known as, Azure DevOps Cloud).

  • Use the cloud plugin (without the on-prem label) if your Azure DevOps instance is publicly accessible. You do not need to configure a relay agent.
  • Use the on-prem plugin to access a Azure DevOps instance that is behind a company network. The on-prem plugin uses a relay agent, meaning you don't need to open up your firewall.

    An on-prem data source uses a relay agent to connect SquaredUp to a data source running on your internal network.

    A relay agent is installed on a server on your internal network, and has access to your data source.

    Using a relay agent means that you don't need to open your firewall to allow access.

Adding a data source

To add a data source click on the + next to Data Sources on the left-hand menu in SquaredUp. Search for the data source and click on it to open the Configure data source page.

Before you start

If you are adding a plugin marked as On-Prem, you will need a relay agent that can access the server hosting your on-prem data source. You do not need a relay agent for cloud plugins.

If you have already created a relay agent in SquaredUp that can access this data source, then you can skip this step and choose the agent group you want to use while Configuring the data source.

You can install an agent on either Windows or Linux:

If you are using the Microsoft-hosted Azure DevOps Services (Cloud) you will have a choice of either Microsoft Entra ID or a Personal Access Token (PAT).

If you are using an on-prem Azure DevOps Server instance you will only have the option of a Personal Access Token (PAT).

  • Personal Access Token (PAT) (available for Azure DevOps Cloud and Azure DevOps Server): This method is recommended for production environments, as it allows you to create a token with granular permissions.
  • Microsoft Entra ID (available for Azure DevOps Cloud only): This method allows the data source instance to access Azure DevOps using a user account. It provides less granular control over permissions, which is not ideal from a "least privilege" perspective, but can be useful when experimenting with the data source in non-production environments.

    For this feature to work, you must ensure that your Azure DevOps organization is connected on the Microsoft Entra tab of the Azure DevOps Settings page - https://dev.azure.com/{Your_Organization}/_settings/organizationAad. 

If you are using Microsoft Entra ID instead of a PAT, you can skip the section below and proceed to the Configuring the Data Source section.

Generating a Personal Access Token

Follow the steps below to generate a Personal Access Token (PAT). For additional information, please refer to the Azure DevOps Documentation.

  1. Sign in to your Azure DevOps instance.
  2. From your home page, open user settings and select Personal access tokens or Security.
  3. Select + New Token.
  4. Name your token, select the organization where you want to use the token, and then set your token to automatically expire after a set number of days.
  5. Select the scopes for this token to authorize. You can select Show all scopes to view all scopes.

    Permission
    Usage
    Projects and Teams (Mandatory)
    Import:
    • Projects
    Agent Pools
    Import:
    • Pools
    • Queues
    • Agents.

    Analytics
    Used to make an organization wide analytics query.
    Build
    Import:
    • Build Pipelines
    • Build Folder
    Code
    Import:
    • Repos
    • Repo folder
    • Branch
    Deployment Groups
    Import:
    • Deployment Groups
    Environments
    Import:
    • Environments
    Packaging
    Import:
    • Artifact Package
    • Artifact Feed
    Release
    Import:
    • Release
    • Release Folder
    Task Groups
    Import:
    • Task Groups
    Work Items
    Import:
    • Queries

  6. Select Create.
  7. Your token is now displayed, select Copy and store the token securely. You will need this when configuring the data source in SquaredUp.

Configuring the data source

  1. Display name:
    Enter a name for your data source. This helps you to identify this data source in the list of your data sources.

  2. Agent group :
    Select the Agent Group that contains the agent(s) you want to use.

  3. Product:
    Choose your Azure DevOps instance hosting location:
    • Select Azure DevOps Cloud for the Microsoft-hosted version.
    • Select Azure DevOps Server if you're using an on-prem Azure DevOps instance.
  4. For Azure DevOps Cloud:
    • Organization name:
      Enter the name of the Azure DevOps organization you want to use for this data source.
    • Authentication type:
      For Azure DevOps Cloud you have a choice of Microsoft Entra ID or the personal access token.
  5. For Azure DevOps Server:
    • Base URL:
      Enter the base URL that your server can be reached on (e.g. https://server.company.tld/)
    • Collection Name:
      Enter the name of the Azure DevOps Server collection.
    • Personal access token:
      Enter the Azure DevOps Personal Access Token generated in Generating a Personal Access Token section.

  6. Restrict access to this data source:
    Optionally, enable this toggle if you only want certain users/groups to have access to the data source, or those with the permission to link it to new workspaces. See data source access control for more information.

  7. Click Test and add to validate the data source configuration. SquaredUp will now attempt to connect to SquaredUp using the provided authentication method.

    • Testing passed – a success message will be displayed and then the configuration will be saved.
    • Testing passed with warnings – warnings will be listed and potential fixes suggested. You can still use the data source with warnings. Select Save with warnings if you believe that you can still use the data source as required with the warnings listed. Alternatively, address the issues listed and then select Rerun tests to validate the data source configuration again. If the validation now passes, click Save.
    • Testing Failed – errors will be listed and potential fixes suggested. You cannot use the data source with errors. You are able to select Save with errors if you believe that a system outside of SquaredUp is causing the error that you need to fix. Alternatively, address the issues listed and then select Rerun tests to validate the data source configuration again. If the validation now passes, click Save.

    You can edit data source configurations at any time from Settings > Data Sources.

Testing and troubleshooting

When the data source is added or edited, SquaredUp validates your selected authentication method against the provided organization to ensure that you have access. If you are using a Personal Access Token, it will also check its validity.

The errors or warnings listed below may be displayed while using the Azure DevOps data source. If you encounter an error, please refer to the guidance provided below or contact our support team in-app or via SquaredUp Support

Error
What to do
Unable to retrieve projects. Check personal access token.
Azure DevOps doesn’t provide an endpoint to verify the token; therefore, SquaredUp attempts to list projects in the provided organization using the chosen authentication method.
If you encounter this error, ensure that the Organization Name is valid and that the provided authentication method has at least Project view permission.
Unable to access global analytics. You may not be able to use analytics based data streams until you grant analytics permissions.
This warning is displayed if SquaredUp encounters an error while trying to access the analytics endpoint. Unless you do not plan to use analytics-based data streams, you must provide Read access to Analytics. Please check that the user has Analytics read permission.
If you are using a Personal Access Token (PAT), ensure that it has read access for Analytics.
WARNING: No permissions to import ‘Type’ (e.g. Build Folders, Builds/ Release Folders/ Releases/ Environment/ Task Groups, Deployment Groups/ Repositories/ Feeds/ Queries’
This warning is displayed if SquaredUp encounters an error while indexing the displayed type from your Azure DevOps organization. This likely indicates a permission error.
Confirm that the token or the user has at least read permission for the displayed type in the warning. If you do not plan to use data streams for that type, you can ignore the warning.

Next Steps

Now that you have successfully added the data source, you can browse the dashboards, view all available data streams, or explore any indexed objects.

Dashboards

When configuring the Azure DevOps data source, you have the option to install pre-built dashboards. This option is recommended as these dashboards provide a great starting point.The following dashboards are included with the Azure DevOps data source:

  • Account Overview
  • Alerts
  • Costs
    • Overview
    • MCA/MCP Overview
    • EA Overview
    • reservations
  • App Insights
  • VM Compute
  • VM Disks

Objects Indexed

SquaredUp indexes various types of objects from your Azure DevOps organization. These objects are used to build dashboards and are visible when searching across SquaredUp. Drilling down into an object will provide useful metrics and properties.

Objects
Properties
Agent
  • Pool Id
  • Max Parallelism
  • Created On
  • Version
Agent Pool
  • Created On
  • Auto Provision
  • Auto Update
  • Auto Size
  • Target Size
  • Agent Cloud Id
  • Created By
  • Owner
  • Scope
  • Is Hosted
  • Pool 'type
  • Is Legacy
  • Options
Artifact Feed
  • Description
  • Project Name
  • Name, Links
  • Feed Id
  • Feed Project Id
  • Feed Scope
  • Capabilities
  • Hide Delete
  • Package Versions
Artifact Package
  • Name
  • Links
  • Artifact Id
  • Protocol Type
  • Feed Name
  • Feed Id
  • Feed Scope
  • Feed Project Id
Build Folder
  • Name
  • Links
  • Created By
  • Created On
  • Path
  • Project Name
Build Pipeline
  • Authored By
  • Created Data
  • Build Id
  • Path
  • Quality
  • Queue Status
  • Project Name
  • Pool Id
  • Queue Id
  • Name
  • Links
Deployment Group
  • Project Name
  • Name, Links
  • Deployment Group Id
  • Machine Count
  • Pool Id
  • Pool Name
Environment
  • Description
  • Created By
  • Project Name
  • Name
  • Links
  • Created Date and Environment Id
Environment Resource
  • Project Name
  • Name
  • Environment Id
  • Environment Resource Id
  • Tag(s)
Pool Queue
  • Project Id
  • Project Name
  • Pool Id
  • Pool Name
  • Is Hosted
  • Is Legacy
  • Options
  • Pool Type
  • Scope
Project
  • Default Team
  • Revision
  • Visibility
  • Description
  • Name
  • Links
Query
  • Project Name
  • Name
  • Links
  • Created By
  • Query Id
  • Path
  • Query Type
  • WIQL
  • Is Public
Release Folder
  • Description
  • Created On
  • Created By
  • Path
  • Project Name
  • Name and Links
Repo
  • Project Name
  • Name
  • Links
  • Repo Id
  • Remote URL
  • SSH URL
  • Default Branch
  • Is Disabled
Repo Folder
  • Project Name
  • Name
  • Links
  • Source
  • Item Path
  • Folder Id
  • Project Id
  • Branch Name
  • Repository Name
  • Repo Id
Release Pipeline
  • Description
  • Created On
  • Created By
  • Path
  • Project Name
  • Name
  • Links
  • Release Id
  • Release Name
  • Format and Source
Task Group
  • Authored By
  • Created Date
  • Task Group Id
  • Runs On
  • Category
  • Tasks
  • Revision
  • Description
  • Project Name
  • Name and Links

Data streams

You can use these data streams to create new tiles to show data, or if there are preconfigured dashboards installed you can copy or edit those.

Data streams standardize data from all the different shapes and formats your tools use into a straightforward tabular format.

While creating a tile you can tweak data streams by grouping or aggregating specific columns.

Depending on the kind of data, SquaredUp will automatically suggest how to visualize the result, for example as a table or line graph.

Data streams can be either global or scoped:

  • Global data streams are unscoped and return information of a general nature (e.g. "Get the current number of unused hosts").
  • A scoped data stream gets information relevant to the specific set objects supplied in the tile scope (e.g. "Get the current session count for these hosts").

See Data Streams for more information.

The following data streams are installed with this plugin.

Agent Pool Consumption (Microsoft Hosted)

Gets analytics reports for a MS hosted agent pool

Parameters
Agent Pool Consumption (Self Hosted)

Gets analytics reports for a self hosted agent pool

Parameters
Agent Pool Diagnostic Logs

Gets diagnostic logs for agent pool performance and issues

Parameters
Agent Runs

Count the number of runs executed by your agents

Agent Usage

Analyze the overall usage of agents across the organization

Agents

Gets individual agents or agents from a pool

Analytics Query

Make an organization wide analytics query without using a specific source

Parameters
Analytics Query (Filter by Object)

Make an organization wide analytics query without using a specific source

Parameters
Artifact Package Versions

View available versions of a specific artifact package

Artifact Packages

View details about packages published in your artifacts

Build Commits

View commits included in specific builds

Parameters
Build Runs

Track all triggered builds within your pipelines

Parameters
Commits

Track code commits across repositories

Parameters
Deployment Pools

Manage and track deployment pools for your environments

Deployment Targets

Manage and track deployment targets for your environments

Deployments

Monitor deployment activities across pipelines

Environment Deployment Build Runs

Analyze build runs linked to environment deployments

Environment Deployment Records

Track deployment records for specific environments

Job Queues

View and manage queued jobs across agents

Pull Requests

View all active and closed pull requests in the organization

Parameters
Release Runs

Monitor the progress and outcome of release pipelines

Parameters
Task Failures

Retrieve failed tasks in pipelines

Task Group Contents

View the tasks within a task group

Task Group References

Identify where task groups are used across pipelines

Task Group Revisions

Track changes and version history of task groups

WIQL Query

Run organization-wide WIQL queries for comprehensive insights

Parameters
WIQL Query (Filter by Object)

Query work items using Work Item Query Language (WIQL)

Parameters
Work Item Queries

Retrieve work item details based on queries

Parameters
Work Items

Retrieve work item details

Parameters

Was this article helpful?

Have more questions or facing an issue?
Submit a ticket