Azure DevOps on-prem plugin

For more information about what this plugin does and the data streams it retrieves, see:

Monitor the Builds and Releases from your Azure DevOps environment.

To add a data source click on the + next to Data Sources on the left-hand menu in SquaredUp. Search for the data source and click on it to open the Configure data source page.

Before you start

SquaredUp supports your Azure DevOps server instance, whether it is publicly accessible or within an internal network. Based on your hosting type you will have to select the correct version of the data source.

Publicly Accessible Instance: If your Azure DevOps server instance is publicly accessible, select the Azure DevOps Data Source that does not have the On-Prem label and instead refer to the Azure DevOps plugin documentation .
Internal Network Instance: If your instance is behind a company network, you will need to use the On-Prem version of the Data Source.
An on-prem data source connects a service running in your internal network to SquaredUp. They require an agent installed on a machine that has access to your internal network.

In addition to requiring a Personal Access Token for authentication,
you will also need to configure a Relay Agent. For additional information, refer to the Relay Agents Documentation.

For guidance on deploying relay agents, see the following depending on your platform type:
- Configuring an agent (Windows platforms)
- Configuring an agent (Linux platforms)

Generating a Personal Access Token

Follow the steps below to generate a PAT.

Sign in to your Azure DevOps instance
From your home page, open user settings from the top right corner and select Security.
Select + New Token.
Name your token, select the organization where you want to use the token, and then set your token to automatically expire after a set number of days.

Select the scopes for this token to authorize. You can select Show all scopes to view all scopes.

Permission	Usage
Projects and Teams (Mandatory)	Import: Projects
Analytics	Used to make an organization wide analytics query.
Build	Import: Build Pipelines Build Folder
Release	Import: Release Release Folder
Code	Import: Repos Repo folder Branch
Deployment Groups	Import: Deployment Groups
Environments	Import: Environments
Packaging	Import: Artifact Package Artifact Feed
Task Groups	Import: Task Groups
Work Items	Import: Queries

Select Create.
Your token is now displayed, select Copy and store the token securely. You will need this when configuring the data source in SquaredUp.

Configuring and deploying an agent

If you have already created an agent in SquaredUp that you can use for this data source, you can skip this step and choose the agent group you want to use while Configuring the data source.

See one of the following, depending on your platform type:

Configuring the data source

Display Name:
Enter a name for your data source. This helps you to identify this data source in the list of your data sources.
Agent Group:
Select the Agent Group that contains the agent(s) you want to use.
Product:
Select Azure DevOps Cloud.
Base URL:
This is your Azure DevOps organization’s base URL.
Collection Name:
This is the name of the Azure DevOps server collection.
Personal Access Token:
Enter the Azure DevOps Personal Access token generated in Generating a Personal Access Token section.
Ignore Certificate Errors:
Ignores certificates error when connecting to your Azure DevOps server Instance.
Restrict access to this data source:
You can enable this option if you only want certain users or groups to have access to the data source, or the permission to link it to new workspaces. See data source access control for more information.
Install sample dashboards:
Leaving this option enabled will install the sample dashboards detailed below into the current workspace.
Click Test and add to validate the data source configuration. SquaredUp will now attempt to connect to SquaredUp using the provided authentication method. If this process fails, see Testing and troubleshooting for assistance with the corresponding errors.

Testing and troubleshooting

When the data source is added or edited, SquaredUp validates your selected authentication method against the provided organization to ensure that you have access. If you are using a Personal Access Token, it will also check its validity.

The errors or warnings listed below may be displayed while using the Azure DevOps data source. If you encounter an error, please refer to the guidance provided below or contact [email protected] for assistance.

Error	What to do
`DNS lookup failure. Please check server URL.`	This error indicates that the URL in the Base URL field couldn’t be resolved to an IP address. This usually means there is a typo in the URL, or the DNS server cannot locate the domain. Verify the Base URL is correct, and ensure your server is publicly accessible if not using the On-Prem data source. For on-prem data sources, ensure the Azure DevOps server instance is accessible from the Relay Agent’s location.
`Connection timed out. Please check firewall rules, and connectivity.`	This error suggests that the connection attempt took too long and was aborted, possibly due to network issues, incorrect firewall settings, or the server being unreachable.
`Connection reset. Please check firewall rules, server URL, and IIS bindings.`	A "connection reset" error occurs when the server abruptly terminates the connection. This can happen due to incorrect firewall rules, issues with the server's configuration (like IIS bindings), or the server being down.
`Unauthorized. Please check your access token and collection name.`	This means that the request failed authentication, typically due to an invalid or expired Personal Access Token, or an incorrect collection name. Verify the Personal Access Token is valid, and the collection name is correct.
`Forbidden. Please check your access token.`	This error is displayed if SquaredUp encounters an error due to invalid personal access token or if the token does not have sufficient privilege. Verify that the token is valid and configured with the required permissions.
`Unable to access global analytics. You may not be able to use analytics based data streams until you grant analytics permissions.`	This warning is displayed if SquaredUp encounters an error while trying to access the analytics endpoint. Unless you do not plan to use analytics-based data streams, you must provide Read access to `Analytics`. Check that the user has `Analytics` read permission. If you are using a Personal Access Token (PAT), ensure that it has read access for `Analytics`.
`This Azure DevOps version does not support the requested analytics version`	When this error is shown, you should verify the Azure DevOps server version. Some server versions do not ship with certain analytics versions.
`Endpoint not available in this analytics version`	This error is shown when handling changes between versions, misspellings and similar issues.

Next Steps

Now that you have successfully added the data source, you can browse the sample dashboards, view all available data streams, or explore any indexed objects.

Sample dashboards

When configuring the Azure DevOps data source, you have the option to Install sample dashboards. This option is recommended as these dashboards provide a great starting point. The following dashboards are included with the Azure DevOps data source:

Pipeline: Overview: Displays key metrics for the selected pipeline, such as Build Status (Failures, Total Runs, Run Duration), Task Failures, and Test Statistics.
Pipeline: Trends: Provides an in-depth analysis of the selected pipeline to understand various trends such as Run Duration, Build Success Rate, Automated Test Run Duration Trend, Agent Usage, and more.
Release & Deployment Overview: Displays key release and deployment metrics for the selected release pipeline, such as a list of releases and deployments, release and deployment frequencies, and the latest releases and deployments.
Repos Overview: Displays metrics for the selected repository, such as PR statistics (recent PRs, open PRs, PRs by author, and more), branch state, and latest commits.
Work Item Overview: Displays a summary of work items for the selected project, such as Bugs and Epic statistics, and a summary of your current sprint.

Objects Indexed

SquaredUp indexesfifteentypes of objects from your Azure DevOps organization. These objects are used to build dashboards and are visible when searching across SquaredUp. Drilling down into an object will provide useful metrics and properties.

Objects	Properties
Project	Default Team Revision Visibility Description Name Links
Build Folder	Name Links Created By Created On Path Project Name
Build Pipeline	Authored By Created Data Build Id Path Quality Queue Status Project Name Pool Id Queue Id Name Links
Release Folder	Description Created On Created By Path Project Name Name and Links
Release Pipeline	Description Created On Created By Path Project Name Name Links Release Id Release Name Format and Source
Environment	Description Created By Project Name Name Links Created Date and Environment Id
Task Group	Authored By Created Date Task Group Id Runs On Category Tasks Revision Description Project Name Name and Links
Branch	Project Name Name, Links Source Ref Name Branch Id Project Id Repository Name Repo Id
Repo Folder	Project Name Name Links Source Item Path Folder Id Project Id Branch Name Repository Name Repo Id
Artifact Feed	Description Project Name Name, Links Feed Id Feed Project Id Feed Scope Capabilities Hide Delete Package Versions
Artifact Package	Name Links Artifact Id Protocol Type Feed Name Feed Id Feed Scope Feed Project Id
Query	Project Name Name Links Created By Query Id Path Query Type WIQL Is Public
Deployment Group	Project Name Name, Links Deployment Group Id Machine Count Pool Id Pool Name
Repo	Project Name Name Links Repo Id Remote URL SSH URL Default Branch Is Disabled
Environment Resource	Project Name Name Environment Id Environment Resource Id Tag(s)

Within the Map, the relationships between the objects are:

Project ——contains——> Build Folder ——contains——> Build Pipeline Project ——contains——> Release Folder ——contains——> Release Pipeline Project ——contains——> Environment ——contains——> Environment Resource Project ——contains——> Task Group Project ——contains——> Repo Folder ——contains——> Repo ——contains——> Branch Project ——contains——> Artifact Feed ——contains——> Artifact Package Project ——contains——> Query Project ——contains——> Deployment Group

Data streams

The following data streams are installed with this plugin.

Data streams standardize data from all the different shapes and formats your tools use into a straightforward tabular format.

While creating a tile you can tweak data streams by grouping or aggregating specific columns.

Depending on the kind of data, SquaredUp will automatically suggest how to visualize the result, for example as a table or line graph.

Data streams can be either global or scoped:

Global data streams are unscoped and return information of a general nature (e.g. "Get the current number of unused hosts").
A scoped data stream gets information relevant to the specific set objects supplied in the tile scope (e.g. "Get the current session count for these hosts").

See Data Streams for more information.

Data stream	Description
Active Pull Requests	Returns pull request(s) for the selected branch(s) or repo(s). Optionally, parameters can be used to filter the data returned. Supported parameters are: Use tile timeframe: Limits pull requests to the tile timeframe. Top: Limits the number of pull requests based on the specified number. Creator: Limits pull request(s) to those created by the specified creator. Reviewer: Limits pull requests to those reviewed or reviewing by the specified reviewer. Tag(s): Limits pull requests to those with the specified tag(s).
Agent Pool Consumption	Returns agent pool consumption for the selected project(s). Optionally, OData Query Parameters and Filter Expressions can be used to get data based on your requirements. For example: OData Query Parameter: `{ "$apply": "filter(IsRunning eq true AND IsHosted eq true)", "$orderby": "SamplingTime asc" }` JSON Filter Expression: `{ "and": [ { "IsRunning": true }, { "IsHosted": true } ] }`
Agent Runs	Returns agent run data for the selected project(s).
Agent Usage	Returns agent usage in the specified build folder(s), build pipeline(s) and/or Project(s).
All Pull Requests	Returns all pull request(s) associated with the selected branch(s) and/or in the selected repo(s) Use tile timeframe: Limits pull request to the tile timeframe. When selected you can choose the timeframe basis on either PRs Created or Closed time. Top: Limits the number of pull requests based on the specified number. Creator: Limits pull requests to those created by the specified creator. Reviewer: Limits pull requests to those reviewed or reviewing by the specified reviewer. Tag(s):Limits pull requests to those with the specified tag(s).
Analytics (Global)	Returns data based on your analytics query without using a specific source. Generally, scoped access should be used over global access unless there is a specific reason not to, such as needing data from multiple projects. Scoped access is faster and less likely to run into permissions issues, like accidentally trying to access a project you do not have access to. Parameters must be provided to return data. Supported parameters are: Entity Type (Endpoint): Specifies the endpoint to call, effectively the table from which you want to start your query. This is the only mandatory field, but the API will give you a warning if you do not configure any query options, as the response may be too large to handle. Query: Enter your query in the Query field. You can use mustache to replace statically defined values like pipeline ID and dates, allowing you to peg queries tightly to a specific object or dashboard time frame. For more information on creating a query, see the Azure DevOps documentation.. A mustache parameter is a dynamic value, the actual value will be inserted to replace the field in curly braces. For example, `{{timeframe.start}}` will insert the start time based on the timeframe configured within the tile, or `{{name}}` will insert the name of the object(s) in scope. Select: Limits the names of the columns to return. Top: Limits the number of data returned. Skip: Skips a specified number of records.
Analytics (Scoped)	Returns data based on your analytics query with the selected Build Pipeline and / or Project. Parameters must be provided to return data. Supported parameters are: Entity Type (Endpoint): Specifies the endpoint to call, effectively the table from which you want to start your query. This is the only mandatory field, but the API will give you a warning if you do not configure any query options, as the response may be too large to handle. Query: Enter your query in the Query field. You can use mustache to replace statically defined values like pipeline ID and dates, allowing you to peg queries tightly to a specific object or dashboard time frame. For more information on creating a query, see the Azure DevOps documentation.. A mustache parameter is a dynamic value, the actual value will be inserted to replace the field in curly braces. For example, `{{timeframe.start}}` will insert the start time based on the timeframe configured within the tile, or `{{name}}` will insert the name of the object(s) in scope. Select: Limits the names of the columns to return. Top: Limits the number of data returned. Skip: Skips a specified number of records.
Artifact Package Versions	Returns published version(s) of the selected Artifact Package(s).
Artifact Packages	Returns package(s) for the selected Artifact Feed(s).
Branch Status	Returns branch status for the selected branch(es) or for all branch(es) in the selected repo(s).
Branch Durations	Returns build duration for the selected pipeline(s) and/or for all build pipeline(s) in the selected build folder(s) and/or project(s).
Build Failures	Returns build failure(s) for the selected pipeline(s) and/or for all build pipeline(s) in the selected build folder(s) and/or project(s).
Build Runs	Returns build run(s) for the selected pipeline(s) and/or for all build pipeline(s) in the selected build folder(s) and/or project(s). Optionally, the following parameters can be used to filter the data returned: Top: Limits the number of build run(s) returned based on the specified number. Trigger: Limits the build run(s) returned based on the specified trigger. Result: Limits the build run(s) returned based on the specified result. Status: Limits the build run(s) returned based on the specified status. Some status filters cannot be used with result filters. For example, an In Progress build will never have a result as it has not finished yet. Build IDs: Limits the build run(s) returned based on the specified comma-separated list of build IDs. Branch: Limits the build run(s) returned based on the specified branch. The specified branch must start with the prefix `refs/heads/` to function correctly.
Build Runs with Stages	Returns build run(s) with stages for the selected multistage build pipeline(s)
Build in Progress	Returns build(s) in progress for the selected pipeline(s) and/or for all build pipeline(s) in the selected build folder(s) and/or project(s).
Commits	Returns commit(s) in the selected Branch(s), Repo(s) and/or Repo Folder(s). Optionally, the following parameters can be used to filter the data returned: Branch Name: Limits the number of commit(s) returned based on the specified branch. It defaults to default branch if not specified. Relevant when using Repo(s) or Repo Folder(s) as your object Author: Limits the number of commits to the specified author. Top: Limits the number of commit(s) returned based on the specified number.
Commits in Build Runs	Returns commit(s) for the selected pipeline. This data stream has the potential to make many API calls therefore we have implemented a limit on the number of builds that can be pulled in one go. We also offer the choice to limit the number of builds pulled by filtering to various specific statuses. Parameters can be used to filter the data returned. Supported parameters are: Top Builds: Mandatory field that allows you to limit the number of commit(s) returned based on the specified number. Build Trigger: Limits the commits to build run(s) returned based on the specified trigger. Result: Limits the commits to build run(s) returned based on the specified result. Status: Limits the commits to build run(s) returned based on the specified status. Some status filters cannot be used with result filters. For example, an In Progress build will never have a result as it has not finished yet.
Completed Pull Requests	Returns completed pull request(s) associated with the selected branch(s) and/or in the selected repo(s). The following parameters are available: Use tile timeframe: Limits completed pull request(s) to the tile timeframe. When selected the timeframe basis can be selected on either PRs Created or Closed time. Top: Limits the number of completed pull request(s) based on the specified number. Creator: Limits completed pull request(s) to those created by the specified creator. Reviewer: Limits completed pull request(s) to those reviewed by the specified reviewer. Tag(s): Limits completed pull request(s) to those with the specified tag(s).
Deployment Pools	Returns deployment pool(s) in your organization.
Deployment Targets	Returns deployment target(s) for the selected deployment group(s).
Deployments	Returns a list of deployments for a release.
Environment Deployment Build Runs	Returns build run(s) for the selected environment(s).
Environment Deployment Records	Returns deployment record(s) for the selected environment(s).
Job Queues	Returns agent queues data for the selected build folder(s), build pipeline(s), and/or project(s).
Pipeline Run Activity Results	Returns pipeline run activities for pipeline(s) in the selected build folder(s), build pipeline(s), and/or pipeline(s) in project(s). OData query parameters which is mandatory, and filter expressions can be used to get data based on your requirements. OData Query Parameter: Enter your query in the Query field. You can use mustache to replace statically defined values like pipeline ID and dates, allowing you to peg queries tightly to a specific object or dashboard time frame. See Stage wise failures, Task duration and Task duration trend documentation for some examples. The following example of OData query filters pipeline run activities for a specific pipeline ({{buildId}}) within a given timeframe ({{timeframe.start}}). It includes activities that are either tasks or have no specified type. The results are then ordered by the pipeline run completion date in ascending order. This query only works when the selected object is of Build Pipeline Type. `{ "$apply": "filter(Pipeline/PipelineId eq {{buildId}} and ActivityCompletedDate ge {{timeframe.start}} and (ActivityType eq null or ActivityType eq 'Task'))", "$order": "PipelineRunCompletedDateSK asc" }` You may see encounter an error Function.ResponseSizeTooLarge: Response payload size exceeded maximum allowed payload size (6291556 bytes). In this scenario, provide a filter to only get the data you are interested in. JSON Filter Expression: Filters the data based on the provided filter expression. For Example, the following filter will only return pipelines where the run outcome and task outcome are failed. `{ "and": [ { "PipelineRunOutcome": "Failed" }, { "TaskOutcome": "Failed" } ] }`
Query Work Items	Returns work items based on the selected query(s) saved in Azure DevOps Queries.
Release Runs	Returns releases for the selected release pipeline(s) and/or pipeline(s) in the selected project and/or release folder(s). Optionally, parameters can be used to filter or show more information. The following parameters are supported: Top: Returns a specific number of results. Useful if you only want the latest release etc. Status: Limits the release run(s) based on the specified status. Show stages: Displays individual release stage(s) and their status. When selected, you can choose the column style: Detail: Displays stage columns with headers reflecting the stage name and shows the status in the column cell. Summary: Displays stage columns with numerical headers for each stage and shows the status in the column cell. This choice is useful when there are release runs in the data with various stages, as it can significantly reduce the number of columns. Hover over a status cell in the column to view the stage name. Show artifacts: Displays artifacts used for the release Show tags: Displays the release pipeline's tag(s). If you experience excessive timeouts with this data stream, try using it without tags, artifacts, and stages, as each incurs added API costs.
Task Failures	Returns task(s) that have failed in builds for the selected build pipeline(s), build folder(s), and/or projects.
Task Group Contents	Returns task(s) for the selected Task Group(s).
Task Group References	Returns task references, showing where the tasks are used in build or release pipeline(s), for the selected task group(s).
Task Group Revisions	Returns task group revision(s), showing when they were changed, for the selected task group(s).
WIQL Query	Returns work item(s) based on the provided query for the selected project(s). The following parameter is mandatory: WIQL Query: Returns work items based on the provided Work Item Query Language. For more information on creating a query, see the Azure DevOps documentation.

Was this article helpful?

Have more questions or facing an issue?

Submit a ticket