Deployment success rate

Deployment success rate tracks how often deployments complete without failure. A declining success rate is often an early indicator of pipeline instability, test coverage gaps, or environmental drift that can increase overall change risk.

Configuring the tile

Configure the following in the tile editor:

1. Data Source: Select Azure DevOps.
2. Data Stream: Select Deployments.
3. Objects: Select the projects you want data from.

4. Timeframe: Select the timeframe you want to track. Note that after adding a monitor or configuring a KPI the Use dashboard timeframe option is disabled.

5. Filter | Group | Sort: Select SQL, enter the query below, then click Execute.

   The query runs against a table called dataset. Queries run using DuckDB, which is PostgreSQL-compatible. Table and column names containing spaces or mixed case must be enclosed in double quotes.

   1. Copy

      SELECT
        COUNT(*) AS total_deployments,
        SUM(CASE WHEN LOWER("deploymentStatus") = 'success' THEN 1 ELSE 0 END) AS successful_deployments,
        ROUND(
          SUM(CASE WHEN LOWER("deploymentStatus") = 'success' THEN 1 ELSE 0 END) * 100.0 / NULLIF(COUNT(*), 0),
          1
        ) AS deployment_success_rate
      FROM dataset
      WHERE "CompletedOn" IS NOT NULL;

6. Columns: Click Edit
   next to the Deployment Success Rate column and configure the following:
   • Type: Select Percent.
   • Decimal Places: Enter 0.

1. Click Save.

1. Visualization: Select Gauge.
   1. Mapping > Value: Select Deployment Success Rate.
   2. Range > Min: Select Fixed and enter 0.
   3. Range > Max: Select Fixed and enter 100.

2. Click Save.

Adding a monitor

With a healthy baseline deployment success rate of around 95%, we trigger a warning when success drops by roughly 5-10%, and an error when it falls below an acceptable operational threshold. This keeps the monitor sensitive to meaningful degradation without reacting to the occasional isolated failure.

Configure the following on the Monitoring tab:

1. Monitoring: Enable the toggle.
2. Type: Select Threshold.

1. Value: Select top.
2. Column: Select Deployment Success Rate.
3. Conditions:
   1. Error: Enable the toggle, then configure as Less than, and supply an appropriate value. For our example, we’ll enter: 85.
   2. Warning: Enable the toggle, then configure as Less than, and supply an appropriate value. For our example, we’ll enter: 90.

4. Click Save.

Publishing a KPI

To give even more value to this dashboard, we can promote key operational metrics to first-class KPIs.

Once published, this KPI can be selected as a data stream when configuring a tile, making it easier to track this metric over time and to reuse this data across your other dashboards.

Create the KPI type before assigning it to a tile:

1. Go to Settings > KPI > KPI Types.
2. Click Add KPI type.

1. Name: Enter Deployment success rate.

2. Click Save.

Configure the following on the KPI tab:

1. KPI: Enable the toggle.

1. Type: Select Deployment success rate.

2. Click Save.

Was this article helpful?