Intermediate Lesson 5 of 16

YAML Pipelines Deep Dive

Master every keyword, expression, and structural element in an azure-pipelines.yml file. From triggers and pools to parameters, conditions, resources, and runtime vs compile-time expressions — this lesson is your definitive reference for writing production-grade YAML pipelines in Azure DevOps.

🧒 Simple Explanation (ELI5)

Imagine you have a recipe book that tells a robot chef exactly how to make dinner. The recipe book is your YAML pipeline file. Every recipe has clear sections:

When to cook (trigger) — "Start cooking when a guest arrives" = "Start the pipeline when code is pushed to main."
Which kitchen to use (pool) — "Use the big kitchen with the pizza oven" = "Use an Ubuntu agent with Docker installed."
Ingredients list (variables) — "2 cups flour, 1 tsp salt" = "IMAGE_NAME=myapp, BUILD_CONFIG=Release." You write them once and reuse them everywhere in the recipe.
Recipe sections (stages) — "Appetizer → Main Course → Dessert." Each section can only start after the previous one is done (unless you have two ovens and can bake dessert in parallel).
Cooking stations (jobs) — Inside the "Main Course" section, one sous-chef grills the steak while another makes the sauce at the same time. Each job runs on its own cutting board (agent).
Individual instructions (steps) — "Chop the onions. Heat the oil. Sauté for 3 minutes." Each step runs one after another at a single station.
Special instructions (conditions) — "IF the guest is vegetarian, SKIP the steak station." Conditions let you skip or include steps based on variables.
Borrowed recipes (resources) — "For the pasta sauce, follow Grandma's recipe from Recipe Book #2." Resources let your pipeline pull in code, templates, or containers from other repositories.
Customizable recipes (parameters) — "How spicy? Mild / Medium / Hot." Parameters let you choose options when you start the pipeline, like picking a branch name or toggling a feature flag.

💡

Why This Matters

Just like a detailed recipe prevents cooking disasters, a well-structured YAML pipeline prevents deployment disasters. Every keyword has a purpose. Once you understand the anatomy, you can build any CI/CD workflow from scratch — multi-environment deployments, matrix builds, conditional releases, and more — all version-controlled in a single file.

🔧 Technical Explanation

An azure-pipelines.yml file is the single source of truth for a YAML pipeline. Let's break down every top-level keyword and how they compose together.

trigger — When the Pipeline Runs

The trigger keyword defines the CI trigger — events that automatically start the pipeline when code is pushed.

yaml

# ── Branch trigger ──
trigger:
  branches:
    include:
      - main
      - release/*
    exclude:
      - feature/experimental-*

  # ── Path trigger ──
  paths:
    include:
      - src/**
      - Dockerfile
    exclude:
      - docs/**
      - README.md

  # ── Tag trigger ──
  tags:
    include:
      - v*
    exclude:
      - v*-beta

You can also define PR triggers to validate pull requests before merging:

yaml

# ── PR trigger ──
pr:
  branches:
    include:
      - main
      - release/*
  paths:
    include:
      - src/**
  drafts: false  # Don't run on draft PRs

Scheduled triggers use cron syntax:

yaml

schedules:
  - cron: '0 2 * * 1-5'          # 2 AM UTC, weekdays only
    displayName: 'Nightly build'
    branches:
      include: [main]
    always: false                  # Only run if code changed since last scheduled run

⚠️

UI Override Trap

The Azure DevOps UI can override YAML triggers. Go to Pipeline → Edit → ⋮ → Triggers and check if "Override the YAML continuous integration trigger from here" is enabled. If it is, the UI settings take precedence over what's in your YAML file — a common source of "why didn't my pipeline trigger?" bugs.

To completely disable automatic triggers and make the pipeline manual-only:

yaml

trigger: none
pr: none

pool — Where the Pipeline Runs

The pool keyword specifies which agent pool executes the jobs. You can set it at the pipeline level (applies to all jobs) or per-job (overrides the pipeline default).

yaml

# Microsoft-hosted agent — simplest form
pool:
  vmImage: 'ubuntu-latest'

# Other vmImage options:
#   'windows-latest'
#   'macos-latest'
#   'ubuntu-22.04'  (pin specific version)

yaml

# Self-hosted agent pool with demands
pool:
  name: 'SelfHostedLinux'
  demands:
    - Agent.OS -equals Linux
    - docker
    - kubectl

demands filter which self-hosted agent in the pool can run the job. The agent must advertise matching capabilities. If no agent matches, the job waits indefinitely.

variables — Reusable Values

Variables let you define values once and reference them throughout the pipeline. There are three ways to define them:

yaml

# ── Inline variables ──
variables:
  buildConfiguration: 'Release'
  imageName: 'myapp'
  tag: '$(Build.BuildId)'            # Predefined variable

# ── Variable group (linked from Library) ──
variables:
  - group: 'production-secrets'       # Defined in Pipelines → Library
  - name: buildConfiguration
    value: 'Release'

# ── Variable template (from another file) ──
variables:
  - template: vars/common-vars.yml    # Imports variables from a template file
  - name: environment
    value: 'staging'

⚡

Variable Syntax: Three Flavors

Azure DevOps has three variable syntaxes that behave differently:

Macro syntax $(variableName) — Expanded at runtime by the agent. Works in task inputs and scripts. This is the most common form.
Template expression ${{ variables.variableName }} — Expanded at compile time before the pipeline runs. Used in YAML structure (conditions, templates). Cannot access runtime-only values like output variables.
Runtime expression $[ variables.variableName ] — Expanded at runtime but only in specific contexts: variable definitions and conditions. Supports functions like eq(), ne(), etc.

stages → jobs → steps — The Execution Hierarchy

The core of any YAML pipeline is the three-level hierarchy: stages contain jobs, which contain steps.

yaml

stages:
  - stage: Build
    displayName: 'Build & Test'
    jobs:
      - job: BuildJob
        displayName: 'Compile and Unit Test'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          # ── checkout: fetch the source code ──
          - checkout: self
            clean: true
            fetchDepth: 1               # Shallow clone for speed

          # ── script: inline shell command ──
          - script: |
              echo "Building $(buildConfiguration)..."
              dotnet build --configuration $(buildConfiguration)
            displayName: 'Build the application'

          # ── bash: explicitly Bash ──
          - bash: |
              echo "Running on $(Agent.OS)"
            displayName: 'Bash step'

          # ── powershell: PowerShell Core (cross-platform) ──
          - powershell: |
              Write-Host "PowerShell Core step"
            displayName: 'PowerShell step'

          # ── task: a built-in or marketplace task ──
          - task: DotNetCoreCLI@2
            displayName: 'Run unit tests'
            inputs:
              command: test
              projects: '**/*Tests.csproj'
              arguments: '--configuration $(buildConfiguration)'

  - stage: Deploy
    displayName: 'Deploy to Staging'
    dependsOn: Build                    # Runs after Build stage completes
    condition: succeeded()              # Only if Build succeeded
    jobs:
      - deployment: DeployStaging
        displayName: 'Deploy to Staging Environment'
        environment: 'staging'          # Links to an Azure DevOps environment
        strategy:
          runOnce:
            deploy:
              steps:
                - script: echo "Deploying to staging..."
                  displayName: 'Run deployment'

Step types available:

Step Type	Description	When to Use
`script`	Runs a shell command (Bash on Linux/macOS, cmd on Windows)	Quick inline commands, platform-adaptive
`bash`	Explicitly runs Bash (even on Windows if Bash is available)	When you need guaranteed Bash syntax
`powershell`	Runs PowerShell Core (pwsh) — cross-platform	PowerShell scripts on any OS
`pwsh`	Alias for `powershell`	Same as `powershell`
`task`	Runs a versioned, pre-built task (built-in or marketplace)	Complex operations — Docker, Kubernetes, .NET, npm, etc.
`checkout`	Clones a Git repository	Fetching source code (auto-added for `self`)
`download`	Downloads pipeline artifacts	Retrieving artifacts from previous stages/pipelines
`publish`	Publishes a pipeline artifact	Storing build outputs for later stages
`template`	Includes steps from another YAML file	Reuse and DRY — shared step sequences

💡

Implicit Stage & Job

For simple pipelines, you can skip stages: and jobs: entirely and just write steps: at the top level. Azure DevOps wraps them in an implicit single stage and single job. This is perfectly valid for basic CI-only pipelines.

conditions — Conditional Execution

Conditions control whether a stage, job, or step executes. They use expression functions:

yaml

# ── Common condition functions ──
condition: succeeded()                          # Default — run if all previous succeeded
condition: failed()                             # Run only if a previous stage/job failed
condition: always()                             # Run regardless of success or failure
condition: canceled()                           # Run only if the pipeline was canceled

# ── Equality / comparison ──
condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')
condition: ne(variables['Build.Reason'], 'PullRequest')
condition: startsWith(variables['Build.SourceBranch'], 'refs/heads/release/')
condition: contains(variables['Build.SourceVersionMessage'], '[skip ci]')

# ── Logical operators ──
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
condition: or(eq(variables['env'], 'staging'), eq(variables['env'], 'production'))
condition: not(eq(variables['skipTests'], 'true'))

# ── Compile-time conditions (template expressions) ──
${{ if eq(parameters.environment, 'production') }}:
  pool:
    name: 'ProdAgents'
${{ else }}:
  pool:
    vmImage: 'ubuntu-latest'

Runtime Expressions vs Compile-Time Expressions

This is one of the most confusing areas of Azure Pipelines YAML. Understanding the difference is critical:

Aspect	Compile-Time `${{ }}`	Runtime `$[ ]`
When evaluated	Before the pipeline runs (during YAML parsing)	During pipeline execution on the agent
Can access	Parameters, template variables, static variable values	Variables (including output variables from previous jobs), predefined variables
Cannot access	Runtime-only values (output variables, dynamic values)	Parameters (already resolved at compile time)
Used for	Conditional insertion of YAML blocks, template logic, `each` loops	Conditional variable values, dynamic conditions
Syntax example	`${{ if eq(parameters.env, 'prod') }}`	`$[ eq(variables['Build.SourceBranch'], 'refs/heads/main') ]`

yaml

# Compile-time: conditionally include a whole job
stages:
  - stage: Deploy
    jobs:
      - ${{ if eq(parameters.deployTarget, 'aks') }}:
        - job: DeployToAKS
          steps:
            - script: echo "Deploying to AKS"

      - ${{ if eq(parameters.deployTarget, 'appservice') }}:
        - job: DeployToAppService
          steps:
            - script: echo "Deploying to App Service"

# Runtime: set a variable based on branch
variables:
  isMain: $[ eq(variables['Build.SourceBranch'], 'refs/heads/main') ]

steps:
  - script: echo "Is main branch: $(isMain)"

⚠️

Common Pitfall

You cannot use compile-time expressions ${{ }} to access output variables from previous jobs — those values don't exist yet when the YAML is compiled. Use runtime expressions $[ ] or macro syntax $() for output variables. Mixing them up causes silent empty-string substitutions that are painful to debug.

Make YAML Easier to Read

The main reason Azure Pipelines YAML feels confusing is not the syntax itself. It is that many teams write the file in execution order only, not in reader order. A readable pipeline answers four questions top to bottom: when does it run, where does it run, what are the shared values, and what stages does it execute.

Read This First	Why	Typical Keywords
Trigger	Tells you what event starts the pipeline.	`trigger`, `pr`, `schedules`
Execution context	Shows what agent and resources are available.	`pool`, `resources`
Shared inputs	Explains which values change between environments or runs.	`variables`, `parameters`
Stage flow	Shows the release path and where approvals or conditions apply.	`stages`, `jobs`, `dependsOn`, `condition`

Bad → Better Example 1: Flat Steps vs Named Stages

yaml

# Harder to reason about
steps:
- script: npm ci
- script: npm test
- task: Docker@2
  inputs:
    command: buildAndPush
- script: helm upgrade --install webapp ./charts/webapp

yaml

# Easier to read and debug
stages:
- stage: Validate
  jobs:
  - job: Test
    steps:
    - script: npm ci
    - script: npm test

- stage: Build
  dependsOn: Validate
  jobs:
  - job: BuildImage
    steps:
    - task: Docker@2
      inputs:
        command: buildAndPush

- stage: Deploy
  dependsOn: Build
  jobs:
  - deployment: DeployToAKS
    environment: staging
    strategy:
      runOnce:
        deploy:
          steps:
          - script: helm upgrade --install webapp ./charts/webapp

Bad → Better Example 2: Hardcoded Values vs Shared Variables

yaml

# Hard to reuse
- script: |
    docker build -t contosoregistry.azurecr.io/webapp:latest .
    docker push contosoregistry.azurecr.io/webapp:latest
    helm upgrade --install webapp ./charts/webapp --namespace production

yaml

# Easier to promote across environments
variables:
- group: aks-shared
- name: imageRepository
  value: webapp
- name: imageTag
  value: $(Build.BuildId)

- script: |
    docker build -t $(acrLoginServer)/$(imageRepository):$(imageTag) .
    docker push $(acrLoginServer)/$(imageRepository):$(imageTag)
    helm upgrade --install webapp ./charts/webapp \
      --namespace $(namespace) \
      --set image.tag=$(imageTag)

Bad → Better Example 3: Hidden Deploy Rules vs Explicit Conditions

yaml

# Production behavior is implicit and easy to miss
- stage: Deploy
  jobs:
  - job: ProdRelease
    steps:
    - script: echo "Deploying"

yaml

# Production behavior is obvious on first read
- stage: Deploy_Production
  dependsOn: Deploy_Staging
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
  jobs:
  - deployment: ProductionDeploy
    environment: production
    strategy:
      runOnce:
        deploy:
          steps:
          - script: echo "Deploying to production after approval"

💡

Clarity Rule

If a new engineer cannot tell where production deployment happens within 15 seconds of opening the YAML file, the pipeline is too implicit. Prefer explicit stage names, explicit conditions, and a small number of clearly named variables.

resources — External Dependencies

The resources keyword declares external resources the pipeline depends on:

yaml

resources:
  # ── Other repositories ──
  repositories:
    - repository: templates            # Alias used in the pipeline
      type: git                        # 'git' = Azure Repos, 'github' = GitHub
      name: MyProject/pipeline-templates
      ref: refs/heads/main

    - repository: shared-scripts
      type: github
      name: myorg/shared-scripts
      endpoint: 'github-service-connection'

  # ── Other pipelines (for artifact consumption or triggers) ──
  pipelines:
    - pipeline: buildPipeline          # Alias
      source: 'MyApp-CI'              # Name of the source pipeline
      trigger:
        branches:
          include: [main]

  # ── Container images (for container jobs) ──
  containers:
    - container: buildtools
      image: myregistry.azurecr.io/buildtools:latest
      endpoint: 'acr-service-connection'

Once declared, you reference resources by their alias:

yaml

# Using a repository resource for templates
stages:
  - template: stages/build.yml@templates   # @templates = the repository alias

# Using a container resource for a job
jobs:
  - job: BuildInContainer
    container: buildtools                   # Run this job inside the container
    steps:
      - script: dotnet build

parameters — Pipeline Inputs

Parameters let users provide input values when manually triggering a pipeline (or when calling a template). They support typed inputs with defaults and allowed values:

yaml

parameters:
  - name: environment
    displayName: 'Target Environment'
    type: string
    default: 'staging'
    values:
      - dev
      - staging
      - production

  - name: runTests
    displayName: 'Run test suite?'
    type: boolean
    default: true

  - name: dotnetVersion
    displayName: '.NET SDK Version'
    type: string
    default: '8.0.x'

  - name: replicas
    displayName: 'Number of replicas'
    type: number
    default: 3

  - name: regions
    displayName: 'Deployment regions'
    type: object
    default:
      - eastus
      - westeurope

  - name: additionalSteps
    displayName: 'Extra steps to run'
    type: stepList
    default: []

Parameter types:

Type	Description	Example Value
`string`	Free-form text or constrained by `values` list	`'staging'`
`number`	Integer value	`3`
`boolean`	True or false — renders as a checkbox in the UI	`true`
`object`	YAML object or array — for complex structured data	`[eastus, westeurope]`
`stepList`	A list of steps — allows callers to inject custom steps into a template	`- script: echo hi`

Reference parameters with compile-time syntax: ${{ parameters.environment }}

yaml

# Using parameters in stages
stages:
  - stage: Deploy
    displayName: 'Deploy to ${{ parameters.environment }}'
    jobs:
      - job: DeployApp
        steps:
          - script: |
              echo "Deploying to ${{ parameters.environment }}"
              echo "Replicas: ${{ parameters.replicas }}"

          # Conditional step based on boolean parameter
          - ${{ if eq(parameters.runTests, true) }}:
            - script: dotnet test
              displayName: 'Run tests'

          # Loop over object parameter
          - ${{ each region in parameters.regions }}:
            - script: echo "Deploying to ${{ region }}"
              displayName: 'Deploy to ${{ region }}'

          # Inject additional steps from stepList parameter
          - ${{ parameters.additionalSteps }}

📊 YAML Pipeline Hierarchy Diagram

azure-pipelines.yml — Structural Hierarchy

trigger / pr / schedules
When does the pipeline start?

↳

pool (vmImage / name + demands)
Where does it run?

↳

variables / parameters
Inputs and reusable values

↳

Stage: Build

Stage: Test

Stage: Deploy

↳

Job: Compile
runs on Agent A

Job: UnitTest
runs on Agent B

Job: Lint
runs on Agent C

↳

checkout

script

task

publish

Expression Evaluation Timeline

Compile Time

⏱️ Runs before pipeline starts

📝 Syntax: ${{ expression }}

✅ Parameters, static variables

❌ No output variables

🔧 if/else, each loops

Runtime

⏱️ Runs during pipeline execution

📝 Syntax: $[ expression ]

✅ All variables, output vars

❌ No parameters (already resolved)

🔧 Conditions, dynamic values

Macro $()

⏱️ Expanded by the agent

📝 Syntax: $(variableName)

✅ Task inputs, scripts

❌ Not in conditions

🔧 Most common syntax

📄 Complete YAML Pipeline Example

A real-world .NET + Docker pipeline that builds, tests, pushes a container image, and publishes artifacts — fully annotated:

yaml

# ╔══════════════════════════════════════════════════════════════╗
# ║  azure-pipelines.yml — .NET 8 + Docker CI Pipeline          ║
# ╚══════════════════════════════════════════════════════════════╝

# ── TRIGGER: run on push to main or release branches ──
trigger:
  branches:
    include:
      - main
      - release/*
  paths:
    exclude:
      - docs/**
      - '*.md'

# ── PR TRIGGER: validate pull requests ──
pr:
  branches:
    include: [main]
  drafts: false

# ── PARAMETERS: user inputs when running manually ──
parameters:
  - name: buildConfiguration
    displayName: 'Build Configuration'
    type: string
    default: 'Release'
    values: ['Debug', 'Release']
  - name: pushImage
    displayName: 'Push Docker image to ACR?'
    type: boolean
    default: true

# ── VARIABLES ──
variables:
  - group: 'acr-credentials'                   # Variable group from Library
  - name: solution
    value: '**/*.sln'
  - name: imageRepository
    value: 'myapp'
  - name: containerRegistry
    value: 'myregistry.azurecr.io'
  - name: dockerfilePath
    value: 'src/MyApp/Dockerfile'
  - name: tag
    value: '$(Build.BuildId)'
  - name: isMainBranch
    value: $[ eq(variables['Build.SourceBranch'], 'refs/heads/main') ]

# ── RESOURCES ──
resources:
  repositories:
    - repository: templates
      type: git
      name: MyProject/pipeline-templates
      ref: refs/heads/main

# ════════════════════════════════════════════════════════════════
#  STAGES
# ════════════════════════════════════════════════════════════════
stages:

  # ── STAGE 1: BUILD & TEST ──
  - stage: Build
    displayName: 'Build & Test'
    jobs:
      - job: BuildAndTest
        displayName: 'Restore → Build → Test'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          # Checkout source code (shallow clone for speed)
          - checkout: self
            clean: true
            fetchDepth: 1

          # Install .NET SDK
          - task: UseDotNet@2
            displayName: 'Install .NET 8 SDK'
            inputs:
              packageType: 'sdk'
              version: '8.0.x'

          # Restore NuGet packages
          - task: DotNetCoreCLI@2
            displayName: 'Restore packages'
            inputs:
              command: restore
              projects: '$(solution)'

          # Build the solution
          - task: DotNetCoreCLI@2
            displayName: 'Build solution'
            inputs:
              command: build
              projects: '$(solution)'
              arguments: '--configuration ${{ parameters.buildConfiguration }} --no-restore'

          # Run unit tests with code coverage
          - task: DotNetCoreCLI@2
            displayName: 'Run unit tests'
            inputs:
              command: test
              projects: '**/*Tests.csproj'
              arguments: '--configuration ${{ parameters.buildConfiguration }} --collect:"XPlat Code Coverage"'

          # Publish test results
          - task: PublishTestResults@2
            displayName: 'Publish test results'
            inputs:
              testResultsFormat: 'VSTest'
              testResultsFiles: '**/*.trx'
            condition: always()          # Publish even if tests fail

          # Publish build output as artifact
          - task: DotNetCoreCLI@2
            displayName: 'Publish application'
            inputs:
              command: publish
              projects: 'src/MyApp/MyApp.csproj'
              arguments: '--configuration ${{ parameters.buildConfiguration }} --output $(Build.ArtifactStagingDirectory)'

          - publish: $(Build.ArtifactStagingDirectory)
            artifact: 'app-drop'
            displayName: 'Upload build artifact'

  # ── STAGE 2: DOCKER BUILD & PUSH ──
  - stage: Docker
    displayName: 'Docker Build & Push'
    dependsOn: Build
    condition: and(succeeded(), eq(variables.isMainBranch, true))
    jobs:
      - job: DockerBuildPush
        displayName: 'Build and push container image'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: Docker@2
            displayName: 'Build Docker image'
            inputs:
              containerRegistry: '$(containerRegistry)'
              repository: '$(imageRepository)'
              command: 'build'
              Dockerfile: '$(dockerfilePath)'
              tags: |
                $(tag)
                latest

          - ${{ if eq(parameters.pushImage, true) }}:
            - task: Docker@2
              displayName: 'Push to ACR'
              inputs:
                containerRegistry: '$(containerRegistry)'
                repository: '$(imageRepository)'
                command: 'push'
                tags: |
                  $(tag)
                  latest

🛠️ Hands-on: Build a Multi-Job Pipeline

Let's build a pipeline with four parallel/sequential jobs: Lint → Build → Test → Publish Artifact. This exercise uses every major keyword you've learned.

Step 1 — Create the pipeline file

In your repository root, create azure-pipelines.yml:

yaml

# ── Trigger: CI on main, PR validation ──
trigger:
  branches:
    include: [main]
  paths:
    exclude: ['docs/**', '*.md']

pr:
  branches:
    include: [main]

# ── Parameters: allow manual configuration ──
parameters:
  - name: buildConfig
    displayName: 'Build configuration'
    type: string
    default: 'Release'
    values: ['Debug', 'Release']

  - name: skipLint
    displayName: 'Skip linting?'
    type: boolean
    default: false

# ── Variables ──
variables:
  solution: '**/*.sln'
  artifactName: 'app-package'

# ── Pool (pipeline-level default) ──
pool:
  vmImage: 'ubuntu-latest'

Step 2 — Define stages and jobs

yaml

stages:
  # ═══ STAGE: VALIDATE ═══
  - stage: Validate
    displayName: 'Lint & Validate'
    jobs:
      - job: Lint
        displayName: 'Run linter'
        # Condition: skip if parameter says so
        condition: ne('${{ parameters.skipLint }}', 'true')
        steps:
          - checkout: self
            fetchDepth: 1

          - script: |
              echo "Installing lint tools..."
              dotnet tool install -g dotnet-format
            displayName: 'Install dotnet-format'

          - script: |
              echo "Running linter on all .cs files..."
              dotnet format $(solution) --verify-no-changes --verbosity diagnostic
            displayName: 'Lint: dotnet format'

  # ═══ STAGE: BUILD ═══
  - stage: Build
    displayName: 'Build Application'
    dependsOn: Validate
    condition: succeeded()
    jobs:
      - job: Compile
        displayName: 'Compile the solution'
        steps:
          - checkout: self
            fetchDepth: 1

          - task: UseDotNet@2
            displayName: 'Install .NET SDK'
            inputs:
              packageType: 'sdk'
              version: '8.0.x'

          - task: DotNetCoreCLI@2
            displayName: 'Restore NuGet packages'
            inputs:
              command: restore
              projects: '$(solution)'

          - task: DotNetCoreCLI@2
            displayName: 'Build solution'
            inputs:
              command: build
              projects: '$(solution)'
              arguments: '--configuration ${{ parameters.buildConfig }} --no-restore'

          # Set an output variable for downstream jobs
          - bash: |
              echo "##vso[task.setvariable variable=buildSucceeded;isOutput=true]true"
            name: buildResult
            displayName: 'Set output variable'

  # ═══ STAGE: TEST ═══
  - stage: Test
    displayName: 'Run Tests'
    dependsOn: Build
    condition: succeeded()
    jobs:
      # Two test jobs run IN PARALLEL
      - job: UnitTests
        displayName: 'Unit tests'
        steps:
          - checkout: self
          - task: DotNetCoreCLI@2
            displayName: 'Run unit tests'
            inputs:
              command: test
              projects: '**/*UnitTests.csproj'
              arguments: '--configuration ${{ parameters.buildConfig }}'

      - job: IntegrationTests
        displayName: 'Integration tests'
        steps:
          - checkout: self
          - task: DotNetCoreCLI@2
            displayName: 'Run integration tests'
            inputs:
              command: test
              projects: '**/*IntegrationTests.csproj'
              arguments: '--configuration ${{ parameters.buildConfig }}'

  # ═══ STAGE: PUBLISH ═══
  - stage: Publish
    displayName: 'Publish Artifact'
    dependsOn: Test
    condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
    jobs:
      - job: PublishArtifact
        displayName: 'Publish build artifact'
        steps:
          - checkout: self
          - task: DotNetCoreCLI@2
            displayName: 'dotnet publish'
            inputs:
              command: publish
              projects: 'src/MyApp/MyApp.csproj'
              arguments: '--configuration ${{ parameters.buildConfig }} --output $(Build.ArtifactStagingDirectory)'

          - publish: $(Build.ArtifactStagingDirectory)
            artifact: '$(artifactName)'
            displayName: 'Upload artifact: $(artifactName)'

💡

Keywords Used in This Exercise

trigger, pr, parameters (string, boolean), variables, pool, stages, dependsOn, condition (succeeded, ne, and, eq), jobs, steps, checkout, script, bash, task, publish, output variables (##vso[task.setvariable]), and compile-time expressions (${{ }}).

Step 3 — Run and verify

Commit and push azure-pipelines.yml to your repository.
In Azure DevOps, go to Pipelines → New Pipeline → Azure Repos Git → select your repo → Existing Azure Pipelines YAML file → pick /azure-pipelines.yml.
Click Run. In the parameters panel, you'll see the buildConfig dropdown and skipLint checkbox.
Watch the stages execute: Validate → Build → Test (Unit + Integration in parallel) → Publish.
Open the Publish stage and confirm the artifact appears under the run's "Published" tab.

🐛 Debugging Scenarios

Scenario 1: "Unexpected value 'stage'" Error

Symptom: The pipeline fails immediately with a YAML parsing error like /azure-pipelines.yml: Unexpected value 'stage'.

yaml — BROKEN

# ❌ WRONG — indentation error
stages:
- stage: Build
  jobs:
  - job: BuildJob
    steps:
     - script: echo hello     # ← 1 space indent instead of expected level
      displayName: 'Greet'    # ← this line is misaligned

Root cause: YAML is indentation-sensitive. Misaligned keys cause the parser to interpret them as unexpected values. The most common causes: mixing tabs and spaces, wrong nesting level, or a missing - prefix for list items.

yaml — FIXED

# ✅ CORRECT — consistent 2-space indentation
stages:
- stage: Build
  jobs:
  - job: BuildJob
    steps:
    - script: echo hello
      displayName: 'Greet'

Fix: Use the Azure DevOps pipeline editor's "Validate" button to catch syntax errors before committing. Configure your editor to use spaces (not tabs) and show whitespace characters.

Scenario 2: "Variable not expanding" — Stays as literal text

Symptom: Your script outputs the literal string $(myVar) instead of the variable's value.

Root cause: You're using the wrong expression syntax for the context:

Problem	Cause	Fix
Script outputs `$(myVar)` literally	Variable is not defined or name is misspelled (undefined macro variables are left as-is)	Check variable name spelling and scope. Variables from other jobs need `dependencies` syntax.
Condition always evaluates to empty string	Using `${{ variables.myVar }}` (compile-time) for a runtime-only variable	Switch to runtime: `$[ variables.myVar ]` or `eq(variables['myVar'], 'value')`
Output variable from Job A is empty in Job B	Using `$(myVar)` — output variables require dependency mapping	Use: `$[ dependencies.JobA.outputs['stepName.myVar'] ]`
Secret variable not available in script	Secret variables are NOT automatically mapped to environment variables	Explicitly pass it: `env: { MY_SECRET: $(secretVar) }`

⚠️

Debugging Tip

Add variables: { system.debug: 'true' } to your pipeline to enable verbose logging. This shows how every variable is resolved and which expression syntax was used at each step.

Scenario 3: "Condition not evaluating" — Stage/Job always runs or never runs

Symptom: A condition like condition: eq(variables.isRelease, 'true') always evaluates to false, even when isRelease is set to true.

Root cause: Expression syntax mismatch. In condition: fields, you must use bracket notation for variables:

yaml

# ❌ WRONG — dot notation doesn't work in condition expressions
condition: eq(variables.isRelease, 'true')

# ✅ CORRECT — use bracket notation
condition: eq(variables['isRelease'], 'true')

# ✅ Also works for predefined variables
condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')

Other common causes:

Boolean vs string: A variable set to the YAML boolean true is the string "True" at runtime. Use eq(variables['myBool'], 'True') — note the capital T.
Missing succeeded(): When you write a custom condition, you override the default succeeded(). If you want "run only on main AND only if previous stage passed," you need: condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main')).
Stage-level vs job-level: A stage condition checks the status of previous stages. A job condition checks previous jobs within the same stage. Don't confuse their scopes.

Scenario 4: "Resource not authorized" Error

Symptom: The pipeline fails with: Resource not authorized. Waiting for authorization.

Root cause: Azure DevOps requires explicit authorization for protected resources the first time a pipeline uses them. Protected resources include:

Service connections (Azure, Docker, Kubernetes, GitHub)
Agent pools (especially self-hosted pools)
Environments (staging, production)
Variable groups (from Pipelines Library)
Secure files (certificates, key files)

Fix:

Open the failed pipeline run — you'll see a "View" or "Authorize" button next to the blocked resource.
Click Authorize to grant this pipeline access to the resource.
Alternatively, pre-authorize: go to Project Settings → Service Connections → [connection] → Security and add the pipeline to the allowed list.
For environments: Pipelines → Environments → [env] → Security → grant pipeline permissions.

⚡

Security Best Practice

Don't grant "all pipelines" access to sensitive resources. Authorize each pipeline individually, especially for production service connections and environments. This follows the principle of least privilege and prevents a compromised pipeline from accessing resources it shouldn't.

🎯 Interview Questions

Beginner

Q: What is a YAML pipeline in Azure DevOps?▼

A YAML pipeline is a CI/CD workflow defined as code in a .yml file (typically azure-pipelines.yml) stored in your Git repository alongside your application code. It describes the entire build, test, and deployment process using a structured YAML syntax with keywords like trigger, pool, stages, jobs, and steps. Because it lives in the repo, the pipeline definition is version-controlled, supports code review via pull requests, allows branching (each branch can have its own pipeline configuration), and provides a full audit trail. It replaces the older Classic (GUI-based) pipeline editor and is Microsoft's primary investment going forward.

Q: What is the execution hierarchy of a YAML pipeline?▼

The hierarchy from top to bottom is: Pipeline (the YAML file itself) → Stage (a logical boundary like Build, Test, Deploy — runs sequentially by default) → Job (a unit of work that runs on a single agent — jobs within a stage can run in parallel) → Step (an individual operation — script, task, checkout, publish — runs sequentially within a job). For simple pipelines, stages and jobs are optional — you can define steps directly at the top level, and Azure DevOps wraps them in an implicit stage and job automatically.

Q: What are the different trigger types in Azure Pipelines?▼

There are four trigger types: 1. CI trigger (trigger:) — fires on code pushes to specified branches, with optional path and tag filters. 2. PR trigger (pr:) — fires when a pull request is created or updated, running a validation build. 3. Scheduled trigger (schedules:) — fires on a cron schedule (e.g., nightly builds). 4. Pipeline resource trigger (resources: pipelines:) — fires when another pipeline completes, enabling pipeline chaining. You can combine multiple trigger types in one YAML file. Setting trigger: none disables CI triggers, making the pipeline manual-only.

Q: What is the difference between a task and a script step?▼

A task is a pre-built, versioned, reusable unit of pipeline logic — either built-in (e.g., DotNetCoreCLI@2, Docker@2) or from the Azure DevOps Marketplace. Tasks have typed inputs, outputs, and are maintained by Microsoft or the community. A script step is an inline shell command — you write raw Bash, PowerShell, or cmd commands directly in the YAML file. Use tasks for complex, well-tested operations (Docker builds, Kubernetes deployments, test publishing). Use scripts for quick, simple commands (echo, file operations, custom logic). Tasks are more portable and maintainable; scripts are more flexible and immediate.

Q: What is the pool keyword and what are the two agent types?▼

The pool keyword specifies which agent pool runs the pipeline jobs. Azure DevOps has two agent types: Microsoft-hosted agents are fully managed VMs provided by Microsoft — you specify vmImage: 'ubuntu-latest' (or windows/macOS) and get a fresh VM for each job with pre-installed tools. They require zero maintenance but cannot access private networks. Self-hosted agents are machines you own and manage — specified by name: 'MyPool' with optional demands. They provide private network access, persistent caching, and full tool control, but you handle OS patches, updates, and scaling. The pool can be set at the pipeline level (default for all jobs) or overridden per-job.

Intermediate

Q: Explain the three variable syntaxes in Azure Pipelines: macro, template expression, and runtime expression.▼

Macro syntax $(variableName) is the most common — it's expanded at runtime by the agent and works in task inputs and scripts. If the variable is undefined, the literal $(variableName) string remains. Template expressions ${{ variables.variableName }} are expanded at compile time before the pipeline runs. They're used in YAML structure: conditional insertion of blocks, template logic, each loops. They can access parameters and static variables but NOT output variables from previous jobs. Runtime expressions $[ variables.variableName ] are expanded during execution and are used in variable definitions and conditions. They can access all variables including output variables. The key distinction: compile-time = YAML structure manipulation; runtime = dynamic value resolution; macro = agent-level string replacement.

Q: How do you pass data between jobs in a YAML pipeline?▼

Jobs run on separate agents, so they don't share state. There are two ways to pass data: 1. Output variables: In Job A, a step sets an output variable with echo "##vso[task.setvariable variable=myVar;isOutput=true]myValue" (the step must have a name:). In Job B, you declare a dependency and map the variable: variables: { myVar: $[ dependencies.JobA.outputs['stepName.myVar'] ] }. 2. Pipeline artifacts: Job A publishes files with - publish: $(path). Job B downloads them with - download: current (for same pipeline) or - download: pipelineAlias (for cross-pipeline). Output variables are for small values (strings, flags). Artifacts are for files (binaries, configs, test results).

Q: What are pipeline parameters and how do they differ from variables?▼

Parameters are typed inputs defined at the top of a YAML file (or template). They support types: string, number, boolean, object, stepList. They're resolved at compile time and accessed via ${{ parameters.name }}. They can constrain values (via values: list) and appear as form fields in the manual run UI. Variables are key-value pairs resolved at runtime (macro/runtime) or compile time (template). They don't have type enforcement, can be set dynamically during execution, can come from variable groups or Key Vault, and can be secret. Key differences: parameters are typed and compile-time only; variables are untyped strings and can be dynamic. Use parameters for pipeline inputs (environment selection, feature flags). Use variables for build configuration, secrets, and dynamic values.

Q: What is the resources keyword used for?▼

The resources keyword declares external dependencies that the pipeline needs. It supports three resource types: 1. repositories — other Git repos (Azure Repos or GitHub) used for shared YAML templates, scripts, or source code. Declared with an alias and referenced with @alias. 2. pipelines — other Azure Pipelines whose artifacts you want to consume or whose completion should trigger this pipeline. Enables pipeline chaining. 3. containers — Docker images used as the runtime environment for container jobs (the job runs inside the container instead of directly on the agent VM). Each resource type requires a service connection for authentication if accessing external services (GitHub endpoint, ACR connection, etc.).

Q: How do conditions work in YAML pipelines? What happens if you override the default?▼

Conditions control whether a stage, job, or step executes. The default condition is succeeded() — the entity runs only if all previous dependencies succeeded. When you specify a custom condition:, you completely replace the default. This is a common mistake: writing condition: eq(variables['Build.SourceBranch'], 'refs/heads/main') means the stage runs on main even if the previous stage failed, because you removed the succeeded() check. The correct pattern is: condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main')). Conditions use expression functions: eq(), ne(), and(), or(), not(), startsWith(), contains(), succeeded(), failed(), always(), canceled(). Variables in conditions must use bracket notation: variables['name'].

Scenario-Based

Q: Your team has a monorepo with a backend (src/api/) and a frontend (src/web/). You want separate pipelines that only trigger when their respective code changes. How do you set this up?▼

Create two YAML pipelines with path-based triggers: Pipeline 1 — Backend (azure-pipelines-api.yml): Set trigger: { paths: { include: ['src/api/**', 'shared/**'] } }. This pipeline only fires when files under src/api/ or shared/ change. It builds and tests the API. Pipeline 2 — Frontend (azure-pipelines-web.yml): Set trigger: { paths: { include: ['src/web/**', 'shared/**'] } }. Both pipelines include shared/ because changes there could affect either project. In Azure DevOps, create two pipelines pointing to the same repo but different YAML files. Each pipeline runs independently — a commit touching only src/web/ triggers only the frontend pipeline. A commit touching shared/ triggers both. This saves agent time and gives focused feedback per team.

Q: You need to deploy the same application to five Azure regions sequentially, with a manual approval between each region. How do you design this in YAML?▼

Use multi-stage pipeline with environments and approval gates: Create five stages — Deploy-EastUS, Deploy-WestEurope, Deploy-SoutheastAsia, etc. Each stage uses a deployment job targeting a named environment (e.g., prod-eastus, prod-westeurope). In Azure DevOps, configure each environment with a manual approval check — an approver must click "Approve" before that stage executes. Chain stages with dependsOn for sequential execution. For the parameters approach, use an object parameter listing regions and a ${{ each }} loop to generate stages dynamically — this avoids copy-pasting five identical stages. The loop generates a stage per region, each targeting its own environment with approval configured.

Q: A developer reports that a pipeline variable set in one job is empty when accessed in a downstream job. What's the issue and how do you fix it?▼

Issue: Jobs run on separate agents with independent workspaces. A variable set with ##vso[task.setvariable variable=myVar]value only exists within that job. It does NOT automatically propagate to other jobs. Fix: The developer needs to: 1. Mark the variable as an output: echo "##vso[task.setvariable variable=myVar;isOutput=true]value". The step must have a name: property (e.g., name: setVarStep). 2. In the downstream job, declare a dependency and map the output variable using runtime expression syntax: dependsOn: JobA and variables: { myVar: $[ dependencies.JobA.outputs['setVarStep.myVar'] ] }. 3. Reference it normally: $(myVar) in scripts. The key insight: output variables require explicit wiring between jobs via the dependencies object.

Q: Your pipeline needs to build a .NET app on both Windows and Linux simultaneously, running the same steps. How do you avoid duplicating the entire job definition?▼

Use a matrix strategy or a template with parameters: Option 1 — Matrix strategy: Define a single job with strategy: { matrix: { linux: { vmImage: 'ubuntu-latest' }, windows: { vmImage: 'windows-latest' } } }. The job runs twice in parallel, once per matrix entry, with $(vmImage) substituted. All steps are defined once. Option 2 — Job template: Create a template file templates/build-job.yml with a parameters: [{ name: vmImage }] and the full job definition using ${{ parameters.vmImage }}. In the main pipeline, reference it twice: - template: templates/build-job.yml with parameters: { vmImage: 'ubuntu-latest' } and again with 'windows-latest'. Templates are better for complex jobs because they support typed parameters and conditional logic. Matrix is simpler for straightforward parallel runs.

Q: A production deployment pipeline suddenly starts failing with "Resource not authorized" after a colleague made changes. Nothing in the YAML changed. What happened?▼

Most likely causes: 1. Pipeline was renamed or re-created: If someone deleted and re-created the pipeline (even with the same YAML file), it gets a new pipeline ID. All resource authorizations (service connections, environments, variable groups, agent pools) are tied to the old pipeline ID and must be re-authorized. 2. New resource was added in Library: Someone may have updated a variable group or added a new service connection in the YAML that hasn't been authorized for this pipeline yet. 3. Security policy change: An admin may have changed the resource's security settings from "Grant access to all pipelines" to per-pipeline authorization. 4. Environment permissions reset: The target environment's approval and checks were modified, removing this pipeline's authorization. Fix: Open the failed run — the error shows which specific resource needs authorization. Click "Authorize" or ask a project admin to grant access in Project Settings → Service Connections/Environments → Security.

🌍 Real-World Use Case

SaaS Company Migrating 30 Classic Pipelines to YAML

Consider a mid-size SaaS company running 30 Classic (GUI-defined) pipelines built up over three years — a mix of CI builds for .NET microservices, npm-based frontends, and release pipelines deploying to Azure Kubernetes Service across dev, staging, and production environments.

The problems with Classic:

No version control: Pipeline changes were invisible to the team. Someone modified a production release pipeline via the GUI, and there was no PR, no review, no way to diff what changed. A deployment failed, and it took two days to realize someone had silently removed a health-check step.
No reuse: The same build logic (restore, build, test, publish) was manually configured in 20 separate pipelines. When the team upgraded from .NET 6 to .NET 8, they had to update all 20 pipelines one at a time through the GUI.
Branch-specific behavior: They needed release branches to deploy to staging with one set of config, and main to deploy to production with another. Classic pipelines don't live in the repo, so all branches share the same pipeline definition — there was no way to have branch-specific pipeline behavior without ugly workarounds.
Audit compliance: Their SOC 2 audit required evidence that pipeline changes were reviewed and approved. Classic pipelines had no change history beyond "last modified by."

The YAML migration strategy:

Created shared templates: Built a pipeline-templates repo containing reusable YAML templates: build-dotnet.yml, build-npm.yml, deploy-to-aks.yml. Each template accepted parameters (project path, .NET version, environment name, Helm values file).
Migrated one pipeline at a time: Started with a low-risk internal tool. Created the YAML file, ran it in parallel with the Classic pipeline for two weeks, compared outputs, then disabled the Classic version.
Used the Classic → YAML export: Azure DevOps has a "View YAML" button on Classic pipelines that generates a starting-point YAML file. It's not perfect (it produces verbose, unoptimized YAML) but it accelerates migration.
Standardized variables: Created variable groups in the Library for each environment (dev-vars, staging-vars, prod-vars) and linked them from YAML using - group: 'staging-vars' — eliminating hardcoded values.
Added branch-specific behavior: Used compile-time conditions to customize pipeline behavior per branch: ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/main') }} to add the production deploy stage only on main.

The result:

All 30 pipelines now live in code — every change goes through a PR with review and approval.
The shared template repo reduced total YAML from ~6000 lines to ~1200 lines. Upgrading .NET version is a one-line change in the template.
SOC 2 audit passed — pipeline changes have full Git history with author, reviewer, and timestamp.
New microservices get a CI/CD pipeline in 15 minutes by referencing the shared templates with project-specific parameters.

📝 Summary

Concept	Key Takeaway
trigger / pr / schedules	Control when pipelines run: push events, PR validation, cron schedules, or pipeline completion. Use `trigger: none` for manual-only.
pool	Specifies where jobs run: `vmImage` for Microsoft-hosted, `name` + `demands` for self-hosted. Can be set globally or per-job.
variables	Three syntaxes: macro `$()` (runtime, most common), template `${{ }}` (compile-time), runtime `$[ ]` (runtime, for conditions/dynamic values). Sources: inline, groups, templates.
stages → jobs → steps	The execution hierarchy. Stages are sequential by default. Jobs within a stage can be parallel. Steps are always sequential. Stages and jobs are optional for simple pipelines.
conditions	Use `succeeded()`, `failed()`, `always()`, `eq()`, `and()`, `or()`. Custom conditions replace the default — always include `succeeded()` if needed. Use bracket notation: `variables['name']`.
Compile-time vs runtime	`${{ }}` runs before pipeline starts — for YAML structure manipulation. `$[ ]` runs during execution — for dynamic values. Mixing them up causes silent bugs.
resources	Declare external dependencies: other repos (for shared templates), other pipelines (for triggers/artifacts), and container images (for container jobs).
parameters	Typed pipeline inputs (string, number, boolean, object, stepList). Resolved at compile time. Appear as form fields in manual run UI. Constrain with `values:` list.
Output variables	Pass data between jobs with `isOutput=true` and `dependencies.JobName.outputs['stepName.varName']`. Cannot use macro syntax across jobs.
Debugging	Enable `system.debug: true` for verbose logs. Validate YAML in the editor before committing. Check UI trigger overrides. Authorize resources explicitly.

← PreviousCI/CD Concepts & Pipeline Types Next →Classic Pipelines

← Back to Azure DevOps Course