CI/CD Concepts & Pipeline Types

Understand Continuous Integration, Continuous Delivery, and Continuous Deployment — the engine behind modern software delivery. Learn how Azure Pipelines executes work through agents, pools, jobs, and tasks, and compare YAML pipelines to Classic pipelines to choose the right approach for your team.

🧒 Simple Explanation (ELI5)

Imagine a toy car factory with a long assembly line. At one end, raw materials go in. At each station along the line, a worker does one thing — one cuts the metal, the next paints it, the next attaches the wheels, and the last one puts it in a box. The car moves automatically from station to station without anyone carrying it by hand.

Now imagine what happens if there's no assembly line. One person cuts the metal, walks across the factory, hands it to the painter, waits for the paint to dry, carries it to the wheel station… it's slow, error-prone, and someone eventually drops a car on the floor.

• CI (Continuous Integration) = every time a worker finishes a piece, it's immediately tested on the assembly line. If the part is crooked or the wrong color, the alarm goes off right away — before it reaches the next station. In software, CI means every code commit is automatically built and tested. Bugs are caught in minutes, not weeks.
• CD (Continuous Delivery) = the car goes all the way down the assembly line, passes every quality check, and arrives at the loading dock — boxed up and ready to ship. But a manager presses a button to actually load it on the truck. In software, the code is always in a deployable state, but a human approves the final push to production.
• CD (Continuous Deployment) = same assembly line, but there's no manager at the loading dock. The car rolls straight onto the truck automatically. In software, every change that passes all tests is deployed to production without human approval.
• Pipeline = the assembly line itself — the automated sequence of build, test, and deploy steps.
• Agent = the robot arm that does the actual work at each station. Azure gives you robots (Microsoft-hosted agents) or you bring your own (self-hosted agents).

🔧 Technical Explanation

CI vs CD: The Three Meanings

The acronym "CD" is frustratingly overloaded. Let's be precise:

Pipeline Execution Model in Azure DevOps

Azure Pipelines has a well-defined hierarchy for organizing and executing work:

Pipeline → Trigger → Stage → Job → Step → Task

• Pipeline: The top-level definition of your CI/CD workflow. Defined in a YAML file (e.g., azure-pipelines.yml) or via the Classic editor UI. A single pipeline can build, test, and deploy your application.
• Trigger: The event that starts the pipeline — a code push, a PR, a schedule (cron), or a manual run. Triggers can filter by branch, path, or tag.
• Stage: A logical boundary within the pipeline — e.g., Build, Test, Deploy-Staging, Deploy-Production. Stages run sequentially by default but can run in parallel. Each stage can have approval gates and conditions.
• Job: A unit of work that runs on a single agent. A stage contains one or more jobs. Jobs within a stage can run in parallel on different agents. Each job gets a fresh workspace.
• Step: A single operation within a job — either a task (a pre-built action from the marketplace) or a script (inline Bash/PowerShell). Steps run sequentially within a job.
• Task: A packaged, reusable unit of pipeline logic — e.g., DotNetCoreCLI@2 (build .NET), Docker@2 (build/push images), KubernetesManifest@0 (deploy to K8s). Tasks have inputs, outputs, and versioning.

Agents and Agent Pools

An agent is the compute that actually runs your pipeline jobs. It's a machine (or container) with the Azure Pipelines agent software installed. Agents are organized into pools.

Agent Capabilities

Each agent advertises capabilities — key-value pairs describing what the agent can do (e.g., node=18.x, docker=installed, Agent.OS=Linux). Pipelines specify demands — requirements that an agent must satisfy to run the job.

• System capabilities: Auto-detected by the agent — OS, installed tools, environment variables.
• User capabilities: Manually added by an admin — custom labels like GPU=true or region=us-east.
• Demands: Specified in the pipeline — demands: Agent.OS -equals Linux. The pipeline scheduler matches demands to capabilities to find a compatible agent.

YAML Pipelines vs Classic Pipelines

Azure DevOps offers two ways to define pipelines:

YAML Pipelines — Pipeline-as-Code. You write the entire pipeline definition in a .yml file stored in your repository alongside your application code.

# azure-pipelines.yml — YAML Pipeline example
trigger:
  branches:
    include: [main]
  paths:
    exclude: [docs/*, README.md]

pool:
  vmImage: 'ubuntu-latest'

stages:
- stage: Build
  jobs:
  - job: BuildApp
    steps:
    - task: DotNetCoreCLI@2
      displayName: 'Restore packages'
      inputs:
        command: restore
        projects: '**/*.csproj'

    - task: DotNetCoreCLI@2
      displayName: 'Build'
      inputs:
        command: build
        projects: '**/*.csproj'
        arguments: '--configuration Release'

    - task: DotNetCoreCLI@2
      displayName: 'Run tests'
      inputs:
        command: test
        projects: '**/*Tests.csproj'

    - task: PublishBuildArtifacts@1
      displayName: 'Publish artifact'
      inputs:
        pathToPublish: '$(Build.ArtifactStagingDirectory)'
        artifactName: 'drop'

- stage: DeployStaging
  dependsOn: Build
  jobs:
  - deployment: DeployToStaging
    environment: 'staging'
    strategy:
      runOnce:
        deploy:
          steps:
          - script: echo "Deploying to staging..."
            displayName: 'Deploy to Staging'

Classic Pipelines — GUI-based. You build the pipeline using a visual drag-and-drop editor in the Azure DevOps web portal. No YAML file needed.

• Created via Pipelines → New Pipeline → Use the classic editor
• Pipeline definition is stored in Azure DevOps (not in your repo)
• Tasks are added and configured through form fields and dropdowns
• Good for teams unfamiliar with YAML or for quick prototyping

📊 Pipeline Execution Model & Comparison

📋 YAML Pipelines vs Classic Pipelines

🖥️ Microsoft-Hosted vs Self-Hosted Agents

🛠️ Hands-on: Your First Pipeline

Follow these steps to create a YAML pipeline from scratch, run it, inspect the agent logs, and optionally configure a self-hosted agent.

Prerequisites

• An Azure DevOps project with a Git repository (created in Lesson 1)
• Basic familiarity with YAML syntax (indentation = spaces, not tabs)
• Contributor permissions on the repository

Step 1: Create a Minimal YAML Pipeline (Hello World)

In the root of your repo, create a file named azure-pipelines.yml:

# azure-pipelines.yml — Hello World pipeline
trigger:
  branches:
    include:
      - main

pool:
  vmImage: 'ubuntu-latest'

steps:
- script: echo "Hello, Azure Pipelines!"
  displayName: 'Run a one-line script'

- script: |
    echo "Pipeline ID:   $(Build.BuildId)"
    echo "Repository:    $(Build.Repository.Name)"
    echo "Branch:        $(Build.SourceBranchName)"
    echo "Commit:        $(Build.SourceVersion)"
    echo "Agent name:    $(Agent.Name)"
    echo "Agent OS:      $(Agent.OS)"
    echo "Working dir:   $(System.DefaultWorkingDirectory)"
  displayName: 'Print pipeline variables'

Commit and push to main:

git add azure-pipelines.yml
git commit -m "ci: add hello world pipeline"
git push origin main

Step 2: Create the Pipeline in Azure DevOps

1. Navigate to Pipelines → Pipelines → New Pipeline
2. Select your repository source (Azure Repos Git)
3. Select your repository
4. Choose "Existing Azure Pipelines YAML file"
5. Select branch main and path /azure-pipelines.yml
6. Click Run

Step 3: View the Run and Inspect Agent Logs

1. Navigate to Pipelines → Pipelines and click on your pipeline
2. Click the latest run to see the stages and jobs
3. Click the job (e.g. "Job") to expand the step-by-step log output
4. Click on each step to see its detailed log:
   • "Initialize job" — shows agent provisioning, capabilities, and variable resolution
   • "Checkout" — shows the Git clone operation (repo URL, branch, commit SHA)
   • "Run a one-line script" — shows your echo output
   • "Print pipeline variables" — shows the resolved system variables
   • "Post-job: Checkout" — cleanup step

Key things to look for in the agent logs:
──────────────────────────────────────────
✅ Agent name → e.g., "Hosted Agent" (Microsoft-hosted)
✅ Agent.OS → "Linux" (ubuntu-latest image)
✅ Agent.Version → the agent software version
✅ System.DefaultWorkingDirectory → /home/vsts/work/1/s
✅ Build.SourceBranchName → "main"
✅ Build.SourceVersion → the Git commit SHA
✅ Exit code of each step → 0 = success, non-zero = failure

To enable verbose debug logging for troubleshooting, add a pipeline variable:

variables:
  system.debug: true

Step 4: Create a Custom Agent Pool

1. Navigate to Organization Settings → Agent Pools
2. Click "Add pool"
3. Select Pool type: Self-hosted
4. Name it (e.g., my-linux-pool)
5. Optionally grant access to all pipelines
6. Click Create

Step 5: Configure a Self-Hosted Agent

Set up a self-hosted agent on a Linux VM (or WSL):

# 1. Create a PAT (Personal Access Token) in Azure DevOps
#    User Settings → Personal Access Tokens → New Token
#    Scope: Agent Pools (Read & Manage)

# 2. Download the agent
mkdir ~/myagent && cd ~/myagent
curl -fkSL -o vsts-agent-linux-x64.tar.gz \
  https://vstsagentpackage.azureedge.net/agent/3.248.0/vsts-agent-linux-x64-3.248.0.tar.gz
tar zxvf vsts-agent-linux-x64.tar.gz

# 3. Configure the agent
./config.sh \
  --url https://dev.azure.com/{your-organization} \
  --auth pat \
  --token {your-pat-token} \
  --pool my-linux-pool \
  --agent my-agent-01 \
  --acceptTeeEula

# 4. Run the agent interactively (for testing)
./run.sh

# 5. Install as a systemd service (for production)
sudo ./svc.sh install
sudo ./svc.sh start
sudo ./svc.sh status

Now update your pipeline to use the self-hosted pool:

# Use self-hosted agent pool instead of Microsoft-hosted
pool:
  name: 'my-linux-pool'
  demands:
    - Agent.OS -equals Linux

steps:
- script: |
    echo "Running on self-hosted agent: $(Agent.Name)"
    echo "Agent machine name: $(Agent.MachineName)"
    hostname
    uname -a
  displayName: 'Verify self-hosted agent'

🐛 Debugging Scenarios

Scenario 1: "No hosted parallelism has been purchased or granted"

Symptom: Your pipeline is queued and never starts. The log shows: No hosted parallelism has been purchased or granted. To request a free parallelism grant, please fill out the following form: https://aka.ms/azpipelines-parallelism-request.

Root Cause: Since February 2021, new Azure DevOps organizations (especially those using free Microsoft-hosted agents) do not get automatic free parallel jobs. Microsoft introduced this restriction to prevent crypto-mining abuse of free CI/CD resources.

Fix:

• Option 1 (Free): Fill out the parallelism request form at aka.ms/azpipelines-parallelism-request. You'll request a free tier grant for your organization. Approval takes 2-3 business days.
• Option 2 (Paid): Go to Organization Settings → Billing → Pipelines and purchase parallel jobs ($40/month for Microsoft-hosted or $15/month for self-hosted).
• Option 3 (Self-hosted): Configure a self-hosted agent (Step 5 above). Self-hosted agents get 1 free parallel job immediately — no request needed.

Check your current parallelism:
Organization Settings → Pipelines → Parallel jobs
─────────────────────────────────────────────────
Microsoft-hosted: X of Y parallel jobs (0 = not granted)
Self-hosted:      X of Y parallel jobs (1 free by default)

Scenario 2: "No agent found in pool matching specified demands"

Symptom: Your pipeline job shows "Waiting for an agent" indefinitely, or fails with "No agent found in pool 'my-pool' which satisfies the specified demands."

Root Causes & Fixes:

• Pool is empty: No agents registered in the pool. Go to Organization Settings → Agent Pools → [your pool] → Agents tab. If the list is empty, you need to install and configure an agent (Step 5 above).
• Agent is offline: The agent is registered but not running. Check the agent status in the Agents tab (green = online, red = offline). SSH into the machine and run sudo ./svc.sh status or restart with sudo ./svc.sh start.
• Demands mismatch: Your pipeline demands capabilities the agent doesn't have. Compare the pipeline's demands: section with the agent's capabilities in Agent Pools → [pool] → Agents → [agent] → Capabilities. Add missing user capabilities or adjust demands.
• All agents busy: All agents in the pool are running other jobs. You need more agents or more parallel jobs. Check the "Jobs" tab in the pool to see queued/running jobs.

# Check agent status on the self-hosted machine
cd ~/myagent
sudo ./svc.sh status

# If stopped, restart it
sudo ./svc.sh start

# Check agent logs for errors
cat _diag/Agent_*.log | tail -50

Scenario 3: "YAML file not found" / "Pipeline YAML file azure-pipelines.yml was not found"

Symptom: The pipeline fails immediately with an error about the YAML file not being found in the repository.

Root Causes & Fixes:

• Wrong file path: When creating the pipeline, you specified a path that doesn't match the actual file location. Go to Pipelines → [your pipeline] → Edit → ⋮ → Triggers → YAML tab and verify the YAML file path matches the actual file in your repo.
• File not on the trigger branch: The pipeline triggers on main, but you committed the YAML file to a feature branch. Merge/push the YAML file to main.
• Case sensitivity: Azure Repos on Linux agents is case-sensitive. Azure-Pipelines.yml ≠ azure-pipelines.yml. Verify the exact filename casing.
• File in a subdirectory: If the YAML file is at ci/azure-pipelines.yml, the pipeline configuration must point to /ci/azure-pipelines.yml, not /azure-pipelines.yml.

# Verify the file exists on the correct branch
git checkout main
ls -la azure-pipelines.yml

# If missed, create and push it
git add azure-pipelines.yml
git commit -m "ci: add pipeline YAML file"
git push origin main

Scenario 4: "Trigger not firing" — Pipeline doesn't run on push

Symptom: You push code to the repository, but the pipeline doesn't start. No run appears in the Pipelines section.

Root Causes & Fixes:

• Branch filter excludes your branch: If your trigger specifies include: [main] but you pushed to develop, the pipeline won't fire. Add the branch to the include list or use a wildcard:
  yamlCopy

  trigger:
    branches:
      include:
        - main
        - develop
        - feature/*

• Path filter excludes your changes: If you have path filters like exclude: [docs/*, README.md] and you only changed files in docs/, the pipeline won't trigger. Review the paths section in your trigger configuration.
• Pipeline disabled: Someone may have disabled the pipeline. Go to Pipelines → [your pipeline] → ⋮ → Settings and check if "Disabled" is toggled on.
• CI trigger overridden in UI: The Azure DevOps UI can override YAML triggers. Check Pipelines → [pipeline] → Edit → ⋮ → Triggers. If "Override the YAML continuous integration trigger" is checked, the UI settings take precedence over the YAML file.
• YAML syntax error in trigger section: A malformed trigger section can silently disable triggers. Validate your YAML with the pipeline editor's syntax checker.

🎯 Interview Questions

🌍 Real-World Use Case

A Fintech Startup's Journey from Manual Deploys to CI/CD

Consider a fintech startup building a payments API — 8 developers, a Node.js backend, a React frontend, deployments to Azure Kubernetes Service, and a regulatory requirement for audit trails on every production change.

The problem (before CI/CD):

• Deployments were manual — a senior developer SSH'd into the server, pulled code from Git, ran npm install and npm run build, and restarted the service. The process was undocumented and only two people knew how to do it.
• No automated tests — bugs were discovered by QA testers days after code was written. A typo in a payment calculation made it to production and caused a $12,000 overcharge that took 3 days to detect.
• Deployments happened once a month — a "big bang" release with 200+ commits. Debugging failures was painful because any of 200 changes could be the cause.
• No audit trail — regulators asked "who approved this production deployment and when?" and the answer was "we don't know, someone ran a script."

What they built with Azure Pipelines:

• CI pipeline (YAML): Triggers on every push to any branch. Runs npm install, npm run lint, npm test (200+ unit tests), and npm run build. Publishes a Docker image to Azure Container Registry with the Git SHA as the tag. Build time: 4 minutes.
• Multi-stage CD pipeline: Stage 1 (Dev) auto-deploys on push to main. Stage 2 (Staging) deploys after Dev succeeds — runs integration tests against a staging database. Stage 3 (Production) requires manual approval from the Tech Lead and a "Branch control" check that only allows main.
• Agent setup: Microsoft-hosted agents for CI (no private network needed). One self-hosted agent on a VM with VNet peering to the production AKS cluster for CD stages.
• PR validation: A separate pipeline triggers on PRs — runs lint, unit tests, and a security scan (OWASP dependency check). PRs can't merge unless this pipeline passes (branch policy).
• Audit trail: Every production deployment is linked to a work item (via AB# in commit messages), approved by a named person (environment approval), and logged with timestamps. Regulators get a report in minutes.

Results after 6 months:

• Deployment frequency: from once a month to 3-5 times per week
• Lead time for changes: from 3 weeks to under 2 days
• Bug escape rate: reduced 80% — automated tests catch regressions before code reaches staging
• Mean time to recovery (MTTR): from 4 hours to 15 minutes — automated rollback to last-known-good image
• Audit compliance: 100% — every production change has a traceable approval chain
• Developer satisfaction: no more "deployment Fridays" — anyone can deploy safely at any time

📝 Summary