Orchestrating a Microservice Deployment Pipeline with Ansible Playbooks

Before You Begin

• Ansible 2.16+ installed on a control node (macOS/Linux/Windows WSL)
• Two target machines (VMs or containers) for staging and production environments, accessible via SSH with key-based auth
• Docker and Git installed on target machines
• Basic familiarity with YAML and Jinja2 templating
• A container registry accessible from both environments (e.g., Docker Hub or a private registry)

1. Project Structure – The Apple Pipeline

We’ll automate a fictional microservice called apple-auth (a token validation service). The deployment pipeline must:

• Build a Docker image and push it to a registry
• Deploy the image to a staging server, run smoke tests
• Promote the same image to production with a rolling update
• Provide a rollback mechanism if health checks fail

Create a project layout that follows community patterns from GitHub (e.g., ansible-examples):

apple-deploy/
├── ansible.cfg
├── inventory/
│   ├── staging.yml
│   └── production.yml
├── group_vars/
│   ├── all.yml
│   ├── staging.yml
│   └── production.yml
├── roles/
│   ├── build_push/
│   │   └── tasks/main.yml
│   ├── deploy/
│   │   └── tasks/main.yml
│   └── health_check/
│       └── tasks/main.yml
├── playbooks/
│   ├── pipeline.yml
│   └── rollback.yml
└── secrets/
    └── vault.yml (encrypted)

2. Inventory & Group Variables – Multi-Environment with Fruit Flavors

Define two environments. Use fruit-themed group names to keep the example memorable but not confusing:

inventory/staging.yml

all:
  children:
    mango_env:
      hosts:
        staging-server-1:
          ansible_host: 192.168.1.101

inventory/production.yml

all:
  children:
    banana_env:
      hosts:
        prod-server-1:
          ansible_host: 192.168.1.201
        prod-server-2:
          ansible_host: 192.168.1.202

group_vars/all.yml – placeholders that feel natural:

registry_url: "registry.example.com"
apple_image: "{{ registry_url }}/apple-auth:{{ apple_version }}"
apple_version: ""  # overridden at runtime

group_vars/staging.yml – staging-specific tolerances:

health_check_retries: 5
health_check_delay: 10

group_vars/production.yml – more conservative:

health_check_retries: 10
health_check_delay: 15
rolling_update_max_failure: 1

3. The Pipeline Playbook – Build, Test, Deploy, Verify

This playbook orchestrates the entire life cycle. It uses the dockerimage and dockercontainer modules (official Ansible docker modules – see docs).

playbooks/pipeline.yml

---
- name: Build and push apple-auth image
  hosts: localhost
  vars:
    apple_version: "{{ lookup('env', 'CI_COMMIT_SHORT_SHA') | default('latest', true) }}"
  roles:
    - build_push

- name: Deploy to staging & smoke test
  hosts: mango_env
  roles:
    - deploy
    - health_check

- name: Promote to production with rolling update
  hosts: banana_env
  serial: 1  # one server at a time
  roles:
    - deploy
    - health_check

A common pattern in production Ansible codebases on GitLab is to pass the git commit SHA as the image tag. The serial: 1 directive ensures we update one production node at a time, reducing blast radius.

4. Role Breakdown – Building, Deploying, Healing

build_push/tasks/main.yml

Use multi-stage Docker build as recommended by the official Docker documentation. The task below runs a local build and pushes to the registry:

---
- name: Build app-alpine image
  community.docker.docker_image:
    name: "{{ apple_image }}"
    source: build
    build:
      path: "{{ playbook_dir }}/../app"
      dockerfile: Dockerfile
    push: yes
  register: build_result

- name: Ensure build succeeded
  fail:
    msg: "Image build/push failed"
  when: build_result is changed

deploy/tasks/main.yml

This role replaces the running container. Use docker_container with state: started and restart: yes for a clean swap. For a rolling update, rely on Ansible’s serial at the playbook level, not within the role.

---
- name: Pull latest image
  community.docker.docker_image:
    name: "{{ apple_image }}"
    source: pull

- name: Remove existing container
  community.docker.docker_container:
    name: apple-auth
    state: absent

- name: Start new container
  community.docker.docker_container:
    name: apple-auth
    image: "{{ apple_image }}"
    state: started
    restart: yes
    ports:
      - "3000:3000"
    env:
      NODE_ENV: "{{ env_name }}"

Note: Frequent StackOverflow questions revolve around docker_container not restarting properly without restart: yes. Always set it when you want the container to re-read environment variables or image changes.

health_check/tasks/main.yml

Use the uri module to hit the /healthz endpoint:

---
- name: Wait for container to be healthy
  uri:
    url: "http://localhost:3000/healthz"
    status_code: 200
    return_content: yes
  register: health
  until: health.status == 200 and 'healthy' in health.content
  retries: "{{ health_check_retries }}"
  delay: "{{ health_check_delay }}"

- name: Fail if health check times out
  fail:
    msg: "Service unhealthy after deployment"
  when: health is failed

5. Rolling Back When Bananas Go Bad

Rollback playbook (playbooks/rollback.yml) uses the previous image tag stored as a fact. A common approach seen in engineering blogs by companies like Etsy is to keep the last two tags in group_vars as a list and script a fallback.

---
- hosts: banana_env
  serial: 1
  vars:
    apple_version: "{{ previous_apple_version | default('v0.1.0') }}"
  roles:
    - deploy
    - health_check

Store previousappleversion in a file or use ansible-vault to encrypt it per environment. To decrypt safely during CI, use --vault-password-file from a secure store (e.g., GitLab CI/CD variables).

6. Secrets Management – No Plain Fruit Salads

Use ansible-vault for sensitive variables (e.g., registry credentials, database passwords).

ansible-vault create secrets/vault.yml

Inside vault.yml:

registry_password: "s3cr3t"

Then in group_vars/all.yml reference the vault file:

vault_registry_password: "{{ vault_registry_password }}"  # from vault

Never hardcode credentials. The official Ansible documentation warns against it.

7. Common Pitfalls and How to Avoid Them

• Docker module not found: Ensure community.docker collection is installed (ansible-galaxy collection install community.docker). Many StackOverflow threads point to this missing step.
• Serial execution ignoring: If your deploy role includes a wait_for or loop, serial applies only to the host loop, not internal tasks. Use throttle for fine-grained control.
• Image tag collisions: Use unique identifiers (commit SHA + timestamp) instead of latest. A pattern from GitHub Actions is to set apple_version: "{{ lookup('pipe', 'git rev-parse --short HEAD') }}".
• SSH connection timeouts: Increase ansiblesshtimeout in ansible.cfg if your build tasks take longer than 10 seconds.

Your pipeline is now ready to be triggered from a CI webhook or via ansible-playbook -i inventory/staging.yml playbooks/pipeline.yml -e apple_version=2.1.0. This intermediate approach bridges the gap between simple ad-hoc commands and full orchestration with Kubernetes.