Before You Begin

  • A GitHub repository containing at least two microservices (e.g., perth-api, adelaide-worker)
  • Docker and Docker Buildx installed locally for testing
  • An AWS account (or alternative cloud) with ECR and ECS configured, or a self-hosted SSH server for deployment
  • GitHub Environments (staging, production) set up in repository settings
  • OIDC integration for AWS (IAM role with trust policy) to avoid storing long-lived credentials

1. Repository Structure That Demands a Real Pipeline

Organize your monorepo to allow each service to be built independently. A working layout:

infra/
  terraform/                 # Infrastructure as code (optional)
services/
  perth-api/
    Dockerfile
    src/
    package.json
  adelaide-worker/
    Dockerfile
    src/
    requirements.txt
  darwin-gateway/
    Dockerfile
    Cargo.toml
.github/
  workflows/
    ci-cd.yml
    lint.yml
docker-compose.yml          # For local development only
.dockerignore

Building each service separately avoids coupling and lets us use matrix builds in GitHub Actions.

2. The Dockerfile That Scales Across Architectures

A production Dockerfile must be architecture-aware. Use Buildx for multi-platform builds. Here's a real example for perth-api (Node.js):

# Stage 1: Build with platform-specific dependencies
FROM node:20-alpine AS builder
ARG TARGETPLATFORM
ARG BUILDPLATFORM
RUN echo "Building on $BUILDPLATFORM for $TARGETPLATFORM"
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && npm prune --production

# Stage 2: Production runtime โ€“ non-root user
FROM node:20-alpine
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
WORKDIR /app
USER appuser
COPY --chown=appuser:appgroup --from=builder /app/package*.json ./
COPY --chown=appuser:appgroup --from=builder /app/node_modules ./node_modules
COPY --chown=appuser:appgroup --from=builder /app/dist ./dist

ENV NODE_ENV=production
EXPOSE 3000
CMD ["node", "dist/main.js"]

Key decisions: TARGETPLATFORM is injected by Buildx, allowing conditional dependency installation (e.g., native binaries). The non-root user (appuser) is mandatory โ€“ many developers on StackOverflow forget this and introduce security holes.

3. The CI/CD Workflow โ€“ Multi-Arch, Cached, Gated

Create .github/workflows/ci-cd.yml. This is a single workflow that handles build, test, push, and deployment for all services.

name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  workflow_dispatch:
    inputs:
      service:
        description: 'Service to deploy (all or specific)'
        required: true
        default: 'all'

env:
  REGISTRY: ${{ secrets.AWS_ECR_REGISTRY }}

jobs:
  build:
    strategy:
      matrix:
        service: [perth-api, adelaide-worker, darwin-gateway]
        arch: [amd64, arm64]
    runs-on: ubuntu-latest
    outputs:
      image-tag: ${{ steps.tag.outputs.tag }}
    steps:
      - uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          platforms: linux/amd64,linux/arm64

      - name: Configure AWS credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/GitHubActionsOIDC
          aws-region: ap-southeast-2

      - name: Login to Amazon ECR
        uses: aws-actions/amazon-ecr-login@v2

      - name: Generate semantic version tag
        id: tag
        run: |
          echo "tag=$(date +'%Y%m%d%H%M')-$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT

      - name: Build and push multi-arch image
        uses: docker/build-push-action@v6
        with:
          context: services/${{ matrix.service }}
          file: services/${{ matrix.service }}/Dockerfile
          platforms: linux/${{ matrix.arch }}
          tags: |
            ${{ env.REGISTRY }}/${{ matrix.service }}:${{ steps.tag.outputs.tag }}
            ${{ env.REGISTRY }}/${{ matrix.service }}:latest
          cache-from: type=gha,scope=${{ matrix.service }}
          cache-to: type=gha,mode=max,scope=${{ matrix.service }}
          provenance: false

Explanation for engineers rejecting standard tutorials

  • Matrix service ร— arch: Each service is built separately for both architectures, then pushed as a manifest list โ€“ a pattern seen in production Go repositories on GitHub like moby/buildkit.
  • GitHub Actions cache (type=gha): This serializes cache across runs, dramatically speeding rebuilds. Many teams skip this and wait 10 minutes per service.
  • OIDC instead of secrets: Using AWS IAM with a trust policy removes the risk of secret rotation. Official documentation recommends this over long-lived access keys.
  • Semver via date+sha: Not true semver, but fast and unique. For full semver, integrate git-semver using tags.

4. Deployment โ€“ Blue-Green with Approval Gates

After images are pushed, deploy to staging automatically, then require manual approval for production.

deploy-staging:
  needs: [build]
  runs-on: ubuntu-latest
  environment: staging
  steps:
    - name: Update ECS service (blue-green)
      run: |
        aws ecs update-service \
          --cluster cloud-infra-cluster \
          --service ${{ matrix.service }} \
          --force-new-deployment \
          --deployment-configuration '{"deploymentCircuitBreaker":{"enable":true,"rollback":true}}'

Deploy to production only after an environment approval from a senior engineer:

deploy-production:
  needs: [deploy-staging]
  runs-on: ubuntu-latest
  environment: production
  steps:
    - uses: actions/github-script@v7
      with:
        script: |
          github.rest.repos.createDeployment({
            owner: context.repo.owner,
            repo: context.repo.repo,
            ref: context.sha,
            environment: 'production',
            auto_merge: false,
            required_contexts: []
          })

5. Rollback Strategy โ€“ When Sydney Goes Down

Preparation: keep the previous image tag in a GitHub Actions variable or deploy a shell script on failure. Add a rollback job:

rollback:
  if: failure()
  runs-on: ubuntu-latest
  steps:
    - run: |
        PREVIOUS_TAG=$(cat /tmp/last-good-tag)
        aws ecs update-service --cluster cloud-infra-cluster --service perth-api --image $REGISTRY/perth-api:$PREVIOUS_TAG

Store the tag in an artifact from the build job. This is a common pattern in enterprise DevOps blogs.

Sponsored Deal

6. Common Issues and Resolutions

  • Multi-arch build fails on ARM emulation: Use QEMU binfmt support; install it via docker/setup-qemu-action@v3 before buildx action.
  • Cache not saving to GitHub: Ensure GITHUB_TOKEN has write permissions for packages. Set actions: write in the permissions block.
  • OIDC role trust policy malformed: The sub claim must match your GitHub org/repo exactly. AWS documentation includes a sample trust policy for GitHub Actions.
  • ECS deployment circuit breaker triggers falsely: Set minimumHealthyPercent to 100 temporarily for blue-green. The official AWS SDK recommends using CodeDeploy for full control.

7. Next Steps โ€“ Observability and Policy as Code

After the pipeline is solid, add:

  • OpenTelemetry trace injection in the Docker build (layers for instrumented images)
  • Conftest or OPA policies to reject images with critical CVEs before deployment
  • Scheduled container image scanning with Trivy in a separate workflow