CI Pipeline Builder

CI Pipeline — GitHub Actions · Python 3.11 / Poetry · AWS ECS Deploy

Generated for: payments-api · Repo: trunk-based · Cloud: AWS · Runtime: Python 3.11 / Poetry

Branch strategy: Trunk-based. main deploys to the staging ECS cluster; tagged v* releases deploy to production. Feature branches run build, lint, and test only — no deploy.

.github/workflows/ci.yml

# CI/CD Pipeline — Python 3.11 / Poetry / AWS ECS
# Stages: lint → test (matrix) → security-scan → build → deploy
# Auth: OIDC (no long-lived AWS keys). Secrets injected via GitHub Actions secrets.

name: CI

on:
  push:
    branches: [main]
    tags:    ["v*"]
  pull_request:
    branches: [main]

permissions:
  contents: read
  id-token: write   # required for OIDC → AWS

env:
  PYTHON_VERSION: "3.11"
  ECR_REGISTRY:  ${{ vars.ECR_REGISTRY }}        # org-level variable, not secret
  IMAGE_NAME:    my-api

jobs:
  # ── 1. Lint ────────────────────────────────────────────────────────────────
  lint:
    name: Lint & Format Check
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}

      - name: Cache Poetry virtualenv
        uses: actions/cache@v4
        with:
          path: ~/.cache/pypoetry/virtualenvs
          # Key rotates when pyproject.toml or lockfile changes
          key: poetry-${{ runner.os }}-${{ hashFiles('poetry.lock') }}
          restore-keys: poetry-${{ runner.os }}-

      - run: pip install poetry==1.8.2
      - run: poetry install --no-root

      - name: Ruff lint
        run: poetry run ruff check . --output-format=github

      - name: Black format check
        run: poetry run black --check .

      - name: MyPy type check
        run: poetry run mypy src/ --strict

  # ── 2. Test (matrix) ───────────────────────────────────────────────────────
  test:
    name: Test · Python ${{ matrix.python-version }}
    runs-on: ubuntu-22.04
    needs: lint
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12"]
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB:       testdb
        ports: ["5432:5432"]
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 10s --health-timeout 5s --health-retries 5

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Cache Poetry virtualenv
        uses: actions/cache@v4
        with:
          path: ~/.cache/pypoetry/virtualenvs
          key: poetry-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('poetry.lock') }}
          restore-keys: poetry-${{ runner.os }}-${{ matrix.python-version }}-

      - run: pip install poetry==1.8.2
      - run: poetry install

      - name: Run pytest with coverage
        env:
          DATABASE_URL: postgresql://postgres:test@localhost:5432/testdb
        run: |
          poetry run pytest tests/ \
            --cov=src \
            --cov-report=xml \
            --cov-fail-under=85 \
            -v

      - name: Upload coverage report
        uses: actions/upload-artifact@v4
        with:
          name: coverage-${{ matrix.python-version }}
          path: coverage.xml

  # ── 3. Security Scan ──────────────────────────────────────────────────────
  security:
    name: Security Scan
    runs-on: ubuntu-22.04
    needs: lint
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}

      - run: pip install poetry==1.8.2 && poetry install

      - name: Bandit SAST scan
        # No continue-on-error — security failures block the pipeline
        run: poetry run bandit -r src/ -ll -ii

      - name: pip-audit dependency CVE check
        run: poetry run pip-audit --strict

      - name: Trivy filesystem scan (secrets + misconfigs)
        uses: aquasecurity/trivy-action@6e7b7d1fd3e4fef0c5fa8cce1229c54b2c9bd0d8  # v0.19.0 SHA-pinned
        with:
          scan-type: fs
          scan-ref: .
          severity: HIGH,CRITICAL
          exit-code: "1"

  # ── 4. Build & Push Container ─────────────────────────────────────────────
  build:
    name: Build & Push Image
    runs-on: ubuntu-22.04
    needs: [test, security]
    # Only build on main or version tags — not on PRs
    if: github.event_name == 'push'
    outputs:
      image-tag: ${{ steps.meta.outputs.version }}

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502  # v4 SHA-pinned
        with:
          role-to-assume: ${{ secrets.AWS_OIDC_ROLE_ARN }}
          aws-region:     ${{ secrets.AWS_REGION }}

      - name: Log in to ECR
        uses: aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076  # v2 SHA-pinned

      - name: Docker metadata (tags & labels)
        id: meta
        uses: docker/metadata-action@902fa8ec7d6ecbea8a62e9785be63b4de3a6a38b  # v5 SHA-pinned
        with:
          images: ${{ env.ECR_REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=sha,prefix=sha-
            type=semver,pattern={{version}}
            type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}

      - name: Build and push
        uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75  # v6 SHA-pinned
        with:
          context: .
          push: true
          tags:   ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=registry,ref=${{ env.ECR_REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache
          cache-to:   type=registry,ref=${{ env.ECR_REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache,mode=max

  # ── 5a. Deploy → Staging ──────────────────────────────────────────────────
  deploy-staging:
    name: Deploy · Staging
    runs-on: ubuntu-22.04
    needs: build
    environment: staging
    if: github.ref == 'refs/heads/main'

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502
        with:
          role-to-assume: ${{ secrets.AWS_OIDC_ROLE_ARN_STAGING }}
          aws-region:     ${{ secrets.AWS_REGION }}

      - name: Update ECS service (staging)
        env:
          CLUSTER: my-api-staging
          SERVICE: my-api-svc
          IMAGE:   ${{ needs.build.outputs.image-tag }}
        run: |
          aws ecs update-service \
            --cluster  $CLUSTER \
            --service  $SERVICE \
            --force-new-deployment \
            --region   ${{ secrets.AWS_REGION }}

  # ── 5b. Deploy → Production ───────────────────────────────────────────────
  deploy-production:
    name: Deploy · Production
    runs-on: ubuntu-22.04
    needs: build
    environment: production          # requires manual approval gate in GitHub
    if: startsWith(github.ref, 'refs/tags/v')

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials (OIDC)
        uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502
        with:
          role-to-assume: ${{ secrets.AWS_OIDC_ROLE_ARN_PROD }}
          aws-region:     ${{ secrets.AWS_REGION }}

      - name: Update ECS service (production)
        env:
          CLUSTER: my-api-production
          SERVICE: my-api-svc
          IMAGE:   ${{ needs.build.outputs.image-tag }}
        run: |
          aws ecs update-service \
            --cluster  $CLUSTER \
            --service  $SERVICE \
            --force-new-deployment \
            --region   ${{ secrets.AWS_REGION }}

Secrets Reference Table

No long-lived AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY are used. All AWS auth flows through OIDC federated identity. See post-install instructions for trust policy setup.

Cache Keys Explanation

Branch / Tag Trigger Map

Gates Summary

Post-Install README Snippet

## Enabling the CI Pipeline

### 1 — Configure AWS OIDC trust

In your AWS account, create an IAM OIDC identity provider for GitHub:

- **Provider URL:** `https://token.actions.githubusercontent.com`
- **Audience:** `sts.amazonaws.com`

Create three IAM roles (one per scope: ECR push, staging ECS, production ECS) with a trust policy like:

```json
{
  "Effect": "Allow",
  "Principal": { "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/token.actions.githubusercontent.com" },
  "Action": "sts:AssumeRoleWithWebIdentity",
  "Condition": {
    "StringLike": { "token.actions.githubusercontent.com:sub": "repo:YOUR_ORG/YOUR_REPO:*" }
  }
}
```

### 2 — Add secrets to GitHub

Navigate to **Settings → Secrets and variables → Actions** and add:

| Name | Value |
|---|---|
| `AWS_OIDC_ROLE_ARN` | ARN of the ECR-push role |
| `AWS_OIDC_ROLE_ARN_STAGING` | ARN of the staging-deploy role |
| `AWS_OIDC_ROLE_ARN_PROD` | ARN of the production-deploy role |
| `AWS_REGION` | Your target region, e.g. `us-east-1` |

Set `AWS_OIDC_ROLE_ARN_STAGING` on the **`staging` environment**, and `AWS_OIDC_ROLE_ARN_PROD` on the **`production` environment**.

### 3 — Set the org-level variable

Under **Settings → Variables**, add `ECR_REGISTRY` (e.g. `123456789012.dkr.ecr.us-east-1.amazonaws.com`). This is not a secret — it contains no credentials.

### 4 — Enable environment protection rules

Go to **Settings → Environments → production** and enable **Required reviewers** to gate production deploys behind a manual approval step.

### 5 — Verify the pipeline

Push a commit to `main`. You should see five jobs run in order: **lint → test (×2 matrix) → security → build → deploy-staging**.

Coverage must be ≥ 85% and all Bandit/pip-audit/Trivy findings must be clean or the pipeline halts. Push a `v1.0.0` tag to trigger a production deploy (pending approval).

Generated by the ClearPoint Nexus CI Pipeline Builder skill. Validate the OIDC trust policy and ECR permissions in a staging AWS account before pointing the workflow at production.

This sample illustrates the skill's output format. Names, metrics, and operational details are illustrative unless the artifact explicitly analyzes public information.