top of page

GitHub Reusable Workflows: Keep your workflows DRY

  • Writer: Damian Gitto Olguin
    Damian Gitto Olguin
  • Aug 17, 2022
  • 4 min read

Updated: Sep 8


""

It is common in an organizational environment to have multiple applications built with the same technologies or frameworks, and even to have them share a common CI/CD pipeline. This, most of the time, ends up in having to repeat a lot of code to, for example, deploy a Terraform infrastructure, upload a container image to a registry, and other similar jobs. For this reason, to keep your pipelines DRY (Don’t Repeat Yourself), we bring you GitHub Reusable Workflows.


What a normal workflow looks like


Here we have an example of how a normal workflow to upload a container image to ECR looks:


build_n_push:
    name: Build and Push to ECR
    runs-on: ubuntu-latest
    outputs:
      image_tag: ${{ steps.set-image-tag.outputs.image_tag }}
      latest_tag: ${{ steps.set-latest-tag.outputs.latest_tag }}

    steps:
      - name: Checkout code
        uses: actions/checkout@v2
        with:
          submodules: true
          fetch-depth: 0

      - name: Set Image tag
        id: set-image-tag
        shell: bash
        run: |
          echo "::set-output name=image_tag::$(echo ${{ github.sha }} | cut -c1-12)"

      - name: Set LATEST tag
        id: set-latest-tag
        shell: bash
        run: |
          if [ ${{ github.ref }}  == 'refs/heads/develop' ]; then echo "::set-output name=latest_tag::latest"; else echo "::set-output name=latest_tag::latest-staging"; fi

      - name: Configure AWS Credentials
        id: configure-credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Login to Amazon ECR
        id: login-container-registry
        uses: aws-actions/amazon-ecr-login@v1

      - name: Build and Push Container Image
        id: build-container-image
        env:
          ECR_REGISTRY: ${{ steps.login-container-registry.outputs.registry }}
          ECR_REPOSITORY: my_ecr_repo
        run: |
          docker build . \
            --progress plain \
            --file Dockerfile \
            --tag $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-image-tag.outputs.image_tag }}
          docker tag \
            $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-image-tag.outputs.image_tag }} \
            $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-latest-tag.outputs.latest_tag }}
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-image-tag.outputs.image_tag }}
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:${{ steps.set-latest-tag.outputs.latest_tag }}

Now imagine having to copy this workflow on each one of your applications (even you will need to copy your deployment workflows too!). This is a lot of tedious and repetitive work, also this involves a lot of human interaction, therefore making our pipelines more prone to errors.


It would be great to have all this standard code in a place where we can maintain it more easily and that we can use from other repos in our organization. Here is where Reusable Workflows come in handy.


How to make a workflow “reusable”

It is easy to create a reusable workflow for your GitHub Actions CI/CD pipeline, but we have to keep in mind some limitations related to it before starting:


  • Reusable workflows can’t call other reusable workflows

  • You can’t call reusable workflows in a private repository unless you are in the same repository.

  • Environment variables aren’t propagated from the caller workflow to the called workflow (don’t worry, we can solve this with inputs)

So, now we can define our first reusable workflow:

name: Build and push a Docker image to ECR
on:
  workflow_call:
    inputs:
      ECR_REPO:
        required: true
        type: string
      AWS_REGION:
        required: false
        type: string
        default: "us-east-1"

    outputs:
      image_tag:
        description: Image tag created on the workflow
        value: ${{ jobs.build_n_push.outputs.image_tag }}
      latest_tag:
        description: Latest tag created on the workflow
        value: ${{ jobs.build_n_push.outputs.latest_tag }}

    secrets:
      AWS_ACCESS_KEY_ID:
        required: true
      AWS_SECRET_ACCESS_KEY:
        required: true


jobs:

As you can see, with just adding a few things we can create a reusable workflow. We have the following parameters:

  • on.workflow_call: this tells Github that this workflow will be triggered via call of another workflow.

  • inputs: this is non sensitive data that we can pass to our workflow, in this case we are passing the ECR Registry and the AWS Region.

  • outputs: this is data that our workflow will output to other jobs in the caller workflow. In this example we are setting as output the image tag and the latest tag for that image (we can chain this output to our deploy workflow 😉 )

  • secrets: sensitive data that will be used by our workflow.

After all these parameters, all we have to do is to define our workflow like a normal one, specifying all the jobs that will run along with their steps.


How can I use this workflow?

To make use of our new reusable workflow first of all you can have it on the same repo of your application, or a better solution would be to have this workflow on a public repository of your organization. This makes your workflow publicly accessible unless you do a trick I will teach you in the end.


We have two ways to invoke our workflow, the first one if it is on the same repo:

name: Continuous Deployment

on:
  push:
    branches:
      - main
      - develop

jobs:
  build_and_push_image:
    name: Build container image
    uses: .github/workflows/docker_build_and_push.yaml@master
    with:
      AWS_REGION: us-east-1
      ECR_REPO: my-repo
    secrets:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

And now if we have our public repo with the workflow:


name: Continuous Deployment

on:
  push:
    branches:
      - main
      - develop

jobs:
  build_and_push_image:
    name: Build container image
    uses: our-org/our-workflow-repo/.github/workflows/docker_build_and_push.yaml@master
    with:
      AWS_REGION: us-east-1
      APP_NAME: my-repo
    secrets: inherit

About the inherit, you can use it when both your repos have access to the same secrets, and they have the same name in each repo (like using organizational secrets).



BONUS TRACK: Private reusable workflows

Now… maybe you want to have this workflow on a public repository so you can call it from all your applications repositories but you may not like that everyone else can call your workflow. Well there is a simple trick to ensuring only your organization can call your workflow, you can use the following template:


jobs:
  check_org:
    name: Check Caller
    runs-on: ubuntu-latest

    steps:
      - name: Check the calling organization
        if: ${{ github.repository_owner != 'my-org'' }}
        uses: actions/github-script@v3
        with:
          script: |
            core.setFailed('This reusable workflow can only be used by My Org.')
  my_normal_job:
    needs: check_org
    name: This is my normal job
    runs-on: ubuntu-latest
    steps:
         #....

As you can see, we are using a GitHub context variable called repository_owner. This ensures that if the calling repo isn’t from our organization, it can’t use the workflow (at least from our public repo). Remember to add the needs: check_org to your actual job.


Beyond automation, teams can also reduce cloud costs with AWS Spot Instances and Terraform automation, combining workflow efficiency with financial optimization.



""




Juan Wiggenhauser

DevOps Teracloud







bottom of page