Running the Azure Resource Manager Template Toolkit on GitHub Actions

I recently decided I wanted to learn more about GitHub actions. After reading some documentation I thought it'd be a good idea to have a project/goal to work with when going through the learning materials.

After going through the docs, I found that I needed some more context and stumbled upon the Pluralsight course "Buiding Custom Github Actions" by Enrico Campidoglio. It has helped me a great deal and is also a great reference when you need some context or an example of specific functionalities.

The Project

I've recently started playing with the Azure Resource Manger Template Toolkit (arm-ttk). If you're not familiar with it, do check it out. The ARM-TTK will help you test and analyze Azure Resource Manager Templates. If you need a quick getting started post. Check out this post by Sam Cogan.

My idea was to run the toolkit using GitHub actions once I committed a new ARM Template to the repository. Something that should be possible (and it is!). My first question was: which kind of action? I had two options: Writing a JavaScript action or a Docker container action. There is something to be said for both of them but in general the following characteristics define them:

• JavaScript actions run directly on GitHub hosted runners and execute really fast. However, it is limited to binaries already available on the runner machine. You should not depend on binaries that exist outside of the virtual environment
• Docker container actions require the container image to be build (or pulled) and then run. This slows down the action significantly, but as we're talking about containers here, you can do pretty much whatever you want.

I chose to go with a Docker container action. This will allow me to run the same setup outside of GitHub actions (if I have a use for that). Running Docker Containers also has it's limitations but more on that later.

TL;DR: Repository and code here: https://github.com/whaakman/armttk-github-action-demo

Dockerfile

I started with setting up a Dockerfile that I could build and test locally. In the end it should do/contain the following:

• Run an Image with PowerShell installed
• Git
• Azure Resource Manager Template Toolkit
• The ARM Templates from the repository
• A PowerShell file to kick off the logic

I also thought it would be a good idea to only execute PowerShell logic once the container is up and running. Any installation that needed to be done should be ready before that happens (install git, cloning the repositories). Keeping it separated. And that way, once the container configured in it's very basic form, all I had to do was focus on PowerShell.

Dockerfile:

FROM mcr.microsoft.com/powershell:latest

# Labels for GitHub Actions
LABEL "com.github.actions.name"="ARM TTK"
LABEL "com.github.actions.description"="Checks template with the ARM TTK"

LABEL version="0.0.1"
LABEL maintainer="Wesley Haakman" 

# Install Git
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y git

# Clone ARM-TTK Repo
RUN git clone https://github.com/Azure/arm-ttk.git
RUN git clone https://github.com/whaakman/armttk-github-action-demo.git /temp

# Copy the Entrypoint file from the current directory (the one the Dockerfile is located in)
COPY entrypoint.ps1 /entrypoint.ps1

# Start using the Entrypoint file.
ENTRYPOINT [ "pwsh", "/entrypoint.ps1" ]

What we're doing here is pull the image from mcr.microsoft.com/powershell:latest. Which contains an Ubuntu image with PowerShell pre-installed.  

The next step is to install Git by running apt-get and then clone the clone the repository where the ARM-TTK modules are located (there isn't a module available in the PowerShell gallery yet). We also clone the repository that contains the ARM Templates. In this case this is also the repository our action is stored in. Next we configure the Entrypoint and the Dockerfile is finished.

In this example the "entrypoint.ps1" needs to be stored in the same directory within your repository as the Dockerfile (COPY entrypoint.ps1 /entrypoint.ps1).

Note: GitHub actions currently doesn't support passing build args for Docker. Once this is supported it would make sense to use the GitHub Actions environment variables and pass the repository name / URI to clone. This will help scale to different repositories.

Entrypoint file

The PowerShell file and the comments are pretty self explanatory but let's quickly run through it.

Firstly we'll import the arm-ttk module which was cloned into the container (import-module) and run the tests. Note the $TemplatePath = '/temp/' as this is the same directory we cloned our repository during the building of the container image. As the ARM-TTK supports testing all files in a directory (and subdirectories), this also supports having multiple ARM templates stored in a repository. This also means you don't want to accidently set the template path to the root of the system (/ for example) if you want your tests to finish somewhere this week and not be confronted with a bunch of errors.

We're interested in the failed results (if it succeeds, I don't necessarily want all the logging returned. I did this by storing the failures in $TestFailures by reusing the sample from the GitHub repo itself (https://github.com/Azure/arm-ttk/tree/master/arm-ttk).

Now it's a matter of checking whether $TestFailures is indeed being populated by the results of failed tests or if everything is successful and it remains empty. If the tests are not successful, we'll output the results and exit with code "1" telling the GitHub Action something didn't go right. If the tests were successful we do the opposite and exit with exit code "1". This could do with some more logic and actually checking the contents of $TestFailures would be the improvement on the top of the list (to double check we're not providing an exit code 0 when something else happened other than a fail or success). But for a first test, it will do.

entrypoint.ps1

# Import the arm-ttk module we cloned in the dockerfile
Import-Module '/arm-ttk/arm-ttk/arm-ttk.psd1'

# Path we cloned the repository into
$TemplatePath = '/temp/'

$TestResults = Test-AzTemplate -TemplatePath $TemplatePath
# We only want to return failures
$TestFailures =  $TestResults | Where-Object { -not $_.Passed }


# If files are returning invalid configurations
# Using exit code "1" to let Github actions node the test failed
if ($TestFailures) {
    Write-Host "One or more templates did not pass the selected tests:"
    $TestFailures.file.name | select-object -unique
    Write-Host "Results:"
    Write-Output $TestFailures
    exit 1
} 

# Else, all passes and exit using exit code 0 indicating a success
else {
    Write-Host "All files passed!"
    exit 0
}

Having both the Dockerfile and the entrypoint.ps1 set up, we can build and run the container locally and it should run the tests. To test (make sure you have Docker installed locally): run the following from the directory the Dockerfile is living in:

docker image build -t armttktest:1.0 .
docker run -it armttktest:1.0

And.. Result! the azurdeploy.json in the repository (as stored in the Dockerfile) was tested and is using an incorrect API version.

Now that we know the logic is working, it's time to get an action up and running. We'll do this by configuring a workflow.

Workflow file

Workflows can do many things but I'm only using a subset of those capabilities to keep it simple. If you're advancing into building more complicated workflows (maybe deploy some Azure Resources when the ARM Templates pass the test), then reading up on https://help.github.com/en/actions/configuring-and-managing-workflows is recommended. Workflows are stored in the .github/workflows directory. I chose to store my Dockerfile and Entrypoint file in .github/actions, tho the naming of that directory can be anything you like as long as you refer to it. For example .github/actionfiles will also work.

Time for some YAML magic.

The workflow needs a name, which we call "ARM Tests". Additionally we want the workflow to run whenever we push something to the master branch.

Each workflow needs at least one job which we configure using "jobs:". In this example I created a job "test_arm_templates". Next we need to configure the runner. I want my workflow to run on a GitHub hosted runner using Ubuntu. You have some options to chose from tho (https://help.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idruns-on).

Next you need to configure the steps, which are the actual actions. My Workflow consists of two steps:

1. Checkout the repository to the runner
2. Build and run the Docker image

The Checkout action is a public Github Action we can reference. What it does is checkout the contents of your repository to make it available to the workflow. We need that because we need the workflow to access the Dockerfile in the next step.

The next step is to actually run the Dockerfile, much like we did locally. All that is required is point to the directory containing the file. One could argue that it would be a best practice to write an action.yml to kick of the Docker step (and I would agree) but having a Dockerfile in the directory is sufficient and keeps it simple for now :)

Note that I'm also adding an "id" to the step as in a different example I'm using this id to grab output.

Main.yml

# Name the workflow
name: ARM Tests
# Run when code is pushed to the master branch
on:
 push:
  branches:
    - master

jobs:
  test_arm_templates:
    runs-on: ubuntu-latest
    name: Run ARM TTK
    steps:
      # Action that checks out the code to the runner
      - name: Checkout
        uses: actions/checkout@v2
      # Action that runs the container
      - name: Run ARM TTK
        uses: ./.github/actions
        id: action

Last but not least, I want a status badge because well.. It just looks cool. You can add the badge in your Readme by pointing to the badge.svg using markdown like so:

Make sure you have your directories and files set up correctly and then commit everything to your GitHub repository. Starting from the root of my repository this is what my setup looks like:

│   azuredeploy.json
│   README.md
│
└───.github
   ├───actions
   │       Dockerfile
   │       entrypoint.ps1
   │
   └───workflows
       └───ARM Tests
               main.yml

The ARM Template I'm using is pretty basic one that creates a storage account. In this example the test would fail as we're using an API older than 2 years. Just like the example we ran using just Docker earlier.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "location": {
           "type": "string",
           "metadata": {
                "description": "description"
            }
        }
    },
    "variables": {},
    "resources": [
        {
            "name": "storageaccount1",
            "type": "Microsoft.Storage/storageAccounts",
            "apiVersion": "2015-06-15",
            "location": "[parameters('location')]",
            "tags": {
                "displayName": "storageaccount1"
            },
            "properties": {
                "accountType": "Standard_LRS"
            }
        }
    ],
    "outputs": {},
    "functions": []
}

Running the Workflow

Once everything is committed, the workflow will run as we've configured it to run whenever something is committed to the master branch.

By browsing to your repository and click "Actions" on the top you should see something similar to the following.

From the local test we know that the workflow should report a failure as the API version used for the storage account is too old. Upon inspecting the workflow that just ran we can see that the Checkout step went well, but the "Run ARM TTK" step experienced an issue.

Digging into this we can see that the test did fail because of the API version:

That means we need to fix it and commit a better template to the repository. And success! The next run succeeds.

What's next?

The next steps would be to expand the workflow. Maybe trigger an actual deployment or.. just run this as a check for all your templates. A very logical next step would be to add additional tests to the ARM-TTK and either clone them from a different repository (or do a PR on the repo itself!).

Either way, hopefully this will help you with your initial setup. The documentation and Pluralsight videos helped me a great deal but what works best is to actually have a use case to work on when learning new technologies :)