4 min read

Switching between Kubernetes cluster contexts stored in Azure Key Vault

Switching between Kubernetes cluster contexts stored in Azure Key Vault

When I have a Kubeconfig that contains multiple environments I tend to get a little nervous. I usually need access to a couple of development / testing clusters and different customer clusters. My biggest concern is always that config file that contains all the contexts. Sure you can split them up into multiple files, create aliases and switch between them (how to) and even protect them with Azure Active Directory in the case of AKS. However, they're still on your file system.

And we can change that. I usually deal with Azure Kubernetes Service. So why not store those contexts in an Azure Key Vault and give them their own secret? "Why not" is exactly what I thought.

Requirements

This will work on Windows / PowerShell as well. Just keep in mind that the directory for kubeconfig on Windows is located in c:\users\username\.kube\ but can also be referenced to as "~/.kube". The biggest difference in these examples is the function to switch between contexts which needs to be a PowerShell function. Also, encoding and decoding Base64 in PowerShell works a bit different (check here)

Storing the Kubeconfig in Azure Key Vault

The Kubeconfig file contains JSON and even though the context of a single environment is within the size limitations for secrets, just storing the JSON directly as a secret in Azure Key Vault does some damage and when you retrieve it, it's pretty much rendered useless.

I decided to convert the JSON to Base64 first, store the Base64 content in Azure Key Vault and revert the process when I need it. Alright.. let's store the secrets in Key Vault.

I have an existing config stored in ~/.kube/config which is the result of az aks get-credentials. Additionally I have an Azure Key Vault called "kv-kubeconfigs". Let's store that config in Azure Key Vault.

cat ~/.kube/config |base64 > KubeConfigInBase64
az keyvault secret set --vault-name kv-kubeconfigs --name example-cluster-1 --disabled false --value @KubeConfigInBase64
rm KubeConfigInBase64

The break down:

  1. Convert the output to Base64
  2. Store the Base64 in "KubeConfigInBase64"
  3. Create the secret in Azure Key Vault with the name of my cluster (example-cluster-1)
  4. Add the contents of the "KubeConfigInBase64" file (--value @KubeConfigInBase64)
  5. Delete the Base64 file

This results in the secret being created. If we no longer need access to the cluster (for now) we can just delete the ~/.kube/config file. We can repeat this process over and over again and have many different contexts stored as secrets in Azure Key Vault.

Retrieving the configs

The next step is to retrieve the context again and store it in the default config file (~/.kube/config). I'm looking to have access to a single cluster at a time so no merging or whatsoever, I just want to overwrite.

We can do this the using the AzureCLI once more.

az keyvault secret show --vault-name kv-kubeconfigs --name example-cluster-1 --query value -o tsv |base64 --decode > ~/.kube/config

Let's break this down

  1. Get the secret from the Azure Key Vault for example-cluster-1
  2. Query the property "value" as this contains the actual Base64 encoded config
  3. "-o tsv" to get rid of those pesky quotes
  4. Decode the Base64 and output (store) it in ~/.kube/config

This should do it and we should now have access to example-cluster-1 or example-cluster-2 depending on what secret we request.

Great, that works!

Using a functions in bash to set and swap configs

We don't want to type the whole "az keyvault secret etc etc" command every time we create a new context or change contexts. Let's automate that using a function in bash.

Open up your .bashrc (code ~/.bashrc) and append the following functions to the end of the file:

Function of changing context

ksetcluster() 
{
    az keyvault secret show --vault-name kv-kubeconfigs --name "$1" --query value -o tsv |base64 --decode > ~/.kube/config
}

This will create a function called "ksetcluster" that accepts a parameter ($1). which will be the name of the secret you are creating.

Function to create a new secret in Azure Key Vault

We also need a function to create a secret in Azure Key Vault. The following function grabs the current ~/.kube/config contents and does the magic as described earlier in this post. So make sure you just have a single cluster context stored in there.

kstoresecret()
{
    cat ~/.kube/config |base64 > KubeConfigInBase64
    az keyvault secret set --vault-name kv-kubeconfigs --name "$1" --disabled false --value @KubeConfigInBase64
    rm KubeConfigInBase64
}

Save the file, restart bash (or source ~/.bashrc). And we should be good to go.

Now if you name your secrets logically then this function is really helpful. We can now change cluster context and overwrite our kubeconfig with a config stored in Azure Key Vault.

And we should also be able to create a new secret using "kstoresecret"

Success!

Wrap-up

There are many ways to store and change contexts of Kubernetes clusters. But this one in particular has multiple benefits for me:

  1. Only a single context is stored in my kubeconfig file
  2. There is Azure AD authentication required for the use of Azure Key Vault
  3. A secure way to share kubeconfigs with people I collaborate with (not everyone gets access to the subscriptions that clusters are running in)

If you have question or feedback, let me know!