My First Azure OpenAI Assistant

AI and Machine learning always felt like something out of reach because of its complexity. But, things have changed. In the past years I have already invested time in learning about Machine Learning, training models and plenty of IoT but I had never progressed into the realm of building automation, agents and productivity. For me it was always a "wow that's cool, next project!" kind of thing.

With the developments in AI and the rate at which new features are being released I decided to pick things up where I left off and reintroduced myself into the world of AI and it is becoming more and more exciting every day. Especially as I am always geeking out about new Cloud Native things, AI sparks my interest. They go together so very well 😄

Azure OpenAI Assistants

For me, the regular ChatGPT like solutions always came with some limitations. They are very powerful when generating new content based on the perfectly sculpted prompt but interacting with technologies "outside of the sandbox" was always problematic or when provided "out of the box" pretty dangerous.

But then there are Azure OpenAI Assistants. That allow us to do things such as: use Logic Apps. What now? Yes we can connect a Logic App and our Azure OpenAI Assistant can trigger it, gather its response and come back to you. Additionally, we can allow the Assistant to use the built-in code interpreter to run scripts we provide in a multitude of languages. Maybe I'm just over the moon excited for use cases this unlocks and everyone may think I am overreacting, but I'm a sucker for these kinds of developments.

Then there is another benefit here. The documentation, the getting started examples, the content, it's very good. We can get started pretty easily and Azure AI Services come with a Playground, perfect for us to get started.

Before we do so we need to have a Logic App that does something for us. For me, I went went a Logic App that can return the number of Resource Groups I have in my subscriptions. I choose to start small and this only requires Reader level access to my subscription. I have no desire to automatically create or delete resources based on my prompts... for now. But the goal is: Interact with my Azure Subscription through prompts.

Logic App Setup

My Logic App Setup is pretty straight forward. Do note that to leverage an Azure Logic App from An Azure OpenAI Assistant, the Logic App must be deployed as a consumption plan and it most provide a REST endpoint. Luckily for basic things, we already default to the consumption plan and the HTTP Request trigger exists out of the box. In my case the Logic App has the following requirements:

• Consumption Plan
• Managed Identity with Reader permissions on the subscriptions
• HTTP Request Trigger
• Action to List Resource Groups in my subscription
• HTTP Response that returns the number of Resource Groups in my Subscription.

Deploying an Azure Logic App is as straightforward as doing what the Azure Portal tells you to do, or deploy pre-existing template. You can use the following guide(s) to deploy your Logic App: https://learn.microsoft.com/en-us/azure/logic-apps/quickstart-create-example-consumption-workflow

Once deployed it's time to configure. First we will configure the Managed Identity.

On the starting blade of the Logic App, navigate to "Identity" and toggle the switch for the System assigned identity to "On". Give it a couple of seconds and then configure the Azure role assignments.

Click on "Azure role assignments" and on the new blade on "Add role assignment (preview). Configure your identity to have reader permissions on a subscription scope.

Permissions are there, time to set up the Logic App. We will do this through the Logic App Designer.

First we need to configure our Trigger. As Azure OpenAI Assistants require a REST Endpoint, we will go with the trigger "When a HTTP request is received".

We configure a default body so that the Azure OpenAI Assistant picks this up we are provided some context. Note that the HTTP URL will become visible / is generated when you save the Logic App.

In case you want to copy/paste:

{
    "type": "object",
    "properties": {
        "ResourceGroups": {
            "description": "number of resource groups",
            "type": "string"
        }
    }
}

Next we will add an action to list our Resource Groups. We do so by searching for "Azure Resource Manager" and adding the "List Resource Groups" action to your Logic App:

We need to scope it to our Subscription and provide the action with the authorization to leverage the Managed Identity. We do so by configuring the connection to use "Managed identity" as it's Authentication type, by setting the Managed Identity to "System-assigned managed identity" (as we created one before), the Logic App will automatically pick this up and use that identity for any Azure Resource Manager actions.

Next we need to Parse the JSON as we want to return a number, not a series of JSON objects.

And in case you want to copy/paste:

{
    "type": "object",
    "properties": {
        "value": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "id": {
                        "type": "string"
                    },
                    "name": {
                        "type": "string"
                    },
                    "location": {
                        "type": "string"
                    },
                    "properties": {
                        "type": "object",
                        "properties": {
                            "provisioningState": {
                                "type": "string"
                            }
                        }
                    },
                    "tags": {
                        "type": "object",
                        "properties": {}
                    }
                },
                "required": [
                    "id",
                    "name",
                    "location",
                    "properties"
                ]
            }
        }
    }
}

And finally, we need to return a response. This may not be something we do with Logic Apps often (most of Logic Apps I see are to automate Ops processes and are fire and forget), but the assistant needs a response. We do so by adding an action "Response".

We can leave the defaults but as a Body we will return the number of Resource Groups in our Subscription. We can do so by looking at the JSON from the previous step, and counting the values:

length(body('Parse_JSON')?['value'])

Now its time to save our Logic App and deploy our Azure AI Assistant. Don't forget to copy the URL of the HTTP trigger once the Logic App is saved!

Azure OpenAI Assistant Playground

Time to configure our Assistant. Start with deploying the Azure AI Services. Get start through here: https://learn.microsoft.com/en-us/azure/ai-services/openai/assistants-quickstart?tabs=command-line%2Cjavascript-keyless%2Ctypescript-keyless&pivots=programming-language-ai-studio.

Once we have our Playground we can open our newly created assistant and add our Logic App. We do so by click on "Add Function" at the bottom left of our Assistant Interface.

After selecting the "Logic Apps" tab we should see our function and be able to add it.

We add our Logic App by selecting it and pressing "Save". If we inspect our added Function we can see the Function specification. That looks familiar!

It's time to put this to the test. As good quality assurance would have it, you already tested your Logic App by trigger it from your local machine to ensure all that logic works (I know I have!). But If you haven't, the bigger the "wow" will be when we test our assistant. Let's give it a try.

Using the system prompt we tell it that it is allowed to invoke functions. We're then asking it; How many resource groups do I have in my subscription? And with success!

If we inspect the logs we can also see that the content did come from our Azure Logic App:

And we barely wrote any code!

Wrapping up

That was really easy to get started. In fact, it barely required any code to be written. Image taking this further, writing code, automating processes, building this out into something bigger. The possibilities are endless and I am very interested about what is to come. I expect to write plenty more posts regarding use cases with Azure AI Services. Exciting times ahead!