Power Automate on Hat Full of Data

Trigger Microsoft Fabric Data Pipeline using Power Automate

Sun, 06 Oct 2024 14:18:24 +0000

Microsoft Fabric Data Pipelines can be scheduled to happen at regular times. What happens when you want trigger Microsoft Fabric data pipelines to run as a reaction to an event? For example someone pressing a button in an app. For this we can use Power Automate to start a pipeline using a http request.

Fixed

This post has incorrect details for the connector. These have now been fixed.

YouTube Version

Workspace and Pipeline GUIDs

The flow is going to use a http action that uses a URL that includes GUIDs for both the workspace and the data pipeline. Open the data pipeline that will be triggered. Look at the URL in the browser. The Workspace GUID is the string after /groups/ and the data pipeline GUID is the string after /pipelines/.

Start Flow and Add Variables

Now we have the required GUIDS we can start writing the flow. For this example we are going to use a manually triggered flow. Then we add an actions to initialise variables WorkspaceID and PipelineID and select Type of string and put in the values from the previous step.

URL of the Trigger Microsoft Fabric Data Pipeline request

Next I recommend you use a compose step to calculate the URL for the HTTP call. Interestingly the api uses workspaces rather than groups. Make sure you have the corrects /s and no spaces between. If you have called your variables WorkspaceID and PipelineID you can just copy the code below into a Compose action.

1

https://api.fabric.microsoft.com/v1/workspaces/@{variables('WorkspaceID')}/items/@{variables('PipelineID')}/jobs/instances?jobType=Pipeline

The above url including all the flags is documented by Microsoft here – https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api#run-on-demand-item-job

HTTP Request

For this action we are going to use HTTP with Microsoft Entra ID (preauthorized) connection’s Invoke an HTTP request action. Search for HTTP and scroll down till you find it.

If this is the first time you’ve used this action you will need to setup the connection to connect to this api you will need to set it up. The two urls required are below, sometimes they appear the other way around.

Base Resource URL

1

https://api.fabric.microsoft.com/

Microsoft Entra ID Resource URI

1

https://analysis.windows.net/powerbi/api

Once you have the connection sorted you can enter in the details of the request. The Method is POST and the Url of the request is the output from the compose step.

Testing Trigger Microsoft Fabric Data Pipeline Flow

The flow requires no inputs so can just be run. When the flow runs it appears to fail, but if you check the data pipeline run history you will see it was triggered just fine.

If you look in the documentation found at https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api#run-on-demand-item-job. There is a note that says “There is no body returned currently”. Eventually it will return the job id of the pipeline run, so I am guessing because that is not returned Power Automate assumes it failed.

Conclusion

Data pipelines are the orchestration tool of Microsoft Fabric so being able to connect it to the Power Platform orchestration tool makes perfect sense. I look forward to the day I can mark this post as redundant and there is a Power Automate connector with an obvious action for this.

Power Automate – Save a File to OneLake Lakehouse

Thu, 03 Oct 2024 14:48:09 +0000

Currently there is no Microsoft Fabric connector in Power Automate. When we want Power Automate to save a file to OneLake Lakehouse, we need to use a workaround. This post walks through the demo I did for the Microsoft Fabric European Conference 2024. You need to be aware that this uses a premium connector!

YouTube Version

Getting the File Content

Before we can save a file anywhere we need the file content we are going to save. For this post I am going to work with the flow triggering from a new file in a SharePoint library and getting the file content action.

From the SharePoint connector triggers, select the When a file is created (properties only) and populate the site address and library name. Next add a SharePoint action, Get file content. Populate the Site Address same as the trigger. For the File Identifier, select Identifier from the dynamic content from the trigger.

Lakehouse Folder URL

Next we need to know the url of the Lakehouse folder where we want the file to go. This can be found in the Lakehouse by clicking on the three dots next to the required folder and from the menu selecting properties. This will open the properties pane on the right hand side. Click the copy button next to the URL.

To make life easy for ourselves we put this into a compose step followed by name of the file that triggered the flow. This is in the dynamic content from the trigger and is called File name with extension. Please note there is a / before the file name.

Create the file in the LakeHouse

Set up the connection

We are going to use the HTTP with Microsoft Entra ID (preauthorised) action to interact with the Lakehouse to create the file. When you add the new action search for http and then scroll down to find the right action.

If this is the first time you’ve used this action you will need to set-up the connection. Also if this is the first time you’ve connected to the Lakehouse in this way you need to setup a new connection. It asks for two URLS. Once connected you are ready to save a File to OneLake Lakehouse

1

https://onelake.dfs.fabric.microsoft.com/

1

https://storage.azure.com/

Create Blank File

The first stage is to save a file to OneLake Lakehouse that is empty. This is a simple PUT action with a parameter of ?resource=file after the path created in the compose. So select PUT for the method. In the URL of the request but the Output from the compose and add on ?resource=file. I also renamed the action.

Append Content

Finally the last action is to append the file content from SharePoint onto the file. Change the Method to PATCH. The Url of the request is the output from the compose followed by three parameters (copy from below). Expand the Advanced parameters and into the body of the request select File Content from the dynamic content.

1

?action=append&position=0&flush=true

Test Save a File to OneLake Lakehouse

Save your flow and add a file to your SharePoint library and wait a minute or 2 for the flow to trigger. A file should appear in your Lake House.

Conclusion

There are plenty of business cases of use cases to be able to store a file into OneLake. Various legacy systems email daily exports which will need to be saved and processed in the LakeHouse. When / If we get a Fabric connector I look forward to making this post as redundant!

DevOps updates on behalf of another with Power Automate

Mon, 19 Aug 2024 09:34:27 +0000

I had a need to use automation to do DevOps updates on behalf of another person so when the DevOps adoption metrics were done we the stats were slightly more accurate. This can only be done using REST API and will need permissions. This post is part of the DevOps and Power Automate series.

DevOps with Power Automate posts

YouTube Version

It’s on the backlog!

Permission required to makes updates on behalf of another

The trick to being able update on behalf of another is to update the ChangedBy field to another persons email address. This is not normally allowed, it breaks the rules. So the permission we need is to bypass the rules and make illegal updates.

Permissions are managed in Project Settings, you then click on permissions. Click on Users at the top and then click on the user being used in the flow to show their permissions. Under Boards check the permission for Bypass rules on work item updates. This needs to be set to Allow.

Make Updates on behalf of another

Now we have permission is ignore the rules we can test out a flow. In the REST API documentation for updating an item https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/work-items/update you will see there is a boolean parameter bypassRules. This can be added to the relative URI using & to separate the parameters.

In the body section I have added an update to the ChangedBy field. This update updates the State to active and performs it on behalf of Dan. In the task history we can see the update and see I did it on behalf of Dan.

Word of caution

By passing the rules, it also means that the updates behind the scenes that happen are turned off. If we change the state to closed using the same technique the we get an error when we open the work item.

So make sure you have checked the updates you are doing and include all the fields you need to update, even the hidden ones.

Second word of warning is no checks are done on the updates. So you can assign a task to an email address that doesn’t have access, or update to a state that is not valid. So treat this one with caution.

Conclusion

Updates on behalf of another is a very powerful trick. But should be treated with extra caution and it has made me double check permissions on a ew projects so access to by passing the rules is limited.

Update DevOps without Notifications with Power Automate

Fri, 16 Aug 2024 07:52:49 +0000

When a work item is assigned to you, by default you are sent a notification. Notifications can also be sent regarding updates to your items. Now imagine creating over 10 thousand tasks automatically and assigning them to people. I recently did this on a projects and potentially each person was going to get 100s of emails. So I needed Power Automate to assign the tasks without notifications. This post is part of the Power Automate and DevOps series.

DevOps with Power Automate posts

YouTube Version

They are on the backlog!

Permissions for updates without notifications

Updating DevOps without notifications is not something that everyone should be able to do. Or at least the admins will want to be able to control who can do it.

In the Project settings, click on Permissions. Then click on Users to list your users. When you click on a person you get to see all their permissions. They can be Allow, Deny or Not Set for all the different actions. (inherited) means it is set by being a member of a group. If you hover your cursor over the (i) you will get a description telling you which membership the permission as inherited from. A Deny overrides an allow, so you might need to manually override the inherited permission.

The permission we are interested in is Suppress notifications for work item updates.

Updating without Notifications

Now we have the permission we can test it out in an update. This can only be done on updates done by a REST API call. If you look on https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/work-items/update there is a URI parameter suppressNotifications which is a boolean.

URI parameters get added after the ? and are separated by &. So the code to update item 36 without notifications would be

1

{project}/_apis/wit/workitems/36?suppressNotifications=true&api-version=7.0

Item History

Suppressing the notifications does prevent automated updates filling inboxes with emails that will just get deleted. It does not however supress the history of the item being updated. So who did the update and when is still recorded.

Running a Wiql DevOps query in Power Automate

Wed, 14 Aug 2024 11:55:17 +0000

Wiql stands for Work Item Query Language and is the language used to define the queries in Azure DevOps. Its has a very similar syntax to SQL. Power Automate can use a REST API send a query to DevOps to execute the query. This post post is part of the Power Automate and DevOps series.

DevOps with Power Automate posts

The full reference can be found at https://learn.microsoft.com/en-us/azure/devops/boards/queries/wiql-syntax?view=azure-devops

Writing the Query

You can write it by hand, and I’m sure some people do. I use the query tools in DevOps to build the query and then view the Wiql code. For this post the example we are going to is to assign all the items in a hierarchy to one person. For example all the items in the Design Shiny feature.

Wiql Editor

Before we can work with WIQL we need to install the extension Wiql Playground in the organisation. You will need to be an organisation admin to do this. Head to the marketplace for Visual Studio, found here

Extensions for Visual Studio family of products | Visual Studio Marketplace

Search for wiql and click on the Wiql Editor. When the extension information page opens click on Get it free. Then on the next page, select the right organisation and click Download. It will now appear under boards in your project.

Construct Query

Now in Queries construct the query. I am not here to teach you building queries in DevOps, there is a great post to get you started here https://learn.microsoft.com/en-us/azure/devops/boards/queries/using-queries. My query is a tree of work items, matching the top item first and only has 2 criteria of the top item is ID 17 and the child item is not closed or completed.

Once you have your query working and you have saved it you can view the Wiql. Click on the three dots in the top right. Then from the menu select Edit query wiql. This will open a pane containing the code that you can copy to your clipboard.

Note you have to save after any changes for the changes to be in the Wiql code.

Running the Wiql in Power Automate

Now we have the Wiql code we can write the flow. The flow will have the parameters of the Parent Work Item ID and the email address to assign all the tasks to. We will then use a compose action to the Parent Work Item ID and Project name into the Wiql.

The next step is to run the query. The documentation for the REST API call can be found here https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/wiql/query-by-wiql The method is Post and the URI will be the same for all Wiql

1

_apis/wit/wiql?api-version=7.1-preview.2

The body is JSON with only one field query with the output from the compose action.

When we run the flow and enter in the Parent ID 17 and the email Dan@lgb123.onmicrosoft.com it executes and returns us the data.

Handling the Output of the HTTP call

In the successful action we can click on Show raw output in the Outputs section to open a pane and explore the Json output. Scroll past the headers and then in the body section, past columns to find workItemRelations. This will list all the work items returned by the query, not their details, just the id and url in the target section.

To make this easy to work with we need to use a Parse JSON so copy all the output JSON and return to editing the flow. Add a Parse JSON step and add the Body returned by the HTTP to the Content. Next we need to add a schema, this can be generated. Click on Generate from sample. Then paste in the JSON you copied and click Done. And a schema will be added.

The output schema is now known to the flow. So we can use the output to loop through all the items and update the work items.

Looping through workItemRelations

We can loop through the workItemRelations using an Apply to each action. The piece of information I want from each item is the target – id. So I add a compose and add the following expression

1

item()?['target']?['id']

I would then test to make sure you get the number of items you expect and that the compose does return the ids you expect

Finally we can add a step to assign each task to the email given in the parameter. I use a Send HTTP method rather than the Update a Work Item so I can as explained in a future post turn off notifications.

We can then run the flow and check the results in DevOps and see the tasks have been assigned.

Conclusion

this was one example of using Wiql inside Power Automate. I have used it to duplicate hierarchies and delete a hierarchy. The skill comes in being able to write the Wiql to achieve what you need.

Using DevOps REST API in Power Automate

Tue, 13 Aug 2024 16:37:38 +0000

DevOps REST API is a well documented API that we can easily use in Power Automate to perform a huge range of actions. The connector includes an action to make it easy. The API documentation can be found at

https://learn.microsoft.com/en-us/rest/api/azure/devops

In this post we will take some simple examples to get you started. This is the third post in the DevOps and Power Automate series.

DevOps with Power Automate posts

YouTube Version

Its coming honest!

Get Task Details using DevOps REST API

Power Automate includes an action to get the details of a Work Item. When we look we can see it has 4 required fields. Organization name, Project Name and Work Item Id are easy. We will have them. I had problems with Work Item Type, I didn’t always have that. So I looked to DevOps REST API for an alternative.

If we look in the documentation and then search for Get Work Item, you will find the syntax for fetching the details of one work item. The URL needs three pieces of information, shown in the {} brackets, organization, project and id. So no Work item type is required.

https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/work-items/get-work-item

We start with an instant flow. Then we add the a parameter to the trigger of the work item id. Then we add the action Send an HTTP request to Azure DevOps. The Organisation we can use our environment variable (or from the drop down). The method is GET. The Relative URI is everything after the organisation in the url we got from the documentation.

1

{project}/_apis/wit/workitems/{id}?api-version=7.2-preview.3

When we run the flow, we get prompted to enter the number of a work item id. When it completes, we get returned the JSON of that work item details. If you need to extract information from that JSON, you will probably need a Parse JSON action. (Another post!)

Update a Work Item using DevOps REST API

We can update an item using the REST API. For this example the flow will update a given task to have the state Active and be assigned to the given email. So, we add Work Item ID and Email as two parameters to the trigger.

Then we add a Send an HTTP request to Azure DevOps action. The Organisation is the same as before, the method is now Patch as we are updating data. The Relative URI is exactly the same as before. In the headers we need to add Content-Type and application/json-patch+json

In the Body we need to add a JSON object that includes the fields we want to update. For this we need to know the field names, see the previous post in the series for ways to find them. It is just text so we can insert dynamic content, e.g. the email address. The above example picture and code below shows changing the State to Active and assigning it to the Email address

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


[
 {
 "op": "add",
 "path": "/fields/System.State",
 "value": "Active"
 },
{
 "op": "add",
 "path": "/fields/System.AssignedTo",
 "value": "@{triggerBody()['text_1']}"
 }
]

Conclusion

Rest API is a very powerful way to work with Azure DevOps and the action in Power Automate does make it very easy to use. It is worth having a test project to try out some of the options. Other posts will look at using Rest API to make changes without notifications and on behalf of another person. Also a bulk delete process, which needs a post of its own due to a few oddities.

Power Automate Update Fields in Azure DevOps

Mon, 29 Jul 2024 08:09:46 +0000

In a previous post we created items in DevOps. Although the Power Automate create item action had lots of fields, it did not have all the fields. This post shows you how to find the names for those fields and update fields in Azure DevOps using Power Automate.

DevOps with Power Automate posts

YouTube Version

Its coming 🙂

The Problem

If you compare the fields available in a standard Epic item in DevOps with the fields available in Power Automate create work item action you will notice some fields are missing. The obvious missing ones being Start Date and Target Date and other fields like Business Value etc and any custom fields you’ve added will not be there so we need to update fields in Azure DevOps.

The observant ones amongst you will have noticed the Other Fields option in the Power Automate action. It wants a key for your field which is the name of the field. And guess what that name is not going to be StartDate. So our first job is finding that name.

Finding Field Names v1

This is the first method and I think the simplest method. Start by creating an item in DevOps that has values in the fields you would like the names of. I’ve populated Effort, Business Value, Time Criticality, Start Date and Target Date.

Then in a Power Automate instant flow I add an action to get the work item details of that item, notice it is number 21. In the Get work item details I populate Org1, Project, Type=”Epic” and Id=21. Then I run that flow and look at the outputs from that step.

If you scroll through the fields you will eventually find Business Value etc with their full field names. You need to replace the _ with . as I assume JSON objects to dots in field names. Note down the ones you need.

Create Work Item and Update Fields in Azure DevOps

Now we have the field names we can create a work item and populate those fields. Note the date formats are YYYY-MM-DDThh:mm:ssZ.

When this flow is run, it creates an item and will successfully update fields in Azure DevOps.

The Update a work item action has exactly the same Other fields option so you can use the same technique.

Find Field Names v2

Azure DevOps can be accessed via a Rest API2 and this includes the list of fields available in a project. There is quite good documentation for this which can be found at https://learn.microsoft.com/en-us/rest/api/azure/devops/wit/fields/list . From the first code block on that post remove the GET and populate your org and project names. Then in a browser navigate to that url and it will return lots of JSON showing the definition of all the fields.

1

https://dev.azure.com/{organization}/{project}/_apis/wit/fields

Then search for the fields you want and look for the referenceName and use that. As my project got complex I saved this data and then used Power Query to make a simple list of fields (perhaps a resource I should make public).

Conclusion on Update Fields in Azure DevOps

The Other Fields section in Create and Update work item actions gives us the option to update items however we want to. Some fields of course are read only and updating some fields will update other fields, for example if you change the state to Closed it will then populate the Closed date field.

Connecting Power Automate to DevOps

Thu, 25 Jul 2024 10:54:26 +0000

I know Azure DevOps from being on projects. Mostly for tracking tasks, sprints etc. I recently had to automate some item creation and editing across 1000s of tasks in DevOps. So I had to explore connecting Power Automate to DevOps for item creation and item editing. This is the first post I have in a long list of posts regarding DevOps with Power Automate and Power BI, welcome to my way of remembering how I did something.

DevOps with Power Automate posts

YouTube Version

DevOps Connector

While editing a flow if you add action and search for DevOps you will find the Azure DevOps connector. If you look at the actions in the classic format1 you will notice its a premium connector, i.e. you need a premium license. If you have not used this connector before it will prompt you to login. It assumes your DevOps login matches your Power Automate login.

Create an item

Let’s start with a simple action to create a work item in a project. We start by adding the action Create a work item.

The Organisation Name and the Project Name drop downs will contain the organisations and projects your login can see. (See further down about making this solution friendly). The Work Item Type drop down will contain all the types that the project allows from its process. The title is plain text and the Description is rich text, i.e. html.

When you test the flow you should get a work item created that matches the above.

Troubleshoot Connecting Power Automate to DevOps

If the flow fails with an error regarding authorisation, you need to first check you are allowed to add an item in that project in DevOps. Then you need to check the organisation settings in DevOps. Look in Policies and make sure Third-party application access via OAuth is turned on.

Advanced Options in Create an Item

The Create a work item action includes advanced options, really they are just optional options. Two of the options are Link URL and Link Type. In this example I used the URL from the previously created item and the Type of Hierarchy-reverse to make this item a child of the previous item.

Another option is the Assigned To field. I added my email so that when I look in the created item I see that it has been assigned to me.

The first option is Return all fields, which the default is blank. This means the action returns all the item details when it successfully creates the item.

Making it Solution Aware

If this flow is part of a solution that is going to move from a development environment to test and production, you will need to point the DevOps actions to the different Organisation and Project values. Thankfully the text values of the Organisation Name, e.g. HatFullOfData can be used.

So I create 2 environment variables DevOps Organisation and DevOps Project. Then in the create a work item action, in Organisation Name and Project Name drop downs I select Custom Value and then the right value from Dynamic content.

And yes of course after making these changes I test the flow again.

Conclusion on Connecting Power Automate to DevOps

For when you need to automate the creation of items, this is great. It does not replace importing an Excel file to create a whole plan but it does give an extra set of options to consider.

Power Automate – Create Power BI Workspace

Fri, 21 Jun 2024 20:18:44 +0000

We’ve setup the service principal, saved the details securely in Azure Key Vault and used Power Automate to fetch them securely and finally setup the permissions to allow that service principal to use the rest api. This post will finally create the flow to automatically create Power BI workspace making use of all the preparation.

This post is part of Power Automate and Power BI Rest API series.

Start the flow and get service principal details

There will be an instant flow using a manual trigger. Once the flow editor appears, expand the trigger and add a text input for the Workspace Name.

The first action will be to call the child flow that fetches the tenantid, clientid and secret. Select Get Client Tenant and Secret flow from the dropdown and then use PBI-Workspace as the prefix value.

HTTP Call to automatically create Power BI Workspace

FInally we can do the action to create Power BI workspace, for this we add a HTTP action. I’m going to break this down into 3 parts, the URI, the body and the authentication.

From https://learn.microsoft.com/en-us/rest/api/power-bi/groups/create-group#examples we can see the HTTP starts with POST so we pick that as the method. It also gives us the URI. To save you going to fetch it here it is

1

https://api.powerbi.com/v1.0/myorg/groups?workspaceV2=True

The next part is the body. This is a very simple piece of JSON with only one field name. Get the value of name to come from the trigger input Workspace name.

1
2
3


{
 "name": ""
}

The final part is the authentication. Show the Advanced Options to see the boxes to fill in. Firstly Authentication is Active Directory OAuth. Then the Authority can be left blank. Next the Tenant, Client ID and Secret are dynamic values from running the child flow action. Finally the Audience should be the following

1

https://analysis.windows.net/powerbi/api

Your flow is now ready to test. Hopefully when you do this it all works and you get green ticks. But when you go to Power BI the workspace is not listed in workspaces you can see. This is because the workspace was created by the service account, so the service account is the admin, not you. So the next step is to add a workspace admin.

Add Workspace Admin

This is going to require another HTTP call. So if we look at the second example show at https://learn.microsoft.com/en-us/rest/api/power-bi/groups/update-group-user#examples we can see we need 2 pieces of information, the workspace ID and the email address of the workspace admin.

The workspace ID must have been returned when we create Power BI workspace. If you look in the dynamic content it won’t be listed though so we need either code by hand or we use my favourite trick of Parse JSON.

Get the Workspace ID

Go to the test run you did and expand the HTTP call that created the workspace. In the Outputs section there is a Body section which will contain the JSON that was returned. Copy this JSON as this will help complete the Parse JSON.

Then return to editing the flow and add a Parse JSON action. For the content use the Body returned by the HTTP action. JSON schemas are tricky to write by hand so you can generate it from what we copied. Click on the Generate from sample to open a dialog. Paste the output body we copied earlier and click Done. The Parse JSON step will now have a schema.

Request the Admin Email

The flow might not be run by the requester of the workspace, so it makes sense to ask for the admin’s email. Add this as an extra input in the trigger.

New HTTP Request to add User

Add a new HTTP action, and I recommend renaming it. (Yes that should happen to the first one as well!). Again its a POST method and the URI needs id from the Parse JSON step slotted in. The Body is a simple JSON again. Code for both of these can be copied from below. The authentication is exactly the same as the previous HTTP step.

URI

1

https://api.powerbi.com/v1.0/myorg/groups//users

Body

1
2
3
4


{
 "emailAddress": "",
 "groupUserAccessRight": "Admin"
}

Conclusion to Create Power BI Workspace

You should now have a complete flow that when run asks for Workspace Name and Admin Email and creates a workspace with the admin added.

This is the MVP of the solution. There are obvious additions such as approvals, notifying the new admin and other various extras that should be included. They may get blogged, lets see.

Power BI Series

Use Power Automate to Fetch Secrets from Key Vault

Wed, 29 May 2024 08:34:09 +0000

Once you have stored security details in Azure Key Vault as described in the previous post in this series (Create Azure Key Vault to store Tenant ID, Client ID and Secret) you need a way to fetch secrets when you need them. Power Automate connector for Azure Key Vault includes the Get Secret action. This post shows you how to build a reusable child flow to fetch secrets.

This post is part of Power Automate and Power BI Rest API series.

Flow design

The flow we are going to create will be called by other flows, so is a child flow. The flow will have one parameter of the prefix of the ClientID and Secret store in the Azure Key Vault. If you have completed the previous post this would be PBI–Workspace. It will return 3 values, TenantID, ClientID and Secret which will be secured

Because we are using child flows we need to create our flows inside a solution. I recommend creating a solution for these group of flows. I don’t want to make this blog post even longer so please go google creating Power Platform solutions.

Setting up the Trigger

For a child flow we use a instant flow. Type in a descriptive name and select the trigger Manually trigger a flow. When you are in the flow editor, expand the trigger and add an input for the prefix. Adding hint text will help you and other users remember what the input is about.

Adding Actions to Fetch Secrets

The flow is reasonably simple with three repeating actions to fetch secrets. Add an action and search for Key Vault and select it. Then from the list select Get Secret. If this is the first time you have connected to a Key Vault you will need to type in the key vault name and click Sign in. Be aware that every key vault has its own connection.

The action has a very simple drop down. For the TenantID action you can just select it from the drop down. For the ClientID and Secret we want to the prefix value from the trigger so that this flow is reusable. So for the ClientID and Secret select Enter custom value. Then use the dynamic value Prefix followed by -ClientID and -Secret.

Securing the Secret Value

The secret value should be kept secret. So the Action output needs securing. Click on the ellipse (3 dots) on the Get secret – Secret action. Turn on the toggle for Secure Outputs and then click Done at the bottom of the dialog.

This could be done on ClientID as well if required. Each action that has a security setting applied will show a padlock. When a flow run execution is looked at the value of the secret will not be available.

Returning the Values

The final part for this flow is to return the values. And for this flow to run as a child flow it must have a respond action. Click on add action and search for the Power Apps connector. It only one has one action Respond to a Power App or flow. Add three outputs to return TenantID, ClientID and Secret.

Handling the Connections

When an instant flow runs the connections are usually based on who clicked the start on the flow. This does not work for a child flow, so a connection must be selected that will be used for when it runs.

The changes need to be made from the flow details page, save and click the blue arrow in the top left to get there. Click on Edit in the Run only users box. When the pane appears, under connections used select a connection under Azure Key Vault. A message will appear to state users only get access in this flow, click OK to close it and then click Save to save the changes.

Testing the flow

When you first write a child flow I suggest you write an instant flow to test it. The Run a Child Flow action only is available if your flows are in a solution. In order to protect the secret value you also need to protect the outputs of the Run a Child flow action.

This tests that the flow runs a child flow and returns values even though we can’t see them.

Conclusion

We now have an app that can fetch secrets, we’ve secured the authentication details and now written a flow to securely retrieve those values. Next step is to make sure Power BI will allow us to do this.

Power BI Series

Create a Power BI Service Principal Profile

Sat, 17 Feb 2024 18:18:23 +0000

For Power Automate to use the Power BI Rest API it needs a service principal profile. This is created by creating an app registration that has the right permissions. This allows an app, e.g. Power Automate, to establish a trust relationship with the Microsoft identity platform. This is done using Entra Admin Center (previously known as Active Directory)

This post is part of the Power Automate and Power BI Rest API series

Create App Registration for Service Principal

Navigate to https://entra.microsoft.com/. Then in the menu expand Identity and under that Applications. Finally click on App Registrations to open the list of App Registrations. It defaults to list registrations owned by you.

Click on New registration to start creating your App registration. Enter a name for your application, remember to make clear as over time the list will grow. Leave rest of the options as default and press the Register button.

Change App Registration Ownership

It is recommended that you add owners to the registration. Click on Owners, then click on Add Owners and select the owners. App owners and administrators can view and edit app registrations.

Adding API Permissions

Permissions need to be given to the App Registration based on what you want to do. The Power BI Rest API documentation at https://learn.microsoft.com/en-us/rest/api/power-bi/groups/create-group states you need Workspace.ReadWrite.All.

Click on API Permissions and then click on Add a permission. When the list of APIs appear, select Power BI service. Then select Delegated permissions and from the list of permissions select Workspace.ReadWrite.All. It is good practice to only add the permissions you need.

			## Adding a Secret

The final step in App Registration is to add a secret. This will be required by anyone using the app registration.

Click on Certificates & secrets, then click on New client secret. Enter in a description, I chose to keep the default 6 months expiration. Click Add to finish creating the secret.

Once the list appears you must copy the secret value. This is the only chance you get.

Important Information

You will need the Secret value copied from the last section and the Tenant ID and Client ID. The final 2 values can be found by clicking on Overview. These values especially the secret need to be kept secure. So the next post is how to create an Azure Key Vault to store the values.

Conclusion

Understanding how App Extensions and Service Principals work is very useful not only for Power BI Rest api. The Microsoft Graph api can also be accessed using a service principal but that is a whole different series.

Power BI Series

Power Automate and Power BI Rest API

Thu, 15 Feb 2024 14:28:37 +0000

The Power BI Rest API provides the ability to create a workspace. Power Automate includes an action to make a HTTP call, i.e. a Rest API. So we should be able to get a flow to create a workspace. So that was my goal. Writing the first flow and getting it to work involved so many little steps I had to write blog posts so I would remember the steps.

Power BI Rest API Documentation

The documentation refers to workspaces as groups and can be found at https://learn.microsoft.com/en-us/rest/api/power-bi/groups/create-group. It states that the API can be called by a service principal profile and the required scope is Workspace.ReadWrite.All.

Other Settings

You also need to make some changes to Power BI tenant settings as well and I’d recommend creating a security group and using Azure Key Vault to store the credentials. That would make a very long blog post so I’m breaking it down into separate posts.

Create a Power BI Service Principal Profile
Create Key Vault to store Tenant ID, Client ID and Secret
Write a flow to fetch the Key Vault Values
Update Power BI Settings and Create a Security Group
Write a flow to create the workspace
Extras for the workspace

Final Power Automate Flow

The final flow, after post 5, fetches the Service Principal credentials using a child flow, then uses HTTP action to create the workspace and with a bit of parse json action magic uses another HTTP action to change the permissions.

The above flow has flaws, no error handling, no checking the workspace name is allowed and no notifying the admin they have a new workspace.

Next Steps

This is a simple example of using Power Automate with Power BI rest API to perform some admin tasks. This could easily be extended to include other boring admin tasks.

Power BI Series

Power Automate – Write Me a Flow

Wed, 09 Aug 2023 16:44:09 +0000

Power Automate create includes a feature called Describe it to design it which will write me a flow if I can describe it. In Aug 2023 its still in preview, and is not 100% and only copes with simple requests. It doesn’t require a co-pilot licence though!

I started to explore this feature for a session for Scottish Summit 2023, trying to do 3 demos in a 25 minute session, I needed my first demo to be quick.

YouTube Version

Getting Started

On Power Automate site click on Create and then click on the Describe it to design it tile. This starts a 3 step process. The first step is to describe the flow. My simple request was when I added a new file to a SharePoint library that the Power BI report that was based on that library got refreshed.

Once Step 1 appears I enter in my request using a “When X then Y” pattern. It will show you a list of suggestions as you type. Click the arrow to submit the description. If it works you get a suggested flow and you can click Next to move onto Step 2

When it cannot translate your description into a flow, it shows a message of No flow suggestions. This feature is in preview currently so its not yet working 100%. There are links to provide feedback, they can’t fix what you don’t tell them.

Step 2 – Connections

When you click next it takes you to step 2 of 3. Here you are asked to review the connections the flow will use.

Step 3 – Settings

The final step is to add some settings to the flow steps. The drop downs are populated just like in the full flow editor. In some of my explorations it didn’t give me everything I needed but it works mostly. The you click Create flow.

Power Automate did write me a flow!

Very quickly you get presented with a flow. I recommend editing the name in the top left and of course pressing save in the top right. Then test your flow out.

Conclusion

For getting started, asking Power Automate to write me a flow is great. For developing a long complex flow its not going to help much. It does allow you to explore some ideas and with the right description it will build longer flows.

Power Automate Button in a Power BI Report

Mon, 07 Aug 2023 20:39:39 +0000

You can add an action to a Power BI report by adding a Power Automate button. The button will trigger a flow to run and perform actions. You add data to be used in the flow. In this post I will show to use a flow to add a row to a table that the report is connected to.

YouTube Version

Problem Definition

The report is connected to a table in a database that contains the start times of sections in a bike ride. The section times are used to for calculations and colouring charts. There are three measures that calculate the values to put in a new row of data. I want a Power Automate button to insert a row into my database table.

Add a Power Automate Button

From the Insert ribbon tab in Power BI desktop, we select the Power Automate visual. It’s right down the bottom under Other. This will add you a visual that includes instructions. The first step is adding data to the visual that will get passed into the flow. I have added my three measures.

Creating the flow

After you add the data, the next step creates the flow. Click on the … link on the top of the visual and select Edit. This will open a window in Power BI desktop showing you templates and any existing flows that you have.

In the top left, select New and select Instant cloud flow. Then the window moves to give you Power Automate flow editor inside Power BI desktop. Be aware it will be using the credentials you logged into Power BI with.

Then you can click New step and add your actions. In this example I am using SQL Server connection Action to insert a row to a table. I populate the Server name, Database name and Table name. Then the table fields appear. When I click in ActivityID, the first field the Dynamic content shows me the fields available including values I added to the Power Automate button.

When I click on Power BI data Selected Activity ID, measure from my report, it populates the box and it also puts the action into a Apply to each. This is because the values are sent to the flow in a table. When writing the flow it knows its a table so assumes it could be multiple values so therefore adds the apply to each.

There are multiple ways to handle this and convert that table into just three values, or we can accept we know the Apply to each only ever get a single row. Add a comment in your documentation to save future confusion.

Save and Apply the Flow

When you have completed your flow. You need to add a name in the top left. Then click Save and then click Save and Apply. Yes you need to click both!

After you click Save and Apply you should get a success message in a green bar. Then you can click Back to Report in the top left.

The visual in your report will now have changed to be a blue button. And you can test it out. In Power BI desktop you need to Ctrl + click, when you publish you can just click the button.

Formatting the Button

Similar to other Power BI visuals formatting can be applied. The blue isn’t a theme colour so I recommend selecting a theme colour so it will change with themes. You also can tweak padding, shadows and the text to make it look part of the report. I could not find a way to change the arrow icon.

Flow Environments

A Power Automate flow lives in an Environment and developers usually get to select which environment and get to put flows into solutions ready to go through a development life cycle. When writing that flow in Power BI desktop you do not get a choice on environment. It goes into Default environment and therefore cannot be put through any development lifecycle.

One workaround / hack exists. Thank you to James Reeves who pointed it out in a LinkedIn Post. It works currently but its a hack and should be treated as such.

Flow Permissions and licences

When the flow runs it runs as the user who clicks the button. This means firstly the user must have access to the flow, so don’t forget to share it and if like in my example it uses premium connectors the user needs the right licences.

Conclusion

Adding a way to react to the data and do write is great and Power Automate does it pretty well. There are some missing features though, I’d like to select environment for the flow and be able to select a flow from the solution I’ve added the Power BI report to. And then include that in the solution lifecycle management.

Power BI Series

Power Automate – Get data from a Power BI dataset

Tue, 11 Jul 2023 16:05:12 +0000

Power BI datasets contain a rich useful data resource that would be really useful in a Power Automate flow. In this post we will walk through how to get data from a Power BI dataset and then email that data. This is just one example of how you could use data fetched from a Power BI dataset behind a report.

YouTube Version

Starting the Flow

In this example we want to send a monthly quick summary. So I started by creating a scheduled email, repeated every month. After I enter the flow name and change the repeat to every month rather than the default every minute! Now I can press Create, which after a few moments creates a flow with the recurrence step.

Fetch Data using a Power BI step

Click on New step to start the process. In the search box type in Power BI and click on the Power BI tile when it appears. Now you can search the Power BI actions. In the search box enter in query and this will filter to 2 actions. The action we want is Run a query against a dataset.1

When the step appears, I select the Workspace and Dataset I am interested in. The query text is expecting a DAX statement. Don’t worry you don’t have to write the DAX by hand, we can get Power BI desktop to do it for us.

Writing the Query Text

Power BI desktop can write the query text for you. You need to open the Power BI report or a report connected to the dataset. On a blank page add a table visual. Populate the table with the data you want to use in Power Automate remember to apply the correct filters. I also recommend you remove totals unless you want the totals row in your data.

In my example the table shows Total Sales and Sales Growth for products in the previous month, using Offset Month column from my calendar.

After creating the table, on the Optimize ribbon tab click on Performance analyzer2. When the pane appears on the right hand side, click on Start Recording. Then you click on Refresh Visuals. This tool is usually used for analysing how long each visual takes to refresh and part of that is to provide the query being run.

Expand the Table line by clicking on the +. Then click on Copy Query. Now you can return to the flow in Power Automate and paste the query into the query box.

Sending the Email

The final part of this process is to create the email to send the details. The Power BI step returns an array of the rows of data. The Send an Email action would like this array converted into a html table. So we need 2 actions, create HTML table and Send an email. I am not going to make the table or email pretty. That is a separate post!

Testing the flow

I would test as I added each action to the flow, even on a short flow. Obviously once completed I’d test again.

Resources

The report used for this example can be found on my GitHub

Power BI Report – https://github.com/Laura-GB/DemoData/raw/main/Sales%20Data.pbix
Excel file https://github.com/Laura-GB/DemoData/raw/main/Sales%20Data.xlsx

Conclusion

Remember its Power Automate not Power Integrate. So yes you can load a long list into Power Automate but this is not a good tool to load large quantities of data into a new system. This is a brilliant feature though that unlocks data in a dataset for an automation process.

Power BI Series

SVG in Power BI series
Power BI and Project Online series
Slicers series
Dataflow series
Power BI SVG series
Power Automate and Power BI Rest API series
Power BI and DevOps series
The other action I have no idea how it works and have yet to find any documentation [↩]
In earlier versions of Power BI it was on the view ribbon [↩]

Power Automate Child Flow

Thu, 14 Jul 2022 12:34:07 +0000

Occasionally it is useful to break your flow apart into separate flows. For this you need to create a child flow or two. There is one tweak I always forget, hence this post.

Create the Child Flow

When building child flows you need to follow these rules

Child flows need to be built inside a solution
The trigger needs to be a Manually trigger a flow. Add inputs to the trigger.
To return values back to the parent flow the last action should a Respond to PowerApp of Flow action.

Above is a very simple example. The trigger has an input of location and the child flow will return 3 values.

Calling the Child Flow

Now you have created your child flow and tested that it works you can now call it. In the parent flow, add the action Run a child flow. Select your child flow and populate the inputs.

As soon as you add this you will probably get an error in the flow checker. Stating “Update the child flow for action ‘Run a child flow’ to not use ‘run-only user’ connections”

Fixing the Child Flow

Return back to the child flow information page. In the bottom right of the window look for Run only users box. When you have found it, click Edit. For each connection change the Provided by run-only user to the connection listed. You get a dialog stating the run-only users only get access to the connections for this flow. Click OK.

When you have updated all the connections remember to click save.

You can now return to the parent flow. You might need to re-add the action but you will no longer get an error.

Conclusion

Child flows are a great way to re-use logic and break a process down into manageable parts.

Refreshing Datasets Automatically with Power BI Dataflows

Sun, 29 Aug 2021 18:58:12 +0000

I have a report, Open Issues, that has a Power BI dataflow as a data source. If I just use scheduled refreshes, my options are limited. I need to be refreshing datasets automatically after the dataflow has finished so my reports are up to date as soon as possible to avoid a situation where I schedule the dataflow to refresh at 10am and the next slot available might be 10:30am, for example. This can be done using Power Automate.

Create a flow in Power Automate

Head to Power Automate at https://flow.microsoft.com/and click on Create on the left hand side menu. The flow will be triggered by the dataflow refresh finishing. This means it is automated so click on Automated cloud flow.

A dialog appears asking for details of the flow. You need to enter in a name for the flow. Then in the search box enter in power to find relevant triggers. Select When a dataflow refresh completes and click Create.

After you click Create the flow editor appears with the trigger already added. For Group Type select Workspace. Then you can select the workspace name for Group. Then you can select the Dataflow.

We then need to refresh the connected report dataset. So we click New step. In the Choose an operation dialog, we type dataset into the search box. This will find you the action Refresh a dataset. We click on the action to add it to the flow.

Then you need to select the Workspace and Dataset in the action. The flow is complete, so we can click Save.

Testing the Flow

The flow triggers when the dataflow finishes refreshing. If we refresh the dataflow that will test the flow. I clicked refresh on the dataflow at 3pm and it triggered the flow which then refreshed the dataset.

In the workspace we can see the date and time of the refreshes and if the flow has worked correctly the dataset should be refreshed just after the dataflow. If you look at the history of the flow runs there should be a matching flow run just after the dataflow finished refreshing.

When refreshing datasets automatically fails

It is worth pointing out that if the dataset refresh fails the flow will not reflect that and will show succeeded in the flow runs. So it is worth checking that the dataset can refresh successfully.

Conclusion

This idea can be extended to refresh multiple datasets over multiple workspaces that are connected to the dataflow. This simplifies the process of scheduling refreshes. It also could allow for notifications to interested report owners to be told of the refreshes etc.

Power BI Series

Power Query in Flow – Summarising the Data

Mon, 23 Sep 2019 08:33:01 +0000

This is the third post in my series regarding Power Query in Flow. In this post I will look at the different options we have for summarising the data using grouping and pivoting with Power Query.

Here is a list of all the posts in the series so far.

Introduction

Again this post might be redundant if you’ve followed my advice and found out your dba’s favourite biscuits and got them to write you some awesome views in the database.

For those that haven’t got access to the dba lets look at how we can summarise the data. We will look at two different types of summarising the data, firstly grouping and then pivoting. I have assumed you’ve got your data into a simple table using merges etc.

Using Grouping

In this example we will group by a column and include some summary values. We will group by Product and then summary values of count of orders, total quantity and total value.

This is how the data starts.

On the toolbar click Transform Table and select Group By. We will start with a basic grouping of just one column to group by and one summary column of Order Count.

I selected ProdName for the group by , entered a name for the summary column to give Order Count by counting the rows.

To add more columns to the summary I click on the cog wheel on the Grouped rows step to re-open the group by dialog.

I click on Advanced to open up options to add extra grouping and aggregation options. I click on Add aggregation to add extra columns and change Operation to Sum and Column to total up Qty and Value. Clicking OK updates the table.

Using Pivoting

Pivoting is a form of grouping that splits the data across a number of columns, for example total sales for each product across hours of the day. The first step is to reduce your table into just the data you need. So include the data for each row title, one set of data to be the column headings and one to be the data summarised.

So I do Select Columns to reduce down to just Product, Hour of Sale and Qty of items sold.

Next, select the column of data that will make the column headings in your pivot. In my example I select the SaleHour column. Then from Transform Column, select Pivot Column.

In the Pivot column dialog, click on Advanced to reveal more options. In the Value column select the value you want to summarise and select the function you to use. Click OK to pivot the data.

Conclusion

Power Query provides a powerful way for summarising the data ready to process within the flow. As I stated earlier a view written by a dba would be more useful and quicker.

AI Builder Object Detect Model with Power Automate and Power Apps

Mon, 09 Sep 2019 09:38:06 +0000

This series of posts is to introduce using an AI Builder Object detect model with Power Automate to get and save predictions and then display these predictions over the images in a Power App.

Build the Model
Save Predictions
Use Predictions in a PowerApp

Introduction

In this post I will document building an AI Builder object detect model. We will create the list of items to detect, upload images, identify each image and then train and publish the model. I will be using 2 different types of Lego bricks, 1×2 block and 2×2 block.

This post will be split into multiple parts

Create Entity to store object names
Create Model
Upload and Tag images
Train Model

Create Entity to store object names

Before we can start building model we need to create a list of object names stored in the Common Data Store (CDS) in an entity. We need to create the entity and load object names.

We start in Flow.Microsoft.com and then on the left hand menu click Data and then Entities.

From the top ribbon select New entity which will open a pane on the right hand side. Enter a Display Name for the entity. It will complete the other fields for you.

Click Create to create the entity.

The next stage is to add data to the entity. The easiest way to do this is via Excel. Click on the Edit Data in Excel. A file will be downloaded which you need to open in Excel.

You might need to enable the feature to get the pane to the feature in Excel. Enter in the Object Names and then click Publish to save the values to your entity.

Once the values have been published you can check back in your entity by back on the website and clicking on Data.

If Excel won’t play ball, and for one client it just won’t, then you can use a very simple flow to add the entries.

Create the AI Builder Object Detect Model

On the left hand menu select AI Builder and then Build. The AI Builder currently offers 4 types of model. For this post we are going to build an Object detection model, so click on Object Detection. Then enter a name for your model and click Create.

Select Object Names

The next stage in building the model, is to select the names to be used in the model. Click on select object names. From the panel select the entity you created and tick the Name column and click Select Fields.

On the next screen select the values and click Next.

Add Images

After the object names have been selected you then need to add images for the model to be trained from. Click on the Add Images and this will open a file selection window, select files and click Open.

On the next window it will show all the selected files. Click on Upload to start the upload. When the upload has completed click Close to move onto the next stage.

The next window will show you a summary of your uploads. You can click on +Add Images to add more images to improve your model. You need at least 15 images but more is recommended. Click Next to start tagging your images.

Tagging Images

The next stage is tagging the images. I recommend a cup of tea, some music because this takes time.

Click Next to start the tagging. Thumbnails of the images will be shown. Click on an image to enlarge that image and start the tagging. You can drag a box of an object or it will suggest objects with a white dotted box. Select the tag to apply.

On the right is a summary for this image and on the top ribbon you can choose to not use an image. When you click on the arrow it will move you onto the next image.

When you have tagged all the images, click on Done Tagging in the top right hand corner.

Train Object Detect Model

When you click Next you get shown the details of your model so far. Click on the Train button to start the training.

Training takes a while, the above one took 5 minutes to train. I assume its longer when there are more images and more tags. Eventually your list of models will show a time of being trained.

The last step is to publish your object detect model. Click on the model name in the model list and it will show you a score for the training and give you a Publish button. Click the button to publish your model ready to use.

Conclusion to Object Detect Model

This is a new feature still in preview, so it has its clunky parts, but it brings machine learning firmly into the hands of the citizen developer without any Azure bills.

I really hope in 6 months time I need to write this post again because they’ve added new features and made it slightly less clunky.

Joining Tables of Data in Flow’s Power Query

Fri, 06 Sep 2019 20:00:01 +0000

This is the second post in my series regarding Power Query in Flow. In this post I will introduce one of the most powerful parts of Power Query, joining tables. Its the vlookup part of Power Query.

Here is a list of all the posts in the series so far.

Introduction

If you have made best buddy friends with your dba (database administrator) then this post is possibly not required because if your dba will write a view to do the join for you, that is always better. BTW in my experience, dbas like biscuits.

For those who don’t have access to their dba or haven’t bought enough biscuits this what we are going to do. My database contains a table called productsales.

In the Product Sales table is a ProductID column. I am going to use this value to merge in the values from the Product table (i.e. joining tables) so I can get the name and price.

Multiple Tables

We start by adding both tables to our query. If multiple tables are in a query, Power Query only returns one table. You can tell which table will be returned by looking at the list of tables. The tables with italic names will not be returned, therefore the un-italicised table will be returned.

Right click on a table name to open the menu. Then select Enable Load to change the table that will be loaded. In this example I will select that productsales will be loaded.

Merging tables

We want to add the product name and product price to product sales tables. We will do this by merging the two tables.

Select the productsales table. Then from the toolbar, select Combine Tables and Merge Queries.

In the Merge dialog that appears select table you wish to merge with in the drop down, in my example I select products table. Then click on the columns that match, in my example I select product id in both tables.

Leave the join kind as Left Outer as this will keep all the sales records and find matching products. Further details of join types can be found on Radacad’s site. https://radacad.com/choose-the-right-merge-join-type-in-power-bi

Expanding the table column

Clicking OK will update the table to include a column which contains Tables.

Click on the icon in top right of the new column to display the columns available in the tables. Unselect any un-required columns. Unselect the Use original column name as prefix to prevent the new columns being named long names for example products.ProdName. Then click OK to add the new columns.

Conclusion to Joining Tables

Merging is a powerful addition to transforming data. It is better if a view can added to the database by your dba as that will always be more efficient. Power Query merges do offer a great alternative.

Power Query in Microsoft Flow – An Introduction

Mon, 26 Aug 2019 21:34:42 +0000

This is the first post in a new series regarding Power Query in Microsoft Flow. In this post I will cover the basics of connecting to SQL Server and using Power Query to filter and sort the data and finally add calculated columns.

This series assumes that you are not a user of Power Query desktop from Excel or Power BI.

Here is a list of all the posts in the series so far.

Introduction to Power Query in Microsoft Flow

In September 2018 Microsoft announced the integration of Power Query Online for Microsoft Flow. The reasons were to offer an alternative to OData and SQL for getting data from SQL Server connector. They included a hint that other data sources will come in the future but no details when or what.

For those coming from Power Query for Excel or Power BI you need to understand you are currently limited to data stored in SQL Server online or server via a gateway. Also the Power Query Online environment is not as powerful as the desktop app version. It is still written in M, so you can do many of the transformations you can in Excel and Power BI’s Power Query.

Connecting to the Database

Our first action is to connect Power Query to the data. Power Query action is from the SQL Server connector.

If this is the first time you’ve connected to the database you will be prompted for database details and the user id to use with a password.

Then the action then changes to just show a button. This means you have connected to the database successfully.

Selecting Data to Transform

When you click the Edit Query button for the first time, you will be presented with the tables and views in the database. You need to tick the tables and views that you need. For this post I’m just going select a single table Expenses and then click Next.

The next window is the main Online Power Query window, where we will add steps to transform your data.

List of parameters and tables
View of the data at the current step
List of steps
Toolbar used to add new steps

Filtering Data

One of the most common requests on data is to filter the values. Clicking on the arrow next to column header will show you a list of values with tick boxes. Un-tick the unrequired values to filter your data. When you do this it will add a new step to the applied steps.

When you want to change the filtered value, you need to click the cog wheel next to the step. You can click on X to remove any step.

Sorting Data

When you click the drop down arrow next the column heading, you can select Sort ascending or Sort descending. The first time you select sort, the data gets sorted and a new step is added to the applied steps.

If you then sort another column, the existing step is just extended to add the next column to the sort. This is similar to adding a new sort level in Excel.

Adding Columns

One of the most powerful parts of Power Query is to add columns using calculations. The interface can be really clever and also limited and drives me nuts which is perhaps a blog post all of its own.

If you select a column and then click Add Column from the top ribbon you will get offered choices based on the column type. For example, if you have selected a number column you get offered Standard, Scientific and Trigonometry functions and rounding plus others.

The result of the calculation is placed in a new column.

When you select multiple columns before selecting Add column, calculations to combine the columns are offered, such as difference between dates.

Renaming and Selecting Columns

You can rename any column by double clicking on the column header and typing in a new name. The first column you rename will add a new step to the query. If you rename another column straight after the Renamed Columns step is extended.

It is always a good idea to only return the columns that are required by the Flow. Under Manage Columns you can select Choose columns. When the dialog is selected, you can tick or un-tick any column.

The choose columns step has a cog wheel so can be modified at any point. I recommend you use this step rather than delete columns as that cannot be tweaked using the cog wheel.

Finishing

When you have finished transforming your data you can click Create on a new query or Update if you are editing an existing query. You can now use the data returned just as you would for getting items from any source.

Resources for Power Query in Microsoft Flow

As always there are a selection of resources out on the web.

•Microsoft https://flow.microsoft.com/en-us/blog/powerquery-flow/

Chris Webbhttps://blog.crossjoin.co.uk/2018/09/26/using-power-query-and-microsoft-flow-to-automate-the-creation-of-csv-files/

Erik Svensenhttps://eriksvensen.wordpress.com/2018/09/25/powerquery-everywhere-now-in-microsoftflow-as-well/

Conclusion

Power Query is a great addition to the Flow tool set. I am really hoping for more data sources to be allowed and for the online Power Query to include more features from the desktop version.

This is just an introduction so I will include best practices and further features.

Power Automate – Creating Adaptive Cards

Wed, 07 Aug 2019 07:09:31 +0000

This post is to support a video posted on YouTube regarding creating adaptive cards for Microsoft Teams using Power Automate.

Designing an Adaptive Card

Before you can post an adaptive card to a team you need to design the card. Cards are designed online on an interactive website that will build the JSON definition for you. In this post we will create a very simple card.

::: info Instructions

Visit https://adaptivecards.io/
Click on Designer on the top navigation
Click on New Card
Drag TextBlock elements from the options on the left into the Card. Double click on the text to change the text.
Make changes to the layout and style as required. The JSON defining the card will be updated.
Click Copy card JSON to copy the code to the clipboard.

:::

Other elements could be added to the card and formatted as required.

Writing the Flow

For this post the flow is going to be very simple. I start by adding a button trigger with a single string input called Message.

Under the Teams connector, I find the Post your own adaptive card as the Flow bot to a channel. I select the Team and Channel that I want the message to go to.

In the message I paste the Adaptive cards code copied earlier. I double click on the MESSAGE and then click in the Dynamic Content pane on Message to replace the text.

Running the Flow

The flow can be run from your phone and the Flow app. Typing in a message will add the card to the channel in Teams.

Conclusion

This is a very simple example of an adaptive card being posted in Teams. The possibilities are endless and are a great next step up from sending email notifications to multiple people.

PowerApps – Using JSON Function to Save Pictures

Mon, 08 Jul 2019 10:21:38 +0000

In this post I will be explaining how to use the JSON Function to simplify the saving of multiple pictures from a PowerApp to a SharePoint library using Flow. There have been many wonderful workarounds to do this, finally we now have a slightly less Heath-Robinson method.

Set up the PowerApp

I start with a blank app, canvas or phone layout. Because my main client uses Windows 10 tablet / laptops to take the pictures I add a camera control. Then I change the OnSelect property to store the picture into a collection. I include an index column so Flow can calculate a unique filename later.

I then add a blank gallery and change the Items to be the collection populated by the camera, e.g. ccPictures and to make a grid I change the wrap to be 3. I add an Image control to the gallery so as I take pictures it will populate the grid.

Then I take some pictures to populate my collection. Obviously everyone needs photos of their Lego character collection.

In order for the pictures to be passed to Flow to be saved to SharePoint I need to convert the collection into JSON. So I add a button with the following code. Please note the second parameter of the JSON function is IncludeBinaryData as we want to include the pictures which are binary data.

Building the Flow

In the previous section, I created the app to make the JSON string to be processed by a Flow. So I now I will create a new flow with a PowerApp trigger.

The first step to add is a Parse JSON action to convert the JSON into an array. In order for the parameter requested by Flow to be named nicely I rename the step to just “JSON” and then in the Content box I click on Ask in PowerApps. Now the parameter is called JSON_Content. Afterwards you can rename the action back to Parse JSON and it won’t rename the parameter.

For the Schema you need to understand JSON a little. The JSON you return will be multiple rows each with 2 columns, Picture which is a string and Index which is a number. So in a text editor I create small example JSON string with multiple rows of data and copy it.

Then in the Parse JSON action I click on Use sample payload to generate schema and I paste my example in, clicking Done will create the schema.

The next step is for every item in the array to save a picture. So we start with an Apply to each action which will use the Body from the Parse JSON step.

Then we add a SharePoint action to Create file. Site address and path you will need to get from the SharePoint destination. The File Name I create using the Index value.

The File Content uses an Expression. So click on Expression and find the function decodeDataUri. The parameter for the function is the Picture value, which doesn’t appear in the offered Dynamic content, so we need to use the Item() function and then add ?[‘Picture’].

Connecting PowerApp to the Flow

We now have a Flow ready to receive JSON data and save pictures. So we now just need to connect them. In my experience adding a flow to a button that already has code does not end well, I usually loose the existing code. So I add a second button and from the Action ribbon click Flows and connect up my Flow and pass in as the parameter the variable containing the JSON, e.g. vvJSON. Then I copy and paste this code into the button that creates the JSON string.

The app is now ready to test.

Conclusion

The above solution is a very simple example of how the JSON function could be used. Before this went into production I would do better file naming. I have tested the above app to take 30 photos and save them to SharePoint and it worked fine. I assume there is a limit to the size of the JSON.

The JSON function does allow a whole collection to be passed to Flow easily. I will be looking for other possible uses.

Power Automate on Hat Full of Data

Trigger Microsoft Fabric Data Pipeline using Power Automate

Fixed

YouTube Version

Workspace and Pipeline GUIDs

Start Flow and Add Variables

URL of the Trigger Microsoft Fabric Data Pipeline request

HTTP Request

Base Resource URL

Microsoft Entra ID Resource URI

Testing Trigger Microsoft Fabric Data Pipeline Flow

Conclusion

More Power Automate Posts

Power Automate – Save a File to OneLake Lakehouse

YouTube Version

Getting the File Content

Lakehouse Folder URL

Create the file in the LakeHouse

Set up the connection

Create Blank File

Append Content

Test Save a File to OneLake Lakehouse

Conclusion

More Power Automate Posts

DevOps updates on behalf of another with Power Automate

DevOps with Power Automate posts

YouTube Version

Permission required to makes updates on behalf of another

Make Updates on behalf of another

Word of caution

Conclusion

More Power Automate Posts

Update DevOps without Notifications with Power Automate

DevOps with Power Automate posts

YouTube Version

Permissions for updates without notifications

Updating without Notifications

Item History

More Power Automate Posts

Running a Wiql DevOps query in Power Automate

DevOps with Power Automate posts

Writing the Query

Wiql Editor

Construct Query

Running the Wiql in Power Automate

Handling the Output of the HTTP call

Looping through workItemRelations

Conclusion

More Power Automate Posts

Using DevOps REST API in Power Automate

DevOps with Power Automate posts

YouTube Version

Get Task Details using DevOps REST API

Update a Work Item using DevOps REST API

Conclusion

More Power Automate Posts

Power Automate Update Fields in Azure DevOps

DevOps with Power Automate posts

YouTube Version

The Problem

Finding Field Names v1

Create Work Item and Update Fields in Azure DevOps

Find Field Names v2

Conclusion on Update Fields in Azure DevOps

More Power Automate Posts

Footnotes

Connecting Power Automate to DevOps

DevOps with Power Automate posts

YouTube Version

DevOps Connector

Create an item

Troubleshoot Connecting Power Automate to DevOps

Advanced Options in Create an Item

Making it Solution Aware

Conclusion on Connecting Power Automate to DevOps

More Power Automate Posts

Power Automate – Create Power BI Workspace

Start the flow and get service principal details

HTTP Call to automatically create Power BI Workspace

Add Workspace Admin