One of the skills of creating good solutions on any platform is building reusable modules. For a part to be reusable we reduce the hard coding and pass in parameters. This is true for data pipelines. So this post post covers creating a data pipeline with parameters. Initial Scenario In order to demonstrate how to create a data pipeline with...
I set up my dataflow and it was working great. Then it broke on the refresh. Here is how I fixed it. Previously In the previous post we ingested a SharePoint folder into a table in a Lakehouse. That is a well known process and nothing new except for one thing. I split the process into two stages, firstly the...
SharePoint folder has been a connector into Power Query for Excel, Power BI Desktop and dataflows for quite a while. This post is to cover the basics before I write another post to explain a more complex scenario. This post will use GEN2 Dataflows in Microsoft Fabric. Power Query can take files on a SharePoint site and from a selected...
In VBA programming, dictionaries are invaluable tools that allow you to store key-value pairs efficiently. Checking a list of values to see if it contains a value is a regular requirement in many projects. The obvious solution in VBA is the values in an array and loop through till you find or don’t find the value. On a recent project,...
This post is about handling null values in Power Query calculations. If you do a calculation in Power Query that involves a null value the answer returned is null.
One of the challenges of making sure your query works in Power Query is data sources that have a changing schema. Some APIs miss out fields if the data is null and sometimes we need one report able to handle data files that are almost the same but not quite. In this post we will look at fixing 2 scenarios, a single missing column and multiple columns missing.
Good practice is to use parameters for these values so your query is reusable etc. And its always our intention to go and create the new parameter later.
Microsoft Fabric Data Pipelines can be scheduled to happen at regular times. What happens when you want trigger a Microsoft Fabric data pipeline to run as a reaction to an event? For example someone pressing a button in an app. For this we can use Power Automate to start a pipeline using a http request.
Currently there is no Microsoft Fabric connector in Power Automate. When we want Power Automate to save a file to OneLake Lakehouse, we need to use a workaround. This post walks through the demo I did for the Microsoft Fabric European Conference 2024.
There are times it is useful to calculate an inherited value from parent item to child. The target date of a feature to be passed down to the child user story and then onto the child tasks.
