Variable Libraries in Deployment Pipelines

Fri, 17 Jul 2026 00:00:00 +0000

So you’ve done the hard work of building a variable library to make all your Fabric artifacts not have hard coded values. This was promised to be the prep work for making deployment pipelines easier. So lets take a look how that works.

Variable Library Series

Variable libraries should be part of every project. This post is part of my series to help get you started creating the library and then using the variables and finally seeing your hard work pay back when it comes to deployment pipelines.

Getting started with variable libraries
Variable Values in a Fabric Notebook
Variable Values in a Data Pipeline
Variable Values in Lakehouse Shortcuts
Variable Values in Dataflows
Variable Libraries in Deployment Pipelines
Connection variables in a variable library (coming soon)

Scenario

For this post we are keeping the example very simple. We are building a process that ingests data from a file into a table. As part of that process the data is loaded into a dataframe called df. The variable library includes a RowLimit variable. If the RowLimit is 0 load all the data, otherwise trim the data to that many rows. This means in dev we can limit to a few rows but in test and prod we can have all the rows. Here is the code snippet that handles that logic.

1
2
3
4
5
6
7
8
9


# Get RowLimit Value
vl = notebookutils.variableLibrary.getLibrary("Variable_Library")
RowLimit = vl.RowLimit
print(f"{RowLimit=}")

# If RowLimit has a value above 0, truncate the data
if RowLimit > 0:
 df = df.limit(RowLimit)
 print(f"Data limited to {RowLimit} rows")

Setting up the Variable Library

I’ve created a variable library with a variable called RowLimit that is an integer and has a value of 5.

When I run the code we get this

Testing for zero

However confident we are on the logic we should test for 0 working. I do and I get the expected output of RowLimit=0 and no other text and all the data rows come through.

Adding Value Sets

A variable library can store different values for each variable in a value set. You can add multiple value sets, usually one for each deployment pipeline stage. In my example for dev I only want 5 rows of data, in test we will look at 20 and in prod I want no limit. Yes there probably should be a UAT with zero as well… this is just a demo!

📝 Instructions

Click on Add value set on the ribbon

Enter a name and click Create

The value set appears with all the variable set to the same value as the Default value set.

Repeat for as many value sets you need.

Edit the values in datasets. The icon will turn black.

If you need to reset to default click on the black icon

Click Save and Agree if a warning appears.

Deployment Pipeline

I’ve created a deployment pipeline and deployed to Test.

Changing Active Set

In order to use the different values for Test and Prod I need to change the active set in the variable library in the workspaces.

📝 Instructions

Go to the Test workspace

Open the variable library

Click on the three dots on the Test set and select Set as active from the menu.

In the confirmation message that pops up, click Set as Active

not the active badge moves and the test set has a heavier border

Save the variable library

The pipeline does not notice the change and any updates to the variable library from deploying from dev do not reset the active set.

If we now test the notebook in the test workspace we get 20 rows coming through because of the different value in the library.

💡 Tip

Always change the variable library values in your dev workspace and deploy the changes through the pipeline.

Conclusion

This is the big reason Variable libraries are a must have in Fabric solution development. Even if you don’t use deployment pipelines it is sometimes useful to be able to swap values.

Humbled and Honoured

Wed, 15 Jul 2026 00:00:00 +0000

Today I got the news I get to be a Data Platform MVP for another year!

As tradition dictates I am of course Humbled and Honoured. In reality though, I’m hugely grateful to lots of people who let me be me in this community.

Hosts of user groups and conferences who create the place for us to meet and present
Helpers who make conferences work, tell me when to end a session and help me find my room
Humans who came to my sessions, read my blog, watched my youtube channel, asked questions and made it feel worth while. If any non-humans were involved I am of course grateful to them as well.
Hecklers¹
Handful of friends who have shared the highs, the hurdles, headaches and the hilarity of another year in tech.
Hubby, Harry, who has done the airport drop offs and pickups, cooked while I build slides and demos, never complains when head off to another conference or user group.

Turns out H is a great letter.

The award comes with a badge, an updated one this year, but in reality the award opened doors for me to an amazing community. We use the ever changing, name changing, icon changing² tech as an excuse to gather, but we stay because of the community of wonderfully diverse, usually neurodivergent and amazing humans we are.

A huge thank you to my co-speakers this year Claire Edgson, Joe Griffin, Lewis Baybutt and Will Johnson. We rocked!

Also to Mark Pryce-Mayer who makes me laugh and fixes the lakehouses I break, Sue Bayes who makes me smile every week, Alex Powers who encouraged me to keep going, Ian Tweedie who helped me transfer my blog from wordpress to Hugo, so blogging will again be enjoyable and Ash Graham-Brown who did the accessibility work to make sure a screen reader makes some sense on my blog.

And last, but never least, Rie Merritt who looks after us Data Platform MVPs, keeps us in line and always supports us non-males in a male dominated industry.

My advice to anyone looking in on the community, unsure how to join, please reach out. Come talk to a speaker after a session, ask the questions, be brave and say hello. Many of us are incredibly shy, anxious and imposter syndrome is our best skill, so we understand your shyness. You are one of us, I promise. Take a look at my Upcoming Events page and if you are going to be at one of them reach out to me beforehand and we can have a coffee together and I’ll introduce to people you want to meet.

¹ You know who you are
² Microsoft stop it!

2026-2027 on Hat Full of Data