Fabric Lakehouse Tables over ADLS: A Secure Design Using Private End Point and Key Vault

Introduction

With the rapid evolution of Microsoft Fabric, many of us are already comfortable creating tables or shortcuts over Delta tables stored in Azure Data Lake Storage (ADLS). The usual approach is to provide your Fabric workspace with direct access to the storage account.
But in real-world enterprise environments, things aren’t always that simple.

The Challenge

Often, the data is owned by another team, and due to compliance or security restrictions, they cannot grant you direct access through Access Control Lists (ACLs). Instead, they may agree to share SAS tokens to let you access the data.
Here’s where another challenge appears:

• Most projects do not allow storing keys or passwords directly in code.
• The standard practice is to use Azure Key Vault, the most trusted way to manage secrets.

Why Key Vault Alone Isn’t Enough

At first glance, it seems easy—just store the SAS token in Key Vault and let Fabric access it. But there’s a catch:

• Azure Key Vault doesn’t allow public access, project restriction
• Azure Key Vault only works with trusted services or private endpoints.
• Since Fabric is a SaaS service, its IP addresses are dynamic and cannot be whitelisted.

This means Fabric cannot directly access secrets stored in Key Vault.

The Solution: Managed Private Endpoints

The secure way forward is to use Managed Private Endpoints in Fabric:

1. Create a Managed Private Endpoint from your Fabric workspace to the Key Vault
2. Get it approved by the Key Vault administrator.
3. Once approved, Fabric can securely connect to Key Vault and fetch the required secrets (like SAS tokens).

This ensures, No credentials are hardcoded in your code, Access remains fully compliant with enterprise security standards and Data sharing between teams is seamless and secure.

That’s the theory. In the next section, we’ll walk through the practical steps of configuring this setup in Fabric, ADLS, and Key Vault.

Practical Implementation

Step 1: Generate the SAS Token and Store It in Key Vault

Before diving into the practical steps, you need to generate a SAS (Shared Access Signature) token for the storage account where the Delta tables are stored. Once generated, this SAS token should be securely stored in Azure Key Vault.

Step 2: Create the Managed Endpoint in Fabric Workspace

Next, we need to establish a Managed Private Endpoint in your Fabric workspace to securely connect to Azure Key Vault. This step ensures that Microsoft Fabric can access the SAS token stored in Key Vault without exposing sensitive credentials.

• Open the Fabric Workspace and go to Workspace Settings.
• In the Outbound Networking section, you will find the option to create a Managed Private Endpoint.
• In the Outbound Networking section, you will find the option to create a Managed Private Endpoint.
  
• Click Create. This will open a new window for configuring the Managed Endpoint to connect to the Key Vault. Fill in the details as follows
  
  • Name: Choose a name for your private endpoint.
  • Resource Identifier: This is the Resource ID of the Key Vault, which can be found by opening the JSON view in the Overview section of the Key Vault.
  • Fabric will automatically fill the Target Sub-resource based on the Resource ID you provide.

After entering the details, click Create to set up the Managed Endpoint. It will now appear under Managed Endpoints with a Provisioning Status.

Step 3: Get the Managed Endpoint Approved by the Key Vault Administrator

Now, you need the Key Vault administrator to approve the Managed Endpoint connection. This can be done in the Private Endpoint Connections section under the Networking tab of the Key Vault.

Once approved, the status of the Managed Endpoint in Fabric will change to Approved. Note that it may take 5-10 minutes for the status to reflect in the Fabric workspace.

Step 4: Creating the Table in the Lakehouse

With the Managed Endpoint now approved, you can securely access the SAS token from Azure Key Vault. This allows you to connect to the Delta table stored in Azure Data Lake and perform operations as needed.
To create a table in the Lakehouse, follow these steps:

• Open a Notebook in Fabric Workspace and attach the Lakehouse as the default data lakehouse.
• Use the following code snippet to create a table under the Lakehouse.
  

  #Read The Secret from Key vault
  token = mssparkutils.credentials.getSecret("https://kvsubahan.vault.azure.net/", "SASToken")
  
  #Configure the Key to the Session
  spark.conf.set(f"fs.azure.sas.<Container>.<StorageAccount>.dfs.core.windows.net",token)
  
  #Create the Delta Table
  DeltaTableLocation = "abfss://<Container>@<StorageAccount>.dfs.core.net/<TablePath>"
  spark.sql("DROP TABLE IF EXISTS deltaTable")
  spark.sql("CREATE TABLE IF NOT EXISTS deltaTable USING LOCATION '{o}'".format(DeltaTableLocation))

  Once the code is executed, refresh the Table Section under the Lakehouse to see the newly created table.

Conclusion

With the steps above, you’ve securely configured the connection between Microsoft Fabric and Azure Key Vault, allowing you to access Delta tables in Azure Data Lake without exposing sensitive credentials. However here we should remember two importent points

1. SAS Token Access: Since we are reading data using the SAS token, ensure that the configuration code is present in every notebook accessing the tables. Without it, you will encounter an "Access Denied" error.
2. Cross-Tenant Limitations: This method will not work if Fabric and Azure are in different tenants, as cross-tenant applications are not supported for this approach at the moment.

I hope this guide helps you securely manage your data access in Azure using Fabric and Key Vault! If you have any questions or need further clarification, feel free to ask in the comments, and I’ll be happy to assist you.