Overview Along with Managed Private Endpoints, another security feature was released at the same time called Trusted Workspace Access for OneLake Shortcuts. Now, I had actually conflated these 2 features during my testing so I have now separated this into 2 blogs. This blog specifically covers the Trusted Workspace Access, please refer to this blog which discusses the Managed Private Endpoint feature which is how to allow Spark access to secured data services. To be honest I’m surprised we have multiple methods to allow Fabric to connect to secured services. I have a blog coming in the next co ..read more

Managed Private Endpoints in Public Preview Security fans rejoice (and companies that don’t appreciate leaving their data stores open to public access…)! Microsoft Fabric now has Managed Private Endpoints (MPE for the remainder of this blog) in public preview. For those not familiar with MPE, this feature allows a service such as Azure Synapse Analytics, Power BI, and now Fabric, to connect to another service that has been disabled from any public network access and/or hidden behind a firewall. E.G. you can create a storage account and disable it from public access but services will be able to ..read more

Overview I’ve been looking at implementing a proof-of-concept for the Fabric Link for Dataverse functionality to synchronise Dynamics 365 data into Fabric for use in analytics and warehousing. Previously I’ve covered Synapse Link for Dataverse which you can check out here. The official documentation for Fabric Link is available here. My early impressions of implementing Fabric Link for Dataverse are promising, there are certainly less moving parts than implementing Synapse Link, as there’s no requirement to spin-up a Synapse workspace, storage accounts, and Spark pools if you want the data in ..read more

Overview Probably one of the most hotly anticipated services is available within Fabric now, as Copilot in Fabric is now in public preview. See the official blog here for more details. I’ve been quietly interested in what Copilot can bring to Fabric and get hands-on with it to see if this AI-driven revolution is as great as the marketing would have you believe. In this blog I’m going to pick the Data Engineering Copilot feature, which allows you to ask Copilot for help when coding data loading, processing, and transformations in a Fabric Notebook. I know enough Python & PySpark to get by ..read more

Overview As of 12th January 2024, the beta exam for the new Microsoft Certified: Fabric Analytics Engineer Associate is now available. This is the link to the certification page and here is the link to the exam details page. I covered the exam in a previous blog post (and video) available here which breaks down the exam topics. As I previously did with the DP-500 Enterprise Data Analyst certification, I’ve listed each of the skills measured and linked to what I believe to be the relevant topic. There are the Learn modules which cover aspects of the exam, you can find those in the Two Ways ..read more

This blog will be split over 2 parts, in this first part we’ll look at the medallion architecture from the perspective of Microsoft Fabric implementation, (check a new doc from MS on the subject here). What I’ve been interested in looking into is what options we have in data processing within Fabric and what is/isn’t possible, as during my own testing I have found there are caveats and ways of working with these options. This is just my thoughts as I’ve been working through it. Part 2 will then look at actually building an end-to-end solution using the methods and principles outlined in this ..read more

Overview Recently I’ve been reading through Microsoft’s documentation on implementing medallion lakehouse architecture in Microsoft Fabric (doc is here) and thinking about how to structure the workspaces & lakehouses to deliver that methodology. It’s a great article and well worth reading. The one thing that stuck in my mind though is all about OneLake. OneLake is a single, unified, logical data lake for your whole organization. Microsoft You get a single OneLake per Fabric tenant (the name kinda suggests that…), this concept of single storage across a single Fabric tenancy interests me ..read more

Well this new Explore This Data (link) feature in Power BI surprised me, both in terms of usefulness and in how much I’m currently using it. I don’t usually spend a lot of time on the reporting and visualisation aspect of Power BI. The only story-telling my reports and graphs do is to tell a story of how the person who built the report doesn’t know how to! However, there was a new feature released recently that piqued my interest…a way to simply visualise data without that much effort? Yes please! Explore This Data Explore This Data is perfect for me, someone who spends most of the time wit ..read more

Ignite is always exciting as there’s always a bunch of new announcements, and included in this excitement is a new Microsoft certification, the Microsoft Certified: Fabric Analytics Engineer Associate. This is backed by a single exam, the DP-600. This is due to be released into beta in January 2024. Yes, Microsoft has embraced the Analytics Engineer role! I always though the DP-500 Enterprise Data Analyst should have been Analytics Engineer… This makes total sense as working with Fabric items like Lakehouses, Warehouses, Notebook, Pipelines, Dataflows, and also Semantic Models (renamed from ..read more

Here are the slides for my Fabric SQL Warehouse session at #DataWeekender 6.5. Download PDF now. Session Slideshow You can browse the session slides in the image gallery below ..read more