Microsoft Fabric created a stir in the data management community when it became available on May, 23rd in 2023. Our main question was: can the new service live up to the hype? At Stratify, we tested Microsoft Fabric to see if it is competitive. Have a look at our conclusions.
Our verdict in a nutshell: Microsoft Fabric has the potential for greatness. But it falls short in critical functionalities, in maturity, and in cost competitiveness.
The following review doesn´t aim to be exhaustive, but rather focuses on key elements that led to our conclusion.
What is it?
Microsoft Fabric is Microsoft’s new data management solution flagship. It consists of several independent but integrated services that aim to cover most of the day-to-day data management requirements. Like its predecessor – Azure Synapse Analytics – it offers a variety of functionalities around data extraction, processing, storage, automation, and visualization.
Microsoft Fabric’s architecture is designed and built for the cloud. Its core functions cannot be implemented on-premises. In exchange it offers previously unavailable functionalities and improved scalability and flexibility. You can cherry-pick applications and functions including PowerBI without having to commit to the entire portfolio. Microsoft built all services on top of the same “serverless compute” layer. Using Microsoft Fabric functionalities requires pre-purchased capacity for this engine. A simple comparison would be the prepaid card of an arcade hall. The customer pays upfront but has the freedom to choose which games he wants to play.
Pricing
The pricing of Microsoft products has always been very difficult to understand. Microsoft Fabric is no exception to this rule. PowerBI alone has a significant amount of licensing and subscription models. Added to this are different data storage, data processing and other auxiliary services, that make it even harder for the average customer to select services and understand and control costs. The below graph shows a pricing table at the moment this article is written. Note that this is not information provided by Microsoft, but by third parties who made an effort to understand and explain Microsoft Fabric pricing. As a rough estimate, a common medium-sized company with a couple of standard PowerBI reports in each department, plus maybe one or two data science use cases would require at least an F32 solution, and more likely the F64. The “Hourly” option would not be of much use, meaning the minimum monthly cost for an average organization would be somewhere between USD 4.000 and USD 8.000.
The limitations of the “Hourly” option are one of the key shortcomings of Microsoft Fabric compared to industry standard solutions with true variable pricing models (e.g., Snowflake). Microsoft Fabric requires pre-purchased capacity. Once purchased, one can turn this capacity on and off, and only pay for the seconds the capacity is turned on. However, turning on and off capacity has to be done manually. This is a real showstopper. Using capacity only when needed might work for one well versed user during a testing or development phase. But it becomes unmanageable in a multi-user, multi-use-case, production environment, with scheduled overnight and weekends tasks. This, however, is a common reality for almost all organizations, hence the “Hourly” options is not a sustainable option.
Data Extraction and Processing
The most exciting functionality for us was the announcement of “Direct Lake”. It is a new form of loading data in PowerBI at an unprecedented speed. At first, our tests did not disappoint. We went through the somewhat tedious setup as we had to store data in a specific way within Microsoft Fabric. We then achieved impressive loading speed under certain circumstances. Unfortunately, the emphasis is on “under certain circumstances”.
The most important limitation we encountered is that this kind of connection only works with the browser version of PowerBI. However, the browser version that enables this connection does not allow to undo actions (Ctrl+Z). Accidental deletes of a measure, or any other unintended changes, cannot be undone. For that reason alone, it’s very difficult to imagine anyone with mileage in working with PowerBI even considering this option.
In addition, a “Direct Lake” connection does not work for all data types. Microsoft is not clear or explicit as to what that entails. This means that a user could face a case where a specific data field is not supported. He would have to revert back to excruciatingly slow loading times.
A third issue is that the “Direct Lake” connection will fall back to a “Direct Query” connection under certain circumstances. Microsoft mentions large data volumes is not specific on the exact details. This causes a few concerns, one of which is pricing. Direct Queries require significant compute capacity, as opposed to a Direct Lake connection. This means that a low-cost / high-speed solution can turn into the exact opposite. Without the user knowing when or why it happens.
A fourth issue is related to PowerBI Measures created in Microsoft Fabric. They do not port into the PowerBI Desktop version of a report. If a report requires extensive usage of metrics, users face a copy/paste challenge during development. Alternatively, they must use the browser version of PowerBI, with the limitations mentioned previously.
Finally, data extraction and processing in Microsoft Fabric are based on a consumption-based pricing model. Data extraction and transformation logics consume pre-purchased capacity. The model is not uncommon, but it is challenging to manage and control. This aspect alone has the potential to increase the previously mentioned cost estimates significantly. More advanced and mature solutions such as EasyMorph are available. They offer fixed annual licensing models at a much lower cost than Microsoft Fabric.
While the list of perceived shortcomings might be a showstopper today, we fully expect Microsoft to overcome those in time. In addition, there are noteworthy positive aspects of Microsoft Fabric regarding Data Extraction and Processing.
For one, you can now use parameters in data flows. It’s a seemingly small thing that unlocks a lot of data transformation possibilities that were previously unavailable. Another welcome addition is the integration of the Azure Monitoring Hub as a solution to monitor scheduled logics.
Data Storage
The data storage solution is where Microsoft Fabric has made the most significant progress. Especially compared to its predecessor Azure Synapse Analytics. By completely separating the storage and compute layers, three new features stood out to us.
First and foremost, it’s not necessary any more to continuously partition and index stored data. Data loading and consumption is a lot easier. Plus, adjusting compute capacity now requires just a few clicks. It removes the necessity to have an experienced database administrator managing your analytical storage solution.
In addition, the separation between storage and compute enables two very handy functionalities.
One is “Zero Copy Cloning”, where infinite copies of large data sets can be created in seconds. Without requiring redundant physical copies. It makes data sharing a lot simpler and much less risky.
The other functionality is “Time Travel” of data. It´s now possible to go back in time on an object such as a specific table. This allows to undo faulty queries that scramble data. Manual backups are either much less critical or not necessary at all anymore.
While these are praise-worthy improvements, they don’t come with a downside.
For one, Microsoft incorporated a new, proprietary query language called “KQL”. It does not replace SQL, Dax, M, Python, PowerShell or Visual Basic. This is a tricky issue. It deepens the gap between a business-oriented analyst and whatever he needs to accomplish with his data. For us, a key criterion in the tool selection is whether an average analyst can become self-sufficient in less than a month. Adding yet another proprietary query language to the mix makes this unlikely. Have a look at our take on low-code/no-code applications.
In addition, functionalities like Time Travel or Zero Copy Cloning have been around for about a decade now (e.g. Snowflake). Our tests showed that leading industry standard solutions are far more polished and mature in that regard. It is no small feat to have these functionalities incorporated in Microsoft Fabric. But it merely reduces a previously generational gap to a significant gap when compared to its competitors.
Data Visualization
For us, Microsoft PowerBI has been and continues to be the mainstream market leader in data visualizations. Microsoft Fabric doesn´t change that, but it also doesn´t add to it.
However, we are hoping that PowerBI will eventually be able to process large datasets fast. Anyone who had to wait half an hour to update a PowerBI report can probably relate to that. Other tools such as Thoughtspot are at the forefront on this aspect. It´s likely just a question of time until PowerBI or a similar Microsoft service will take care of this.
Summary
Microsoft Fabric lays the foundation for a promising future. It’s an important step in the right direction. To us, the vision behind it is crystal clear and compelling. However, apart from PowerBI, the offered functionalities are not yet comparable to leading industry standard solutions. In addition, it comes at price and complexity level that – in comparison – is significantly more elevated. We believe that Microsoft Fabric has the potential to greatness. But we don’t see a compelling reason to commit to a solution that still is:
- Inferior in functionalities,
- More complex to handle, and
- More expensive than the available alternatives.
It will be interesting to see if the brand name alone is enough for Microsoft Fabric to gain the momentum and market share needed. Microsoft will have to close a gap that will likely require a few more years of development and polishing.
