Incremental refresh is a high-demand option in Power BI. Microsoft already provided it for the Premium capacities, but for the Pro accounts it is still in waiting list.
However, with introduction of dataflows in Power BI Service, an incremental refresh implementation becomes available for Pro accounts too.
I won’t to describe dataflows in detail here since there is a lot of blogs and resources about it (but I’ll provide a few links in the bottom of the post).
All you need to know now on how to implement an incremental refresh, is that
- Dataflow in Power BI Service is a set of web-based Power Query queries (named as ‘entities’).
- Each dataflow could be refreshed manually or by the its own schedule.
- The result of evaluation of a dataflow’s entities then stored in Azure Data Lake Storage Gen2 as tables (more precisely as CSV files).
- Then you can use dataflows (their entities) as a data sources in your Power BI dataset.
Let’s start from this point.
What is the incremental refresh at all? In simple words, it means that in the single data import action we are refreshing (updating) only the part of data instead of loading all the data again and again. In the other words, we are dividing data in two parts (partitions): first part does not need refresh and should remain untouched, second part must be refreshed to bring in updates and corrections.Continue Reading →