Incremental Refresh-Power BI

Why do we need incremental refresh?

By offering automated partition formation and maintenance for dataset tables that often load new and updated data, incremental refresh procedures expand planned refresh activities. For the majority of datasets, this is one or more tables, similar to a fact table in a relational or star database structure, that hold transaction data, which changes often and can increase enormously. The amount of data that needs to be refreshed can be greatly reduced while still ensuring that even the most recent changes at the data source are included in the query results. This can be achieved by using an incremental refresh policy to partition the table, refreshing only the most recent import partition(s), and optionally utilising an additional DirectQuery partition for real-time data.                   

With the Zomato delivery system, I tried to make the incremental refresh notion simpler. The choice thus comes down to an incremental refresh vs. a full refresh. Consider Zomato’s operations while contemplating incremental refresh. I hope you might have ordered with Zomato at-least once

An order may enter a number of states after being placed, including new order, in progress, pending, and closed, before quickly closing. Technically speaking, we comprehend that the entire procedure should be completed in around an hour for a business like Zomato where it is important to deliver on time. Sometimes they even offer you a free meal if the item is not delivered on time

 

To be safe, we can prolong this for 24 hours (for example resolving any escalation or so). Therefore, you can forgo duplicating the static data and switch to incremental mode if you are running a reporting environment where you need to duplicate this data on a daily basis.

Please watch the entire video to see a variety of additional examples that should make it easier for you to understand the complex idea of incremental refresh; we’ll shortly present the implementation portion of this video in the second part of this series, hence stay tuned

Ritesh Bisht

#1 in the world to be Tableau Ambassador and Power BI Super User