Data Lineage and Power BI: Why is it important?

The Microsoft Power BI team have released a preview Data Lineage feature and it is a good start for organizations who are starting to think about data management. The Power BI lineage view displays the lineage relationships between all the artifacts in a workspace, and all its external dependencies.

Businesses need a clear line of sight on data asset ownership and stewardship. This has traditionally been in the hands of the IT department, but unfortunately the projects can become deprioritized as IT fight other battles elsewhere. IT can do so much, but the business do need to bring things to the table. Hence, we now see accessible tools appearing in technology such as Power BI.

In this blog post, I’ll cover the main reasons why this is important, and why every self-respecting Power BI Business Intelligence really needs to care about the issues involved.

Data lineage has always been important but there is renewed attention on it. With CCPA and GDPR, organizations are beginning to grasp that data is an asset that they need to look after. They also need to consider their ROI over their data; their Risk of Incarceration (thank you to Karen Lopez for that one!). It is part of a wider strategy known as data governance.

What is data governance, anyway?

Without data governance, it is hard to have robust business intelligence, data science and artificial intelligence. Even the blockchain is data. Data Governance permeates all of the buzzwords, whether organizations recognise it yet or not.

What are the benefits of data lineage?

Prevent rework – Often, organizations can take the decision to rework rather than trust data assets from previous projects. Data lineage can help identify the data sources which are being used, and it becomes easier to track them over time. This means that the data becomes more trustworthy.

Data accessibility and democratization – Data lineage enables Data Democratization, but a key part of democratization is data accessibility. Simply put, this just means giving people access to their data!

Productivity – Developers can see the root of their issues better, which means that they can solve problems more quickly.

Minimize bottlenecks – reduce Subject Matter Expert bottlenecks by increasing business user and analyst efficiency

Laying a foundation for future data science, AI and ML work – if your business intelligence isn’t in place, then efforts to move into data science, AI and ML will be harder to sustain.

What Data Lineage does not do

Data Lineage does not reduce staff. The reality is that the more data you give away, the more people ask you for. Help them to understand the data.

“I don’t need any more reports or data now, thanks!” – said nobody, ever. Your job is quite safe.

Data Lineage does not look after itself. More often than not, data dictionaries and enterprise glossaries fall out of date.

The Schrodinger’s Cat paradox does not mean that ‘if you don’t look at it, it might be alright’. Nope, in this case, the cat is a goner.

How can you get started?

Tools do not solve problems. People do. You need to think about the processes wrapped around your data.

Here are some mind grenades to consider:

• What problems are you trying to solve?
• What does success look like to the organization?
• Are the objectives tied closely to the enterprise vision, mission and goals; or is data optional (hard as that is to believe?).

People need to be educated about ownership of data. Keep asking ‘why do you need that?’ in order to understand how it ties to business objectives.

What else can you do when you start to consider data governance and data lineage?

Get enterprise executive sponsorship, not just line of business support. Data issues can be horizontal in nature; traversing data silos.

If the data is siloed, so is the analytics; but this does not mean that the problems are contained within those departmental jurisdictions. Data is the lifeblood of the business, and it’s circulation is crucial to understanding data lineage.

Work with partners across the business in order to help solve the issue of data debt, and help facilitate your efforts to become more sustainable and successful.

If you need any help, please get in touch.