Data lineage reconstruction

The connector publishes lineage as data flows between the objects in your Blindata catalog: from a source table, file or column to a target one. Each flow names the Fabric item that moved the data, such as the pipeline, notebook or dataflow, so you can see both where data goes and what moves it.

Lineage from all Fabric products is joined into one graph, across workspaces, from the systems that feed Fabric to the Power BI content your users open.

The example below shows a typical setup. The top part is what exists in Fabric: a pipeline copies orders from an external Azure SQL database into a Bronze lakehouse, a notebook cleans them into Silver, a Gold warehouse view exposes them, and a semantic model in another workspace feeds a report. The bottom part is what Blindata rebuilds from it: each table, view and model column, linked column by column, with the pipeline, notebook or model that moved the data named on every flow.

What a data flow tells you

Open a data flow in Blindata to see:

Descriptions only contain object names. Parameter values, literals from code and the content of configuration tables are never written to Blindata.

Lineage by product

Data Factory

Data Engineering

Data Warehouse and Databases

Real-Time Intelligence

Power BI

Column lineage into semantic models follows products.powerBi.columnLineage : by default, every model column is linked to the source column with the same name, or to the actual source column when Fabric reports it.

Graph

Across items and workspaces

Fabric’s item relations add item-level links between items in the same workspace, such as a report and the semantic model it uses. A relation is published only when both items are cataloged as physical entities. Lakehouses, warehouses and other storage items are represented by their system, not by an entity, so relations to them (such as a notebook’s data-source link to a lakehouse) are not published. The lakehouses a notebook is attached to are listed on the notebook entity instead. Lineage crosses workspaces and products whenever the target or source is cataloged in Blindata, for example a notebook in one workspace writing to a lakehouse in another.

Levels of detail

Every flow carries an evidence level, from the most to the least precise. When several sources describe the same flow, the most precise one is kept.

Every column-level flow is accompanied by the table-level flow between the same tables.

Pipelines with dynamic content

Many pipelines get their source and target names from parameters, variables or a configuration table (a metadata-driven pipeline). Those names are not in the pipeline definition, so the connector can only publish them if it sees them in run history.

With runtime lineage enabled, the connector reads recent pipeline runs and publishes the flows they actually performed. For example, a pipeline that reads a list of tables from meta.copy_tasks and copies each of them in a ForEach produces:

• Flows performed by a child pipeline are attributed to the child, not to the parent.
• Flows with literal names are published even if the pipeline never ran.
• Without run history, the dynamic endpoints are reported as gaps (UNRESOLVED_PARAMETER) instead of being guessed.

Choose how many runs to read per pipeline (the last N successful runs, or all runs of the last N days) in the job configuration .

Notebook code lineage

With notebook code analysis on (products.dataEngineering.notebookLineage), the connector reads the code of Spark and T-SQL notebooks, without running it, and publishes the tables and columns each notebook reads and writes.

What is recognized

Table names built from notebook parameters are filled in with the values the notebook received from its pipeline runs (with runtime lineage on), from a calling notebook, or from the parameters cell. Writes inside an if the connector cannot decide are published and marked conditional=true.

Example. A notebook nb_bronze_copy, run by a pipeline for the tables customers and orders:

landing = spark.table(f"lh_landing.{source_table}")
landing.write.mode("append").saveAsTable(f"lh_bronze.{target_table}")
if landing.dropDuplicates(["id"]).count() < landing.count():
    landing.write.mode("append").saveAsTable(f"lh_bronze.{target_table}_error")

publishes:

lh_landing.customers → lh_bronze.customers         TABLE + columns
lh_landing.customers → lh_bronze.customers_error   TABLE + columns   conditional
lh_landing.orders    → lh_bronze.orders            TABLE + columns
lh_landing.orders    → lh_bronze.orders_error      TABLE + columns   conditional

Not analyzed: notebooks running on the plain Python kernel, Scala and R cells, shell commands, and libraries other than Spark and Delta (for example pandas or Polars). They are listed in the run report as not analyzed.

Keeping lineage up to date

• Each run republishes the lineage it finds, so flows follow your definitions as they change.
• Flows that no longer exist stay in Blindata until a cleanup removes the content not updated for the number of days you choose.
• Flows from run history stay current as long as the run window covers each pipeline’s schedule. For a monthly pipeline, use the default last successful run window rather than a window of a few days.

Why is my lineage missing?

The run report lists every flow that could not be published, with the reason. The most common ones:

Updated on 13 Oct 2026