Microsoft Fabric connector

Configure the Fabric connector

Create the Microsoft Fabric connection and job in the Blindata Agent, with every configuration option explained

This page walks you through the connection, the job, and then every field of the job configuration, from top to bottom. The job type is FABRIC.

Before you start, complete Prepare your Fabric tenant : you need the application’s client ID, client secret and tenant ID.

Create the Fabric connection

Create the connection in the Blindata Agent, on the agent that will run the job. Open the agent, go to Connections, create a connection of type Microsoft Fabric, and fill in the properties below. Use the connection name as the value of connectionName in the job.

Property Required Default Description
clientId Yes none Application (client) ID of the Entra app registration.
pwd Yes none Client secret value of the app registration. Blindata treats pwd as a secret property.
tenantId Yes none Directory (tenant) ID.
loginUrl No https://login.microsoftonline.com Entra login endpoint. The token URL is {loginUrl}/{tenantId}/oauth2/v2.0/token.
fabricApiUrl No https://api.fabric.microsoft.com/v1 Fabric REST API base URL.
oneLakeUrl No https://onelake.dfs.fabric.microsoft.com OneLake (ADLS Gen2 style) endpoint.
powerBiApiUrl No https://api.powerbi.com/v1.0/myorg Power BI REST API base URL. If the value does not end with /v1.0/myorg, the agent appends it.
sqlLoginTimeoutSeconds No 30 SQL endpoint login timeout, from 1 to 300 seconds.
sqlQueryTimeoutSeconds No 120 SQL endpoint query timeout, from 1 to 300 seconds.

Leave the URL properties empty unless you have a reason to change them. The token scopes are fixed to the Azure public cloud, so sovereign clouds (for example China or US Government) are not supported out of the box: contact Blindata support before pointing the connector at one.

How the agent authenticates

For each run, the agent takes the three values clientId, pwd and tenantId from the connection. It requests a token with the OAuth 2.0 client credentials grant, once per resource (Fabric API, OneLake, SQL endpoint, Power BI). It caches each token until about one minute before it expires. No interactive sign-in and no user account is involved.

The connection is used only to read Fabric. Writes go to Blindata through a separate connection, named in outputOptions.targetConnectionName, which must be a Blindata connection (type BLINDATA). That connection is used only when dryRun is false.

Create the job

  1. Open the agent, go to Schedules, and create a schedule with job type FABRIC. Set the cron expression and the time zone.
  2. Paste the job configuration (JSON, below) as the job’s configuration.
  3. Save the schedule. Run it, then check its log in the agent’s Runs tab.

The job configuration is a JSON object. Unknown fields are ignored without an error (see Common mistakes ).

Job configuration parameters

The job configuration has these top-level fields. Only connectionName and dataFlowScope are always required.

Field Type Default Required Description When to change
connectionName string none Yes Name of the Fabric connection on this agent. Always set.
dataFlowScope string none Yes Scope written on every data flow this job creates. The post-run cleanup also uses it. Use a value used by this job only.
discovery object see below No Which workspaces and items are read. To limit scope or switch to workspace-scoped discovery.
capabilities object see below No Which item types and metadata are read. To turn off features you do not need.
catalogFilters object empty No Include and exclude rules for schemas, tables and views. To catalog a subset of the schemas or tables.
systemNaming object see below No How Fabric items become Blindata Systems. To change System names or pin items to existing Systems.
connectionMappings array [] No Links external Fabric connections to Systems already in Blindata. When pipelines or shortcuts read from or write to systems you already catalog.
products object all enabled No Per-product switches and the Systems used for each product. To disable a product or to change its output Systems.
limits object see below No Parallelism, retries and the admin request budget. Only when Fabric throttles the run.
outputOptions object dry run No Where to upload the result, and whether to upload at all. To move from a dry run to a live upload.
cleanupOptions object none No Removes stale content after a full upload. To enable post-run cleanup.

Omitting outputOptions makes the job a dry run.

discovery

Field Type Default Description When to change
mode string TENANT_ADMIN TENANT_ADMIN lists every workspace through the admin APIs. WORKSPACE_SCOPED lists only the workspaces the service principal can see. The value is case-insensitive. Any other value is treated as TENANT_ADMIN. Set WORKSPACE_SCOPED when the read-only admin APIs are not granted.
fallbackToWorkspaceScoped boolean true If the first admin workspace listing is denied (401 or 403), the job lists workspaces again with workspace-scoped APIs and adds a warning to the result. Set false to make the denial a failure.
workspaceIds string[] [] If not empty, only these workspace IDs are read. To read a handful of workspaces.
workspaceNameIncludePatterns regex[] [] A workspace is read only if its name matches at least one pattern. To select workspaces by name.
workspaceNameExcludePatterns regex[] [] A workspace whose name matches any pattern is skipped. To drop sandbox or personal workspaces.
itemNameIncludePatterns regex[] [] An item is read only if its display name matches at least one pattern. To keep only some items.
itemNameExcludePatterns regex[] [] An item whose display name matches any pattern is skipped. To drop temporary or test items.
itemTypesInclude string[] [] If not empty, only items of these Fabric types are read. Exact, case-sensitive type names, such as Lakehouse, Warehouse, DataPipeline. To read only some product types.
itemTypesExclude string[] [] Items of these Fabric types are skipped. To skip a type you do not want.
itemNameFiltersByType object[] [] Extra name filters for one item type. Each entry has itemType (required), includePatterns and excludePatterns. They apply in addition to the global item name filters. For example, keep only the lakehouses whose name starts with Gold.
modifiedSinceDays integer none Only items and workspaces modified in the last N days, with N from 1 to 30. In TENANT_ADMIN mode it uses the admin “modified workspaces” API. In WORKSPACE_SCOPED mode it does not narrow the workspace list. Any value makes the run incremental, so cleanup does not run. For a nightly run that only needs recent changes.

Name patterns are Java regular expressions. They are case-insensitive, and they match when the pattern is found anywhere in the name (a substring match). Use ^ and $ to anchor a pattern to the whole name.

capabilities

Field Type Default Description When to change
dataItems boolean true Lakehouses, warehouses, SQL databases, mirrored databases, and their tables and columns. Used as the default for the data item switches under products. Set false to skip all data item catalogs.
files.enabled boolean true Lists the Files/ area of each lakehouse. Set false to skip file listings.
files.maxDepth integer 3 Folder depth to list under Files/. The root is depth 0. Raise it only if files are deeper than 3 levels.
files.includePatterns regex[] [] If not empty, a file path must match at least one pattern. Matched against the path relative to Files/. To list only some folders.
files.excludePatterns regex[] [] Paths that match any pattern are skipped. To skip folders such as _temp.
files.maxPathsPerItem integer 5000 Maximum paths listed per lakehouse. When the cap is reached the listing stops and the affected entities are marked as truncated. Raise it for lakehouses with more files.
sqlEndpointEnrichment boolean true Opens the SQL endpoint to read T-SQL column types and to enable the children below. When false, no SQL connection is opened for these catalogs. Set false when the SQL endpoint is not reachable.
sqlViews boolean true Reads SQL views and their definitions. Only with sqlEndpointEnrichment. Set false to skip views.
sqlRoutines boolean true Reads warehouse procedures and functions. Only with sqlEndpointEnrichment. Set false to skip routines.
sqlColumnTypes boolean true Replaces Spark column types with T-SQL types from the SQL endpoint. Only with sqlEndpointEnrichment. Set false to keep Spark types.
shortcuts boolean true Reads OneLake shortcuts of lakehouses. Set false to skip shortcuts.
pipelines boolean true Reads data pipelines and their definitions. Set false to skip pipelines.
copyJobs boolean true Reads Copy jobs and their definitions. Set false to skip Copy jobs.
notebooks boolean true Reads notebooks. Set false to skip notebooks.
itemRelations boolean true Reads item-to-item relations inside the workspace (the Fabric relations APIs or the admin scanner). Set false to skip relations (faster, but less lineage).
dataflowsGen2 boolean true Fetches Dataflow Gen2 definitions. Each query becomes an entity with table, path and item lineage. When false, dataflows are inventory only. Set false for inventory only.

The product switches under products fall back to these capability values when they are not set. For example, products.dataFactory.pipelines falls back to capabilities.pipelines.

catalogFilters

Field Type Default Description
schemaIncludePatterns regex[] [] If not empty, a schema must match at least one pattern.
schemaExcludePatterns regex[] [] A schema that matches any pattern is skipped, with all its tables and views.
tableIncludePatterns regex[] [] If not empty, a table must match at least one pattern.
tableExcludePatterns regex[] [] A table that matches any pattern is skipped.
viewIncludePatterns regex[] [] If not empty, a view must match at least one pattern.
viewExcludePatterns regex[] [] A view that matches any pattern is skipped.

These rules are different from the discovery filters. Here a pattern must match the whole name (a full match), and the match is case-insensitive. A pattern without a dot matches the object name only in the item’s default schema (dbo unless the item reports another one). A pattern with a dot matches schema.name. Example: "tableIncludePatterns": ["sales\\..*"] keeps every table of the sales schema.

The rules apply to lakehouses, warehouses, SQL databases, the SQL lineage, and the lineage resolution of the other products. A run with any catalog rule is never a full run, so cleanup does not run.

systemNaming

Field Type Default Description
pattern string <workspaceName> / <itemDisplayName> Name of the System created for each storage item, such as a lakehouse, warehouse, or SQL database. Placeholders: <workspaceName>, <itemDisplayName>.
queryBucketPattern string <systemName> (Fabric SQL) Name of the query bucket that holds the SQL statements of a System. Must contain <systemName>. Placeholder: <systemName>.
lowercaseIdentifiers boolean false Lower-cases the generated System names and the SQL identifiers.
pinnedSystems object[] [] Each entry (fabricItemId, systemName, systemUuid) pins one Fabric item to an existing Blindata System, instead of the name from pattern.

The product Systems (for example Microsoft Fabric Data Engineering) are not named here: set them under products (see below).

connectionMappings

Use this list to connect Fabric pipelines and shortcuts to systems you already catalog in Blindata (for example an Azure SQL database or a Snowflake account). The job never creates these systems. Each entry has these fields:

Field Type Description
fabricConnectionId string Fabric connection ID to match.
fabricConnectionDisplayName string Fabric connection display name to match.
fabricConnectionType string Fabric connection type to match, used together with pathRegex.
pathRegex regex Matched (found anywhere) against the connection path.
blindataSystemName string Name of the existing Blindata System to link to.
blindataSystemUuid string UUID of the existing Blindata System. Use it instead of the name when names are not unique.
defaultSchema string Schema for unqualified names in queries that run on this connection. When empty, dbo is used for SQL databases.

Matching order: the first entry whose fabricConnectionId equals the connection ID wins. If none matches, the first entry whose fabricConnectionDisplayName equals the display name wins. If none matches, the first entry whose fabricConnectionType equals the type and whose pathRegex matches the path wins. An entry with only a type and no pathRegex never matches.

products

Each product has an enabled switch (default true) and its own options. When a product is disabled, its items are counted in the result and not read. A product’s output System can be renamed or pinned with its system object: {"name": "...", "uuid": "..."}. If name is empty, the default name below is used.

Items belong to products as follows:

Product Fabric item types
dataEngineering Lakehouse, Notebook, SparkJobDefinition, AzureDatabricksStorage
dataFactory DataPipeline, Dataflow, CopyJob, MountedDataFactory, ApacheAirflowJob, DataBuildToolJob, MirroredDatabase, SnowflakeDatabase
dataWarehouse Warehouse, SQLEndpoint, WarehouseSnapshot, MirroredWarehouse, MirroredAzureDatabricksCatalog, MirroredCatalog
databases SQLDatabase, CosmosDBDatabase
realTimeIntelligence Eventstream, Eventhouse, KQLDatabase
powerBi Dashboard, Report, SemanticModel, PaginatedReport, Datamart, OrgApp
graph GraphModel

Items of other types (for example Environment or MLModel) are counted as items outside the products and are not cataloged.

dataEngineering

Field Type Default Description
system object Microsoft Fabric Data Engineering Product System (SERVICE).
lakehouses boolean capabilities.dataItems Lakehouse tables and columns.
files boolean capabilities.files.enabled Files/ area listing.
shortcuts boolean capabilities.shortcuts OneLake shortcuts.
notebooks boolean capabilities.notebooks Notebooks.
sparkJobDefinitions boolean true Spark job definitions.
notebookLineage.enabled boolean false Enables notebook code analysis (PySpark, Spark SQL and T-SQL) for lineage. Needs notebooks not set to false.
notebookLineage.maxBindingsPerNotebook integer 500 Maximum table bindings kept per notebook, from 1 to 5000.

dataFactory

Field Type Default Description
system object Microsoft Fabric Data Factory Product System (SERVICE).
pipelines boolean capabilities.pipelines Data pipelines and their activities.
copyJobs boolean capabilities.copyJobs Copy jobs.
dataflowsGen2 boolean capabilities.dataflowsGen2 Dataflows Gen2.
activityEdges boolean true Lineage edges between pipeline activities (for example invoke-pipeline).
mirroredDatabases boolean capabilities.dataItems Mirrored databases.
snowflakeDatabases boolean capabilities.dataItems Snowflake databases mirrored in Fabric.
runtimeLineage object disabled Reads pipeline run history. See below.

runtimeLineage (opt-in):

Field Type Default Description
enabled boolean false Turns on run-history lineage.
window.mode string LAST_N_RUNS LAST_N_RUNS reads the most recent completed runs of each pipeline. SINCE_DAYS reads all finished runs (completed, failed, or cancelled) from the last N days.
window.runs integer 1 Number of completed runs per pipeline, from 1 to 100. Only for LAST_N_RUNS.
window.days integer none Number of days, from 1 to 30. Required for SINCE_DAYS and not allowed for LAST_N_RUNS.
maxActivityRunsPerPipeline integer 10000 Maximum activity runs read per pipeline, from 1 to 100000.
controlInputs boolean true Adds the runtime control inputs of activities to the lineage.
copyDefaultMapping boolean true Also resolves Copy activities that use the default column mapping, not an explicit one.

dataWarehouse: enabled, warehouses, snapshots, mirroredWarehouses, mirroredCatalogs. Each switch defaults to capabilities.dataItems.

databases: enabled, sqlDatabases, cosmosDbDatabases. Each switch defaults to capabilities.dataItems.

realTimeIntelligence: enabled, system (default Microsoft Fabric Real-Time Intelligence), eventstreams, eventhouses, kqlDatabases. Each item switch defaults to true. Eventstream and KQL database definitions are read only when their switch is on.

graph: enabled, system (default Microsoft Fabric Graph), graphModels (default true).

powerBi

Field Type Default Description
enabled boolean true Turns the Power BI product on. Requires the read-only admin APIs (see Prepare your Fabric tenant ).
reportsSystem object Power BI Reports Product System for reports.
datasetsSystem object Power BI Datasets Product System for semantic models and datasets.
onlyConsumersOfMappedDatasources boolean false Keeps only the reports and models that consume a datasource mapped in datasourceMappings.
analyzeReportsPbix boolean true Reads the report layout (PBIX export) to find the fields each visual uses.
dataflowsGen1 boolean true Gen1 dataflows.
datamarts boolean true Datamarts.
orgApps boolean true Organizational apps.
datasourceMappings object[] [] Links a Power BI datasource to an existing Blindata System. See below.
columnLineage string SAME_NAME SAME_NAME creates a column flow for every non-calculated column, using the Fabric source column when known, otherwise the column name. SOURCE_COLUMN_ONLY creates a column flow only when the Fabric source column is known.
fabricBridgeEnabled boolean true Resolves Power BI sources to the Fabric items this job creates.
resolveDirectLakeSources boolean true Resolves Direct Lake semantic models to their lakehouse or warehouse. Needs fabricBridgeEnabled.

datasourceMappings entries: blindataSystemName or blindataSystemUuid (the target), and one of: powerBIDataSourceId (with an optional powerBIDataSourceType), powerBIDataSourceType alone (matches every datasource of that type), or powerBIDataSourceRegex (matched against the serialized datasource JSON, as a whole match). The first entry in the list that matches a datasource wins, and it overrides the Fabric bridge for that datasource.

limits

Field Type Default Description When to change
maxParallelism integer 4 Maximum concurrent Fabric calls. Lower it when Fabric throttles the run (HTTP 429).
lroPollIntervalSeconds integer 5 Seconds between polls of a long-running operation (such as getDefinition). Rarely.
lroMaxWaitSeconds integer 300 Maximum seconds to wait for one long-running operation. Raise it for large definitions.
retryMaxAttempts integer 5 Attempts for a retryable call. Rarely.
adminRequestBudget integer 180 Maximum admin API requests in one run. Above it, admin calls fail with a throttling error. Raise it for very large tenants, or switch to workspace-scoped discovery.

outputOptions

Field Type Default Description
dryRun boolean true true reads Fabric and counts what would be uploaded. Nothing is written to Blindata.
targetConnectionName string none Name of the Blindata connection (type BLINDATA) that receives the upload. Required when dryRun is false.
defaultHidden string none Passed as the isHidden query parameter of the Blindata CSV upload. Leave it empty unless you need it.

cleanupOptions

After a successful upload, the job can delete content in each System it published that has not been updated for a number of days. The cleanup runs only when all the following are true:

  • daysFromLastUpdate is greater than 0.
  • The run is a full run: no modifiedSinceDays, no discovery filter, no catalog filter, all seven products enabled, and all SQL enrichment switches on.
  • The upload created or updated at least one entity.
Field Type Description
daysFromLastUpdate integer Number of days after which a published System that was not updated is cleaned up. The content removed inside a System is determined by Blindata.

The other fields of cleanupOptions (cleanupLevel, schemas, physicalEntitiesUuids, limit and the *DeletionEnabled flags) are passed to Blindata unchanged. The job sets the level to SYSTEM, the System UUID, and the data flow scope from dataFlowScope. The deletion flags are false when omitted. Blindata applies the other fields on its side. Set only daysFromLastUpdate unless you have reviewed how they are applied.

Full example

This example lists every top-level field. Remove what you do not need.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "discovery": {
    "mode": "TENANT_ADMIN",
    "fallbackToWorkspaceScoped": true,
    "workspaceIds": [],
    "workspaceNameIncludePatterns": [],
    "workspaceNameExcludePatterns": [],
    "itemNameIncludePatterns": [],
    "itemNameExcludePatterns": [],
    "itemTypesInclude": [],
    "itemTypesExclude": [],
    "itemNameFiltersByType": []
  },
  "capabilities": {
    "dataItems": true,
    "files": {
      "enabled": true,
      "maxDepth": 3,
      "includePatterns": [],
      "excludePatterns": [],
      "maxPathsPerItem": 5000
    },
    "sqlEndpointEnrichment": true,
    "sqlViews": true,
    "sqlRoutines": true,
    "sqlColumnTypes": true,
    "shortcuts": true,
    "pipelines": true,
    "copyJobs": true,
    "notebooks": true,
    "itemRelations": true,
    "dataflowsGen2": true
  },
  "catalogFilters": {
    "schemaIncludePatterns": [],
    "schemaExcludePatterns": [],
    "tableIncludePatterns": [],
    "tableExcludePatterns": [],
    "viewIncludePatterns": [],
    "viewExcludePatterns": []
  },
  "systemNaming": {
    "pattern": "<workspaceName> / <itemDisplayName>",
    "queryBucketPattern": "<systemName> (Fabric SQL)",
    "lowercaseIdentifiers": false,
    "pinnedSystems": []
  },
  "connectionMappings": [],
  "products": {
    "dataEngineering": {
      "enabled": true,
      "system": { "name": "Microsoft Fabric Data Engineering" },
      "notebookLineage": { "enabled": false, "maxBindingsPerNotebook": 500 }
    },
    "dataFactory": {
      "enabled": true,
      "system": { "name": "Microsoft Fabric Data Factory" },
      "activityEdges": true,
      "runtimeLineage": {
        "enabled": false,
        "window": { "mode": "LAST_N_RUNS", "runs": 1 },
        "maxActivityRunsPerPipeline": 10000,
        "controlInputs": true,
        "copyDefaultMapping": true
      }
    },
    "dataWarehouse": { "enabled": true },
    "databases": { "enabled": true },
    "realTimeIntelligence": {
      "enabled": true,
      "system": { "name": "Microsoft Fabric Real-Time Intelligence" }
    },
    "graph": {
      "enabled": true,
      "system": { "name": "Microsoft Fabric Graph" }
    },
    "powerBi": {
      "enabled": true,
      "reportsSystem": { "name": "Power BI Reports" },
      "datasetsSystem": { "name": "Power BI Datasets" },
      "onlyConsumersOfMappedDatasources": false,
      "analyzeReportsPbix": true,
      "dataflowsGen1": true,
      "datamarts": true,
      "orgApps": true,
      "datasourceMappings": [],
      "columnLineage": "SAME_NAME",
      "fabricBridgeEnabled": true,
      "resolveDirectLakeSources": true
    }
  },
  "limits": {
    "maxParallelism": 4,
    "lroPollIntervalSeconds": 5,
    "lroMaxWaitSeconds": 300,
    "retryMaxAttempts": 5,
    "adminRequestBudget": 180
  },
  "outputOptions": {
    "dryRun": true,
    "targetConnectionName": "blindata-prod"
  },
  "cleanupOptions": {
    "daysFromLastUpdate": 30
  }
}

1. First run: dry run over the whole tenant. Use the defaults and dryRun set to true. Check the result before you enable the upload.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "outputOptions": { "dryRun": true }
}

2. Nightly upload over the whole tenant. Upload, and remove content that has not been updated for 30 days.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "discovery": { "mode": "TENANT_ADMIN", "fallbackToWorkspaceScoped": false },
  "outputOptions": { "dryRun": false, "targetConnectionName": "blindata-prod" },
  "cleanupOptions": { "daysFromLastUpdate": 30 }
}

3. No tenant admin. Use workspace-scoped discovery when the read-only admin APIs are not granted. Disable the Power BI product, because it needs the admin scanner.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "discovery": { "mode": "WORKSPACE_SCOPED" },
  "products": { "powerBi": { "enabled": false } },
  "outputOptions": { "dryRun": false, "targetConnectionName": "blindata-prod" }
}

4. One domain. Read only the workspaces whose names start with Sales, skip personal and test items, and keep the workspace-level inventory.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-sales",
  "discovery": {
    "mode": "TENANT_ADMIN",
    "fallbackToWorkspaceScoped": false,
    "workspaceNameIncludePatterns": ["^Sales "],
    "itemNameExcludePatterns": ["^tmp_", "_test$"]
  },
  "outputOptions": { "dryRun": false, "targetConnectionName": "blindata-prod" }
}

5. Incremental nightly run. Read only what changed in the last day. Cleanup does not run in an incremental run, so keep a separate full run for cleanup.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "discovery": {
    "mode": "TENANT_ADMIN",
    "fallbackToWorkspaceScoped": false,
    "modifiedSinceDays": 1
  },
  "outputOptions": { "dryRun": false, "targetConnectionName": "blindata-prod" }
}

6. Lineage to systems you already catalog. Link a Fabric connection that points to an Azure SQL database already in Blindata. The job does not create that System.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "connectionMappings": [
    {
      "fabricConnectionId": "<fabric-connection-id>",
      "blindataSystemName": "Orders Source Azure SQL"
    }
  ],
  "outputOptions": { "dryRun": false, "targetConnectionName": "blindata-prod" }
}

7. Code lineage and run history. Turn on notebook code analysis and pipeline run history. Run history costs more Fabric calls, so keep the window small.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "products": {
    "dataEngineering": {
      "notebookLineage": { "enabled": true, "maxBindingsPerNotebook": 500 }
    },
    "dataFactory": {
      "runtimeLineage": {
        "enabled": true,
        "window": { "mode": "SINCE_DAYS", "days": 7 }
      }
    }
  },
  "outputOptions": { "dryRun": false, "targetConnectionName": "blindata-prod" }
}

8. Power BI with an explicit datasource mapping. Link a Power BI SQL datasource to an existing System, so that the Power BI lineage points to the right table.

{
  "connectionName": "fabric-prod",
  "dataFlowScope": "fabric-prod-catalog",
  "products": {
    "powerBi": {
      "datasourceMappings": [
        {
          "blindataSystemName": "Corporate Analysis Services",
          "powerBIDataSourceId": "<datasource-id>"
        }
      ]
    }
  },
  "outputOptions": { "dryRun": false, "targetConnectionName": "blindata-prod" }
}

Validation errors

The job rejects the configuration before it reads Fabric when:

  • connectionName or dataFlowScope is empty.
  • dryRun is false and targetConnectionName is empty.
  • discovery.modifiedSinceDays is greater than 30.
  • A discovery or catalog regex does not compile.
  • systemNaming.queryBucketPattern does not contain <systemName>.
  • itemNameFiltersByType has an entry without itemType.
  • notebookLineage.enabled is true while notebooks is false, or maxBindingsPerNotebook is outside 1 to 5000.
  • runtimeLineage.window.mode is not LAST_N_RUNS or SINCE_DAYS, or the window field that does not belong to the mode is set, or days is missing or outside 1 to 30 for SINCE_DAYS, or runs is outside 1 to 100 for LAST_N_RUNS.
  • runtimeLineage.maxActivityRunsPerPipeline is outside 1 to 100000.

Common mistakes

  • Unknown fields are ignored. The job does not fail on a misspelled key, so a filter that you typed wrongly is simply not applied. Check the result counts after the first run.
  • An invalid discovery.mode falls back to TENANT_ADMIN. Only WORKSPACE_SCOPED (any case) selects workspace-scoped discovery.
  • Discovery filters are substring matches, catalog filters are whole-name matches. "^Sales" matches Sales and Sales Gold. The pattern Sales matches Old Sales Archive too. In catalog filters the same pattern matches only the whole name.
  • A product left enabled still calls the admin scanner. The Power BI product calls it in both modes. Disable products.powerBi when the admin APIs are not granted.
  • Cleanup does not run on filtered or incremental runs. A run with any filter, any disabled product, or modifiedSinceDays is not a full run.

Updated on 13 Oct 2026