Databricks

Capabilities

Connection settings

Create the connection on the Blindata Agent from Settings > Agents > Connections with the connection wizard, choosing Databricks in the technology step.

Connection properties

The Blindata Agent does not include this driver: download the vendor JDBC driver and copy the .jar into the agent drivers folder, as described in Add JDBC drivers. Driver properties can be added under Advanced settings in the connection wizard.

Prerequisites & permissions

• A Blindata Agent installed with network access to Databricks.
• The vendor JDBC driver installed on the agent (see Add JDBC drivers).
• A service principal (or user) with USE CATALOG and USE SCHEMA on the objects to document, plus BROWSE (or SELECT) to see table metadata.
• CAN USE on the SQL warehouse.
• SELECT on system.query.history for query history.
• SELECT on tables for data quality, profiling and Chat with your data.

Set up metadata crawling

1. Create the connection to Databricks on the agent (see Connection settings) and use Test to verify it.
2. Create or open the system that represents Databricks in the catalog and link it to the connection.
3. In the CRAWLING tab add a crawling job with strategy SQL and, if needed, replace the default queries with the ones suggested below.
4. Run a TEST to check the number of extracted objects, then RUN or schedule the job.

The job options and the column aliases of every extraction query are described in the Generic JDBC reference.

Suggested extraction queries

Unity Catalog has a three-level namespace (catalog.schema.table). Create one Blindata system and one crawling job per catalog, and replace my_catalog in the queries below. Use the SQL strategy: the information_schema of a catalog lists only the objects the connection principal can access.

Tables query

SELECT
  table_schema,
  table_name,
  table_type,
  comment AS description
FROM my_catalog.information_schema.tables
WHERE table_schema <> 'information_schema'

Columns query

SELECT
  table_schema,
  table_name,
  column_name,
  full_data_type AS data_type,
  ordinal_position
FROM my_catalog.information_schema.columns
WHERE table_schema <> 'information_schema'

Physical constraints query

Informational primary and foreign keys:

SELECT
  tc.constraint_name AS name,
  tc.constraint_schema AS schema,
  CASE tc.constraint_type WHEN 'PRIMARY KEY' THEN 'PRIMARY_KEY' ELSE 'FOREIGN_KEY' END AS type,
  kcu.table_schema AS target_physical_entity_schema,
  kcu.table_name AS target_physical_entity_name,
  kcu.column_name AS target_physical_field_name,
  kcu.ordinal_position AS list_order,
  ref.table_schema AS referenced_physical_entity_schema,
  ref.table_name AS referenced_physical_entity_name,
  ref.column_name AS referenced_physical_field_name,
  NULL AS clause,
  rc.delete_rule AS delete_rule,
  rc.update_rule AS update_rule,
  NULL AS access_method
FROM my_catalog.information_schema.table_constraints tc
JOIN my_catalog.information_schema.key_column_usage kcu
  ON kcu.constraint_schema = tc.constraint_schema
  AND kcu.constraint_name = tc.constraint_name
LEFT JOIN my_catalog.information_schema.referential_constraints rc
  ON rc.constraint_schema = tc.constraint_schema
  AND rc.constraint_name = tc.constraint_name
LEFT JOIN my_catalog.information_schema.key_column_usage ref
  ON ref.constraint_schema = rc.unique_constraint_schema
  AND ref.constraint_name = rc.unique_constraint_name
  AND ref.ordinal_position = kcu.position_in_unique_constraint
WHERE tc.constraint_type IN ('PRIMARY KEY', 'FOREIGN KEY')

Routines query

SQL user-defined functions registered in Unity Catalog:

SELECT
  specific_name AS SPECIFIC_NAME,
  routine_catalog AS ROUTINE_CATALOG,
  routine_schema AS ROUTINE_SCHEMA,
  routine_name AS ROUTINE_NAME,
  routine_type AS ROUTINE_TYPE,
  routine_definition AS ROUTINE_DEFINITION,
  comment AS ROUTINE_COMMENT,
  created AS CREATED,
  last_altered AS LAST_ALTERED
FROM my_catalog.information_schema.routines
WHERE routine_schema <> 'information_schema'
  AND routine_body = 'SQL'

Related guides

• Metadata import with interface tables
• Data quality probes
• Automated SQL lineage
• Scheduling agent jobs

Updated on 16 Oct 2026