Snowflake
Catalog Snowflake databases, schemas, tables, views, columns, procedures and UDFs, with SQL lineage from views, routines and query history.
Capabilities
| Capability | Support | Notes |
|---|---|---|
| Tables, views & columns | Supported | |
| Keys, constraints & indexes | Partial | Primary, unique and foreign keys are informational in Snowflake (not enforced) but are imported. |
| Procedures & functions | Supported | |
| Data lineage | Supported | |
| Lineage from query history | Supported | Statements collected from SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY. |
| Business glossary links | Supported | |
| Data quality probes | Supported | |
| Data profiling | Supported | |
| Data classification | Supported | |
| Chat with your data | Supported |
Connection settings
Create the connection on the Blindata Agent from Settings > Agents > Connections with the connection wizard, choosing Snowflake in the technology step.
| Driver class | net.snowflake.client.jdbc.SnowflakeDriver |
|---|---|
| JDBC URL | jdbc:snowflake://<account>.snowflakecomputing.com/ |
| Authentication |
Service principal (shared technical user) User principal (personal credentials, BYOC) Ephemeral credentials (asked at each run)
|
| Crawling strategy | SQL |
Connection properties
| Property | Description |
|---|---|
warehouse | Virtual warehouse used to run extraction queries (optional, can be set in the URL). |
db | Default database (optional, can be set in the URL). |
schema | Default schema (optional, can be set in the URL). |
role | Role used by the session (optional). |
The Blindata Agent does not include this driver: download the vendor JDBC driver and copy the .jar into the agent drivers folder, as described in
Add JDBC drivers.
Driver properties can be added under Advanced settings in the connection wizard.
Prerequisites & permissions
- A Blindata Agent installed with network access to Snowflake.
- The vendor JDBC driver installed on the agent (see Add JDBC drivers).
- A dedicated role with
USAGEon the warehouse, databases and schemas to document. REFERENCESon tables and views lets the role see metadata without reading data (GRANT REFERENCES ON ALL TABLES IN SCHEMA ...); addFUTUREgrants to keep up with new objects.IMPORTED PRIVILEGESon theSNOWFLAKEdatabase for query history (ACCOUNT_USAGE).SELECTon tables for data quality, profiling, classification and Chat with your data.- Key-pair authentication is recommended for service users: set
private_key_filein the driver properties and attach the key file.
Set up metadata crawling
- Create the connection to Snowflake on the agent (see Connection settings) and use Test to verify it.
- Create or open the system that represents Snowflake in the catalog and link it to the connection.
- In the CRAWLING tab add a crawling job with strategy SQL and, if needed, replace the default queries with the ones suggested below.
- Run a TEST to check the number of extracted objects, then RUN or schedule the job.
The job options and the column aliases of every extraction query are described in the Generic JDBC reference.
Suggested extraction queries
In Snowflake each database has its own INFORMATION_SCHEMA. Set the db connection property (or the database in the URL) and create one crawling job, linked to one Blindata system, per database to document. The SQL strategy is recommended: it is faster than reading metadata through the JDBC driver on large accounts.
Tables query
SELECT
table_schema,
table_name,
table_type,
comment AS description
FROM information_schema.tables
WHERE table_schema <> 'INFORMATION_SCHEMA'
Columns query
SELECT
table_schema,
table_name,
column_name,
data_type,
ordinal_position
FROM information_schema.columns
WHERE table_schema <> 'INFORMATION_SCHEMA'
Physical constraints
Snowflake does not expose key columns in INFORMATION_SCHEMA. Use the JDBC strategy, which reads primary and foreign keys through the driver, or leave the constraints query empty.
Routines query
Stored procedures and SQL user-defined functions. The argument signature is appended to SPECIFIC_NAME so that overloaded routines stay distinct.
SELECT
procedure_name || argument_signature AS SPECIFIC_NAME,
procedure_catalog AS ROUTINE_CATALOG,
procedure_schema AS ROUTINE_SCHEMA,
procedure_name AS ROUTINE_NAME,
'PROCEDURE' AS ROUTINE_TYPE,
procedure_definition AS ROUTINE_DEFINITION,
comment AS ROUTINE_COMMENT,
created AS CREATED,
last_altered AS LAST_ALTERED
FROM information_schema.procedures
UNION ALL
SELECT
function_name || argument_signature,
function_catalog,
function_schema,
function_name,
'FUNCTION',
function_definition,
comment,
created,
last_altered
FROM information_schema.functions
WHERE function_language = 'SQL'
Lineage from query history
The agent can periodically extract executed statements from Snowflake and send them to Blindata, where the SQL lineage engine turns them into dataflows. See the ready-made extraction query in Scheduling and statements extraction.