Amazon Redshift
Catalog Redshift provisioned clusters and Serverless workgroups: schemas, tables, views, columns, procedures and UDFs, with SQL lineage from views and query history.
Capabilities
| Capability | Support | Notes |
|---|---|---|
| Tables, views & columns | Supported | |
| Keys, constraints & indexes | Partial | Keys are informational in Redshift (not enforced) but are imported. |
| Procedures & functions | Supported | |
| Data lineage | Supported | |
| Lineage from query history | Supported | Statements collected from SYS_QUERY_HISTORY / STL_QUERYTEXT. |
| Business glossary links | Supported | |
| Data quality probes | Supported | |
| Data profiling | Supported | |
| Data classification | Supported | |
| Chat with your data | Supported |
Connection settings
Create the connection on the Blindata Agent from Settings > Agents > Connections with the connection wizard, choosing Amazon Redshift in the technology step.
| Driver class | com.amazon.redshift.jdbc42.Driver |
|---|---|
| JDBC URL | jdbc:redshift://<cluster>.<region>.redshift.amazonaws.com:5439/dev |
| Authentication |
Service principal (shared technical user) User principal (personal credentials, BYOC) Ephemeral credentials (asked at each run)
|
| Crawling strategy | SQL |
The Blindata Agent does not include this driver: download the vendor JDBC driver and copy the .jar into the agent drivers folder, as described in
Add JDBC drivers.
Driver properties can be added under Advanced settings in the connection wizard.
Prerequisites & permissions
- A Blindata Agent installed with network access to Amazon Redshift.
- The vendor JDBC driver installed on the agent (see Add JDBC drivers).
USAGEon the schemas to document; metadata is read frominformation_schema,pg_catalogand theSVV_*views.SYSLOG ACCESS UNRESTRICTED(or thesys:monitorrole) to read query history of all users.SELECTon tables for data quality, profiling and Chat with your data.- IAM authentication is supported with
jdbc:redshift:iam://URLs.
Set up metadata crawling
- Create the connection to Amazon Redshift on the agent (see Connection settings) and use Test to verify it.
- Create or open the system that represents Amazon Redshift in the catalog and link it to the connection.
- In the CRAWLING tab add a crawling job with strategy SQL and, if needed, replace the default queries with the ones suggested below.
- Run a TEST to check the number of extracted objects, then RUN or schedule the job.
The job options and the column aliases of every extraction query are described in the Generic JDBC reference.
Suggested extraction queries
The default SQL strategy queries on information_schema work for tables, views and columns. Use the following routines query to import procedures and UDFs with their parameters.
Routines query
The following query returns the function body (prosrc). Redshift has no json_agg / json_build_object; build ROUTINE_PARAMETERS as a JSON string with LISTAGG from information_schema.parameters (same JSON shape the agent expects). Overloads that share proname can collapse — prefer specific_name from information_schema.routines when that view is readable.
SELECT
'put_catalog_name_here' AS ROUTINE_CATALOG,
n.nspname AS ROUTINE_SCHEMA,
p.proname AS ROUTINE_NAME,
p.proname AS SPECIFIC_NAME,
p.prosrc AS ROUTINE_DEFINITION,
COALESCE(
(
SELECT '[' || LISTAGG(
'{"positionIndex":' || CAST(par.ordinal_position - 1 AS VARCHAR)
|| ',"name":"' || REPLACE(par.parameter_name, '"', '\\"')
|| '","dataType":"' || REPLACE(par.data_type, '"', '\\"')
|| '"}',
','
) WITHIN GROUP (ORDER BY par.ordinal_position) || ']'
FROM information_schema.parameters par
WHERE par.specific_schema = n.nspname
AND par.specific_name = p.proname
AND UPPER(COALESCE(par.parameter_mode, 'IN')) IN ('IN', 'INOUT')
AND par.parameter_name IS NOT NULL
AND par.parameter_name <> ''
),
'[]'
) AS ROUTINE_PARAMETERS
FROM pg_catalog.pg_namespace n
JOIN pg_catalog.pg_proc p ON p.pronamespace = n.oid
WHERE n.nspname NOT IN ('information_schema', 'pg_catalog')
Lineage through OpenLineage
Pipelines that run SQL on Redshift can describe it through OpenLineage events, for example the sql facet of a job, or the Redshift Data provider of the OpenLineage integrations. The OpenLineage connector
turns them into table and column lineage on the tables you crawl.
- Namespace: a Redshift dataset is
redshift://<cluster>.<region>:<port>with the namedatabase.schema.table. Add the propertyopenLineageNamespace, with that namespace as value, to the Redshift System you crawl, so datasets resolve to it and no second System is created. stats: the Redshift Data provider publishes output statistics under this key. Blindata reads it asoutputStatistics, recognized by its schema URL, and stores the numbers as run metrics.- SQL: the
sqlfacet of a job with dialectredshiftbecomes a routine analyzed for column lineage.COPYandUNLOADare not SQL for this purpose: declare their lineage with thelineagefacet.
Facet by facet: Facet support reference . A complete example is in Configure your producers .
Lineage from query history
The agent can periodically extract executed statements from Amazon Redshift and send them to Blindata, where the SQL lineage engine turns them into dataflows. See the ready-made extraction query in Scheduling and statements extraction.