PostgreSQL
Catalog schemas, tables, views, columns, keys, indexes and PL/pgSQL functions, with SQL lineage from views, functions and query history.
Capabilities
| Capability | Support | Notes |
|---|---|---|
| Tables, views & columns | Supported | |
| Keys, constraints & indexes | Supported | Primary keys, foreign keys, unique constraints and indexes (from pg_catalog). |
| Procedures & functions | Supported | |
| Data lineage | Supported | |
| Lineage from query history | Supported | Statements collected from pg_stat_statements. |
| Business glossary links | Supported | |
| Data quality probes | Supported | |
| Data profiling | Supported | |
| Data classification | Supported | |
| Chat with your data | Supported |
Connection settings
Create the connection on the Blindata Agent from Settings > Agents > Connections with the connection wizard, choosing PostgreSQL in the technology step.
| Driver class | org.postgresql.Driver |
|---|---|
| JDBC URL | jdbc:postgresql://localhost:5432/postgres |
| Authentication |
Service principal (shared technical user) User principal (personal credentials, BYOC) Ephemeral credentials (asked at each run)
|
| Crawling strategy | SQL |
The driver is included in the Blindata Agent: no installation is needed. Driver properties can be added under Advanced settings in the connection wizard.
Prerequisites & permissions
- A Blindata Agent installed with network access to PostgreSQL.
CONNECTon the database andUSAGEon the schemas to document.- Metadata is read from
information_schemaandpg_catalog; objects are visible only if the user has at least one privilege on them. - To read function bodies owned by other roles, use the
pg_proc.prosrcroutines query below. SELECTon tables is needed only for data quality, profiling, classification and Chat with your data.- For query history: the
pg_stat_statementsextension and thepg_read_all_statsrole.
Set up metadata crawling
- Create the connection to PostgreSQL on the agent (see Connection settings) and use Test to verify it.
- Create or open the system that represents PostgreSQL in the catalog and link it to the connection.
- In the CRAWLING tab add a crawling job with strategy SQL and, if needed, replace the default queries with the ones suggested below.
- Run a TEST to check the number of extracted objects, then RUN or schedule the job.
The job options and the column aliases of every extraction query are described in the Generic JDBC reference.
Suggested extraction queries
The default SQL strategy queries on information_schema work out of the box for tables, views and columns. The following queries extend them with indexes and with a routines query that does not depend on information_schema privileges.
Physical constraints query
This query also extracts index metadata.
select distinct
KCU.constraint_name as name,
KCU.constraint_schema as schema,
'PRIMARY_KEY' as type,
KCU.table_schema as target_physical_entity_schema,
KCU.table_name as target_physical_entity_name,
KCU.column_name as target_physical_field_name,
KCU.ordinal_position as list_order,
NULL as referenced_physical_entity_schema,
NULL as referenced_physical_entity_name,
NULL as referenced_physical_field_name,
NULL as clause,
null as delete_rule,
null as update_rule,
null as access_method
from information_schema.table_constraints AS TC
inner join information_schema.key_column_usage AS KCU
on KCU.constraint_catalog = TC.constraint_catalog
and KCU.constraint_schema = TC.constraint_schema
and KCU.table_name = TC.table_name
and KCU.constraint_name = TC.constraint_name
where TC.constraint_type = 'PRIMARY KEY'
union
select distinct
tc.constraint_name as name,
tc.constraint_schema as schema,
'FOREIGN_KEY' as type,
tc.table_schema as target_physical_entity_schema,
tc.table_name as target_physical_entity_name,
kcu.column_name as target_physical_field_name,
kcu.ordinal_position as list_order,
ccu.table_schema as referenced_physical_entity_schema,
ccu.table_name as referenced_physical_entity_name,
ccu.column_name as referenced_physical_field_name,
NULL as clause,
rc.delete_rule as delete_rule,
rc.update_rule as update_rule,
null as access_method
from information_schema.table_constraints as tc
join information_schema.key_column_usage as kcu
on tc.constraint_name = kcu.constraint_name
and tc.table_schema = kcu.table_schema
join information_schema.constraint_column_usage as ccu
on ccu.constraint_name = tc.constraint_name
and ccu.table_schema = tc.table_schema
join information_schema.referential_constraints rc
on rc.constraint_name = kcu.constraint_name
and kcu.column_name = ccu.column_name
where tc.constraint_type = 'FOREIGN KEY'
union
select distinct
KCU.constraint_name as name,
KCU.constraint_schema as schema,
'UNIQUE' as type,
KCU.table_schema as target_physical_entity_schema,
KCU.table_name as target_physical_entity_name,
KCU.column_name as target_physical_field_name,
KCU.ordinal_position as list_order,
NULL as referenced_physical_entity_schema,
NULL as referenced_physical_entity_name,
NULL as referenced_physical_field_name,
NULL as clause,
null as delete_rule,
null as update_rule,
null as access_method
from information_schema.table_constraints AS TC
inner join information_schema.key_column_usage AS KCU
on KCU.constraint_catalog = TC.constraint_catalog
and KCU.constraint_schema = TC.constraint_schema
and KCU.table_name = TC.table_name
and KCU.constraint_name = TC.constraint_name
where TC.constraint_type = 'UNIQUE'
union
select distinct
i.relname as name,
ixs.schemaname as schema,
'INDEX' as type,
ixs.schemaname as target_physical_entity_schema,
t.relname as target_physical_entity_name,
a.attname as target_physical_field_name,
a.attnum as list_order,
NULL as referenced_physical_entity_schema,
NULL as referenced_physical_entity_name,
NULL as referenced_physical_field_name,
ixs.indexdef as clause,
null as delete_rule,
null as update_rule,
null as access_method
from
pg_class t,
pg_class i,
pg_index ix,
pg_indexes ixs,
pg_attribute a
where i.relname = ixs.indexname
and t.oid = ix.indrelid
and i.oid = ix.indexrelid
and a.attrelid = t.oid
and a.attnum = ANY(ix.indkey);
Routines query
If information_schema permissions hide routine_definition, use pg_proc.prosrc for the body. That catalog does not include a parameters JSON column — add ROUTINE_PARAMETERS from proargnames / proargmodes / argument types (do not paste the default information_schema.parameters subquery here: it is what this fallback is avoiding, and specific_name is not always proname for overloads).
SELECT
'put_catalog_name_here' AS ROUTINE_CATALOG,
n.nspname AS ROUTINE_SCHEMA,
p.proname AS ROUTINE_NAME,
p.proname AS SPECIFIC_NAME,
CASE p.prokind
WHEN 'p' THEN 'PROCEDURE'
ELSE 'FUNCTION'
END AS ROUTINE_TYPE,
p.prosrc AS ROUTINE_DEFINITION,
(
SELECT COALESCE(
json_agg(
json_build_object(
'name', args.argname,
'dataType', format_type(args.argtype, NULL),
'positionIndex', args.ord - 1
)
ORDER BY args.ord
),
'[]'::json
)
FROM unnest(
COALESCE(p.proargnames, ARRAY[]::text[]),
COALESCE(
p.proargmodes,
ARRAY(SELECT 'i'::"char" FROM generate_series(1, COALESCE(p.pronargs, 0)))
),
COALESCE(p.proallargtypes, p.proargtypes)::oid[]
) WITH ORDINALITY AS args(argname, argmode, argtype, ord)
WHERE COALESCE(args.argmode, 'i') IN ('i', 'b')
AND args.argname IS NOT NULL
AND args.argname <> ''
) AS ROUTINE_PARAMETERS
FROM pg_catalog.pg_namespace n
JOIN pg_catalog.pg_proc p ON p.pronamespace = n.oid
WHERE n.nspname NOT IN ('information_schema', 'pg_catalog');
prosrc is already the analyzable body (not CREATE FUNCTION). Modes 'i' / 'b' are IN / INOUT; TABLE/OUT arguments are skipped. prokind exists from PostgreSQL 11; on older versions omit ROUTINE_TYPE.
Lineage from query history
The agent can periodically extract executed statements from PostgreSQL and send them to Blindata, where the SQL lineage engine turns them into dataflows. See the ready-made extraction query in Scheduling and statements extraction.