PostgreSQL

Supported Relational databases

Catalog schemas, tables, views, columns, keys, indexes and PL/pgSQL functions, with SQL lineage from views, functions and query history.

Capabilities

CapabilitySupportNotes
Tables, views & columns Supported
Keys, constraints & indexes Supported Primary keys, foreign keys, unique constraints and indexes (from pg_catalog).
Procedures & functions Supported
Data lineage Supported
Lineage from query history Supported Statements collected from pg_stat_statements.
Business glossary links Supported
Data quality probes Supported
Data profiling Supported
Data classification Supported
Chat with your data Supported

Connection settings

Create the connection on the Blindata Agent from Settings > Agents > Connections with the connection wizard, choosing PostgreSQL in the technology step.

Driver classorg.postgresql.Driver
JDBC URLjdbc:postgresql://localhost:5432/postgres
Authentication
Service principal (shared technical user)
User principal (personal credentials, BYOC)
Ephemeral credentials (asked at each run)
Crawling strategySQL

The driver is included in the Blindata Agent: no installation is needed. Driver properties can be added under Advanced settings in the connection wizard.

Prerequisites & permissions

  • A Blindata Agent installed with network access to PostgreSQL.
  • CONNECT on the database and USAGE on the schemas to document.
  • Metadata is read from information_schema and pg_catalog; objects are visible only if the user has at least one privilege on them.
  • To read function bodies owned by other roles, use the pg_proc.prosrc routines query below.
  • SELECT on tables is needed only for data quality, profiling, classification and Chat with your data.
  • For query history: the pg_stat_statements extension and the pg_read_all_stats role.

Set up metadata crawling

  1. Create the connection to PostgreSQL on the agent (see Connection settings) and use Test to verify it.
  2. Create or open the system that represents PostgreSQL in the catalog and link it to the connection.
  3. In the CRAWLING tab add a crawling job with strategy SQL and, if needed, replace the default queries with the ones suggested below.
  4. Run a TEST to check the number of extracted objects, then RUN or schedule the job.

The job options and the column aliases of every extraction query are described in the Generic JDBC reference.

Suggested extraction queries

The default SQL strategy queries on information_schema work out of the box for tables, views and columns. The following queries extend them with indexes and with a routines query that does not depend on information_schema privileges.

Physical constraints query

This query also extracts index metadata.

select  distinct
    KCU.constraint_name as name,
    KCU.constraint_schema as schema,
    'PRIMARY_KEY' as type,
    KCU.table_schema as target_physical_entity_schema,
    KCU.table_name as target_physical_entity_name,
    KCU.column_name as target_physical_field_name,
    KCU.ordinal_position as list_order,
    NULL as referenced_physical_entity_schema,
    NULL as referenced_physical_entity_name,
    NULL as referenced_physical_field_name,
    NULL as clause,
    null as delete_rule,
    null as update_rule,
    null as access_method
  from information_schema.table_constraints AS TC
  inner join information_schema.key_column_usage AS KCU
  on KCU.constraint_catalog = TC.constraint_catalog
  and KCU.constraint_schema = TC.constraint_schema
  and KCU.table_name = TC.table_name
  and KCU.constraint_name = TC.constraint_name
  where TC.constraint_type = 'PRIMARY KEY'
  union
  select  distinct
    tc.constraint_name as name,
    tc.constraint_schema as schema,
    'FOREIGN_KEY' as type,
    tc.table_schema as target_physical_entity_schema,
    tc.table_name as target_physical_entity_name,
    kcu.column_name as target_physical_field_name,
    kcu.ordinal_position as list_order,
    ccu.table_schema as referenced_physical_entity_schema,
    ccu.table_name as referenced_physical_entity_name,
    ccu.column_name as referenced_physical_field_name,
    NULL as clause,
    rc.delete_rule as delete_rule,
    rc.update_rule as update_rule,
    null as access_method
  from information_schema.table_constraints as tc
  join information_schema.key_column_usage as kcu
  on tc.constraint_name = kcu.constraint_name
  and tc.table_schema = kcu.table_schema
  join information_schema.constraint_column_usage as ccu
  on ccu.constraint_name = tc.constraint_name
  and ccu.table_schema = tc.table_schema
  join information_schema.referential_constraints rc
  on rc.constraint_name = kcu.constraint_name
  and kcu.column_name = ccu.column_name
  where tc.constraint_type = 'FOREIGN KEY'
  union
  select  distinct
    KCU.constraint_name as name,
    KCU.constraint_schema as schema,
    'UNIQUE' as type,
    KCU.table_schema as target_physical_entity_schema,
    KCU.table_name as target_physical_entity_name,
    KCU.column_name as target_physical_field_name,
    KCU.ordinal_position as list_order,
    NULL as referenced_physical_entity_schema,
    NULL as referenced_physical_entity_name,
    NULL as referenced_physical_field_name,
    NULL as clause,
    null as delete_rule,
    null as update_rule,
    null as access_method
  from information_schema.table_constraints AS TC
  inner join information_schema.key_column_usage AS KCU
  on KCU.constraint_catalog = TC.constraint_catalog
  and KCU.constraint_schema = TC.constraint_schema
  and KCU.table_name = TC.table_name
  and KCU.constraint_name = TC.constraint_name
  where TC.constraint_type = 'UNIQUE'
  union
  select  distinct
    i.relname as name,
    ixs.schemaname  as schema,
    'INDEX' as type,
    ixs.schemaname  as target_physical_entity_schema,
    t.relname as target_physical_entity_name,
    a.attname as target_physical_field_name,
    a.attnum  as list_order,
    NULL as referenced_physical_entity_schema,
    NULL as referenced_physical_entity_name,
    NULL as referenced_physical_field_name,
    ixs.indexdef as clause,
    null as delete_rule,
    null as update_rule,
    null as access_method
  from
    pg_class t,
    pg_class i,
    pg_index ix,
    pg_indexes ixs,
    pg_attribute a
  where	i.relname = ixs.indexname
  and t.oid = ix.indrelid
  and i.oid = ix.indexrelid
  and a.attrelid = t.oid
  and a.attnum = ANY(ix.indkey);

Routines query

If information_schema permissions hide routine_definition, use pg_proc.prosrc for the body. That catalog does not include a parameters JSON column — add ROUTINE_PARAMETERS from proargnames / proargmodes / argument types (do not paste the default information_schema.parameters subquery here: it is what this fallback is avoiding, and specific_name is not always proname for overloads).

SELECT
  'put_catalog_name_here' AS ROUTINE_CATALOG,
  n.nspname AS ROUTINE_SCHEMA,
  p.proname AS ROUTINE_NAME,
  p.proname AS SPECIFIC_NAME,
  CASE p.prokind
    WHEN 'p' THEN 'PROCEDURE'
    ELSE 'FUNCTION'
  END AS ROUTINE_TYPE,
  p.prosrc AS ROUTINE_DEFINITION,
  (
    SELECT COALESCE(
      json_agg(
        json_build_object(
          'name', args.argname,
          'dataType', format_type(args.argtype, NULL),
          'positionIndex', args.ord - 1
        )
        ORDER BY args.ord
      ),
      '[]'::json
    )
    FROM unnest(
      COALESCE(p.proargnames, ARRAY[]::text[]),
      COALESCE(
        p.proargmodes,
        ARRAY(SELECT 'i'::"char" FROM generate_series(1, COALESCE(p.pronargs, 0)))
      ),
      COALESCE(p.proallargtypes, p.proargtypes)::oid[]
    ) WITH ORDINALITY AS args(argname, argmode, argtype, ord)
    WHERE COALESCE(args.argmode, 'i') IN ('i', 'b')
      AND args.argname IS NOT NULL
      AND args.argname <> ''
  ) AS ROUTINE_PARAMETERS
FROM pg_catalog.pg_namespace n
JOIN pg_catalog.pg_proc p ON p.pronamespace = n.oid
WHERE n.nspname NOT IN ('information_schema', 'pg_catalog');

prosrc is already the analyzable body (not CREATE FUNCTION). Modes 'i' / 'b' are IN / INOUT; TABLE/OUT arguments are skipped. prokind exists from PostgreSQL 11; on older versions omit ROUTINE_TYPE.

Lineage from query history

The agent can periodically extract executed statements from PostgreSQL and send them to Blindata, where the SQL lineage engine turns them into dataflows. See the ready-made extraction query in Scheduling and statements extraction.