TIBCO Spotfire

Supported BI & analytics

Import Spotfire library reports and generate lineage from source tables and columns to reports, by querying the Spotfire repository database.

Capabilities

CapabilitySupportNotes
Tables, views & columns Supported Library items of type dxp are imported as REPORT physical entities, with their folder path as schema.
Keys, constraints & indexes Not available
Procedures & functions Not available
Data lineage Supported Dataflows from source columns to reports from LIB_RESOLVED_DEPEND and LIB_PROPERTIES.
Lineage from query history Not available
Business glossary links Not available
Data quality probes Not available
Data profiling Not available
Data classification Not available
Chat with your data Not available

Connection settings

Create the connection on the Blindata Agent from Settings > Agents > Connections with the connection wizard, choosing Template of the repository database (Oracle, SQL Server, PostgreSQL) in the technology step.

Driver classoracle.jdbc.OracleDriver / com.microsoft.sqlserver.jdbc.SQLServerDriver
JDBC URLJDBC URL of the Spotfire repository database
Authentication
Service principal (shared technical user)
User principal (personal credentials, BYOC)
Ephemeral credentials (asked at each run)
Crawling strategySQL

The PostgreSQL and SQL Server drivers are included in the Blindata Agent. For an Oracle repository, install the Oracle JDBC driver on the agent. See Add JDBC drivers. Driver properties can be added under Advanced settings in the connection wizard.

Prerequisites & permissions

  • A Blindata Agent installed with network access to TIBCO Spotfire.
  • Read-only access to the Spotfire repository tables LIB_ITEMS, LIB_ITEM_TYPES, LIB_RESOLVED_DEPEND and LIB_PROPERTIES.

Set up metadata crawling

  1. Create the connection to TIBCO Spotfire on the agent (see Connection settings) and use Test to verify it.
  2. Create or open the system that represents TIBCO Spotfire in the catalog and link it to the connection.
  3. In the CRAWLING tab add a crawling job with strategy SQL and, if needed, replace the default queries with the ones suggested below.
  4. Run a TEST to check the number of extracted objects, then RUN or schedule the job.

The job options and the column aliases of every extraction query are described in the Generic JDBC reference.

Suggested extraction queries

Spotfire stores its library in a repository database (Oracle, SQL Server or PostgreSQL). Create a connection to that database and a crawling job with the SQL strategy: the following queries extract Spotfire reports and generate dataflows from the tables and columns of a source system to the reports in the Spotfire system.

Reports query (physical entities)

SELECT distinct
li.TITLE as table_name,
'REPORT' as table_type,
left((
  coalesce (folders5.title || '/', '') ||
  coalesce (folders4.title || '/' , '') ||
  coalesce (folders3.title || '/', '') ||
  coalesce (folders2.title || '/', '') ||
  coalesce (folders.title, '')
), 255) as table_schema,
coalesce (li.DESCRIPTION, '') as description
FROM LIB_ITEMS li
left join lib_items folders
on li.parent_id = folders.item_id
left join lib_items folders2
on folders.parent_id = folders2.item_id
left join lib_items folders3
on folders2.parent_id = folders3.item_id
left join lib_items folders4
on folders3.parent_id = folders4.item_id
left join lib_items folders5
on folders4.parent_id = folders5.item_id
WHERE li.ITEM_TYPE = (SELECT LIB_ITEM_TYPES.TYPE_ID FROM LIB_ITEM_TYPES WHERE LIB_ITEM_TYPES.LABEL = 'dxp')

Data flows query

SELECT distinct
'insert_dataflow_scope_here' as scope,
left((
	 coalesce(SchemaName.Property_Value || '.', '') ||
	 coalesce(TableName.Property_Value || '.','') ||
	 coalesce(ColumnName.Property_Value,'') ||
	 '->' ||
	 coalesce(SchemaName.ReportFolder || '/', '') ||
	 coalesce(SchemaName.ReportName, '')
   ),
   255
) as name,
'insert_data_flow_description_here' as description,
'insert_source_system_uuid (optional)' as from_system_uuid,
'insert_source_system_name' as from_system_name,
SchemaName.Property_Value as from_physical_entity_schema,
TableName.Property_Value as from_physical_entity_name,
ColumnName.Property_Value as from_physical_field_name,

'insert_spotfire_system_uuid (optional)' as to_system_uuid,
'insert_spotfire_system_name' as to_system_name,
left(coalesce(SchemaName.ReportFolder, ''), 255) as to_physical_entity_schema,
SchemaName.ReportName as to_physical_entity_name
FROM
( SELECT A.ReportName, D.* FROM (SELECT TITLE ReportName, ITEM_ID
         ReportItemID
    FROM lib_items) A
JOIN LIB_RESOLVED_DEPEND B ON A.ReportItemID = B.DEPENDENT_ID
JOIN LIB_RESOLVED_DEPEND C ON B.REQUIRED_ID  = C.DEPENDENT_ID
JOIN LIB_PROPERTIES      D ON D.ITEM_ID      = C.REQUIRED_ID
WHERE D.PROPERTY_NAME = 'table') TableName
JOIN (SELECT A.ReportName, D.*
FROM (SELECT TITLE ReportName, ITEM_ID ReportItemID
        FROM lib_items ) A
JOIN LIB_RESOLVED_DEPEND B ON A.ReportItemID = B.DEPENDENT_ID
JOIN LIB_RESOLVED_DEPEND C ON B.REQUIRED_ID = C.DEPENDENT_ID
JOIN LIB_PROPERTIES D ON D.ITEM_ID = C.REQUIRED_ID
WHERE D.PROPERTY_NAME = 'column') ColumnName
   ON TableName.ITEM_ID    = ColumnName.ITEM_ID AND
      TableName.ReportName = ColumnName.ReportName
JOIN (SELECT A.ReportName, D.*
FROM (SELECT TITLE ReportName, ITEM_ID ReportItemID
FROM lib_items) A
JOIN LIB_RESOLVED_DEPEND B ON A.ReportItemID = B.DEPENDENT_ID
JOIN LIB_RESOLVED_DEPEND C ON B.REQUIRED_ID = C.DEPENDENT_ID
JOIN LIB_PROPERTIES D ON D.ITEM_ID = C.REQUIRED_ID
WHERE D.PROPERTY_NAME = 'catalog') DatabaseName
   ON ColumnName.ITEM_ID      = DatabaseName.ITEM_ID AND
      DatabaseName.ReportName = ColumnName.ReportName
JOIN ( SELECT A.ReportName, D.*, left(
 (
 coalesce (folders5.title || '/', '') ||
coalesce (folders4.title || '/' , '') ||
coalesce (folders3.title || '/', '') ||
coalesce (folders2.title || '/', '') ||
coalesce (folders.title, '')
), 255) as ReportFolder
               FROM ( SELECT TITLE ReportName, ITEM_ID ReportItemID, parent_id
                        FROM lib_items ) A
JOIN LIB_RESOLVED_DEPEND B ON A.ReportItemID = B.DEPENDENT_ID
JOIN LIB_RESOLVED_DEPEND C ON B.REQUIRED_ID = C.DEPENDENT_ID
JOIN LIB_PROPERTIES D ON D.ITEM_ID = C.REQUIRED_ID

left join lib_items folders
on A.parent_id = folders.item_id
left join lib_items folders2
on folders.parent_id = folders2.item_id
left join lib_items folders3
on folders2.parent_id = folders3.item_id
left join lib_items folders4
on folders3.parent_id = folders4.item_id
left join lib_items folders5
on folders4.parent_id = folders5.item_id

WHERE D.PROPERTY_NAME = 'schema' ) SchemaName
   ON DatabaseName.ITEM_ID    = SchemaName.ITEM_ID AND
      DatabaseName.ReportName = SchemaName.ReportName