(refer to the Data Product page )

Data Product Registry

Blindata provides its own Data Product Catalog (refer to the Data Product page ), which allows users to initialize and manage Data Products before officially registering them within the platform.

This integration enables a bidirectional synchronization between Blindata and the Data Platform, ensuring that Data Products are consistently managed across both systems.

The Registry Service is a core component of the Data Platform, designed to manage Data Products and their versions. It provides APIs to create, edit, retrieve, and delete Data Products and Data Product Versions, using the Data Product Descriptor as a blueprint.

Beyond basic management, the Registry Service includes syntactic and semantic validation to ensure the accuracy and compliance of each Data Product. It can also be configured to integrate with other platform services, such as those for policy computation or event notifications.

Key Concepts:

• Data Product: A fundamental unit in the data platform, defined and managed according to the Data Product Descriptor Specification (DPDS).
• Data Product Version: A specific release of a Data Product, used to track changes, introduce updates, and maintain compatibility with downstream systems. Each version may include schema modifications, feature updates, or bug fixes.

The Data Product Descriptor Specification (DPDS)

The Data Product Descriptor Specification (DPDS) defines a standard for structuring and detailing all aspects of a Data Product. It outlines key elements such as the fully qualified name, ownership, domain, version, interface components, and internal elements.

The DPDS serves as a foundational document for managing Data Products within the platform. It enables consistent registration, governance, and lifecycle management by providing a complete view of each Data Product, facilitating communication between consumers and the underlying Data Platform.

For detailed information about the DPDS and its components, refer to the official documentation: Data Product Descriptor Specification .

The Blindata Observer for ODM Platform

The Blindata Observer for ODM is an adapter service within the ODM Platform that integrates with Blindata to manage data product updates, metadata synchronization, and lifecycle details. Its primary role is to listen for ODM events and execute configurable actions by sending relevant information back to Blindata. For each subscription, specific actions can be defined, allowing the Observer to manage data product updates, metadata, and lifecycle details effectively.

Configurable Actions:

• DATA_PRODUCT_UPLOAD: Uploads key data product information and assigns ownership.
• DATA_PRODUCT_VERSION_UPLOAD: Uploads metadata for data product ports and assets.
• STAGES_UPLOAD: Uploads the data product’s lifecycle stages.
• DATA_PRODUCT_REMOVAL: Removes the data product from Blindata.
• POLICIES_UPLOAD: Uploads policy evaluation results for a data product.

To explore more about the Observer and its functionalities, visit the Blindata Observer for ODM GitHub Repository .

What can i do with Blindata and the Data Product Registry?

The Data Product Registry offers several intuitive features to streamline the management of data products:

1. Data Product List

   • Displays all registered data products in the platform.
   • Includes options to create new data products or quickly search through existing ones.

2. Data Product Details

   • Provides a comprehensive overview of a selected data product, including key metadata, domain associations, and version history.
   • Users can easily access specific versions or proceed to lifecycle management options.

3. Version Management

   • Displays a detailed history of all data product versions.
   • Enables users to upload descriptors, view version-specific metadata, and manage new version releases.

4. Publishing Data Product Versions

   • Offers flexible publishing methods for data product versions:
     • Via File: Upload a JSON or YAML descriptor directly.
     • Via Repository: Link to a source repository for automated version control and updates.