Computational policies in Blindata play a crucial role in governing the lifecycle of data products, ensuring compliance and governance throughout various stages of data product management. These policies are sets of rules and guidelines that are automatically enforced by the Blindata platform.

Common Lifecycle Phases Covered by Computational Policies

Computational policies are applied during key lifecycle phases of a data product. For more information, check the ODM Platform Policy Documentation .

Data Product Registration

The policy engine interacts with the registry to block the creation or update of data product objects that do not comply with global policies. This ensures that only compliant data products are registered. For example, policies can be implemented to check that names, domains, and descriptive metadata align with your organization’s policies.

Data Product Version Publishing

During version publishing, the registry validates data product versions against global policies. Non-compliant updates or creations can be blocked to maintain governance standards. Some examples include:

• Policies that enforce the adherence of descriptors to the specification, ensuring that all necessary metadata is present. This is critical to enable the shift-left approach.
• Policies on backward compatibility that ensure breaking changes are avoided.

Data Product Deployment

Computational policies validate multiple aspects during deployment, including:

• The validity of stage transitions.
• The correctness of task results.
• The accuracy of the application status at the end of an activity, ensuring it reflects the status defined in the contract.

Policies on cloud resource configuration can be enforced to restrict cloud regions, backup policies, encryption settings, and other organizational constraints. These policies operate prior to the deployment of cloud resources, preventing undesired changes even before they take place.

For example, policies can analyze outputs such as Terraform plans to verify compliance before progressing to production.

Event Types

A policy is a set of rules, guidelines, or standards designed to govern the lifecycle of data products. Policies are configured to be triggered by specific event types, ensuring compliance and governance throughout various stages of data product management.

Policies can be triggered by the following event types:

• DATA_PRODUCT_CREATION: The creation of a new data product (major version).
• DATA_PRODUCT_UPDATE: The update of an existing data product (major version).
• DATA_PRODUCT_VERSION_CREATION: The release of a new data product version (minor version).
• ACTIVITY_STAGE_TRANSITION: A transition from one DevOps activity stage to the next.
• TASK_EXECUTION_RESULT: The execution result of an individual activity task.
• ACTIVITY_EXECUTION_RESULT: The execution result of an entire activity.

Each event includes currentState and afterState fields, which describe the state of the relevant resources before and after the event, depending on the event type.

Event State Structures

The content that policies receive depends on the event they are registered on. This allows policies to access the specific information needed to make informed decisions and enforce rules effectively.

Each event includes a currentState and afterState field, which describe the state of the relevant resources before and after the event. In addition, events may include other relevant data, such as timestamps, user information, and specific details related to the event.

DATA_PRODUCT_CREATION, DATA_PRODUCT_UPDATE, and DATA_PRODUCT_VERSION_CREATION

{
  "currentState": {
    "dataProductVersion": {
      // The data product descriptor of the most recent version, if present
    }
  },
  "afterState": {
    // The data product descriptor of the new data product version
  }
}

For DATA_PRODUCT_CREATION and DATA_PRODUCT_UPDATE, only the info field of the data product descriptor is populated.

ACTIVITY_STAGE_TRANSITION

{
  "currentState": {
    "lifecycle": { // Describes the terminated stage
      "id": "",
      "dataProductId": "",
      "dataProductVersion": "",
      "stage": ""
    },
    "activity": { // Errors and results of the terminated stage
      "id": "",
      "dataProductId": "",
      "dataProductVersion": "",
      "stage": "",
      "status": "",
      "results": "",
      "errors": "",
      "createdAt": "",
      "startedAt": "",
      "finishedAt": ""
    },
    "tasks": [ // Errors and results of each task in the stage
      {
        "id": "",
        "activityId": "",
        "executorRef": "",
        "callbackRef": "",
        "template": "",
        "configurations": "",
        "status": "",
        "results": "",
        "errors": "",
        "createdAt": "",
        "startedAt": "",
        "finishedAt": ""
      }
    ]
  },
  "afterState": {
    // Analogous to the current state structure
  }
}

TASK_EXECUTION_RESULT

{
  "currentState": {
    "activity": { // Errors and results of the task's activity
      "id": "",
      "dataProductId": "",
      "dataProductVersion": "",
      "stage": "",
      "status": "",
      "results": "",
      "errors": "",
      "createdAt": "",
      "startedAt": "",
      "finishedAt": ""
    },
    "task": { // Errors and results of the terminated task
      "id": "",
      "activityId": "",
      "executorRef": "",
      "callbackRef": "",
      "template": "",
      "configurations": "",
      "status": "",
      "results": "",
      "errors": "",
      "createdAt": "",
      "startedAt": "",
      "finishedAt": ""
    }
  },
  "afterState": {
    // Analogous to the current state structure
  }
}

ACTIVITY_EXECUTION_RESULT

{
  "currentState": {
    // Empty
  },
  "afterState": {
    "activity": { // The terminated activity with its results
      "id": "",
      "dataProductId": "",
      "dataProductVersion": "",
      "stage": "",
      "status": "",
      "results": "",
      "errors": "",
      "createdAt": "",
      "startedAt": "",
      "finishedAt": ""
    },
    "dataProductVersion": { // The complete data product descriptor
    }
  }
}