Configure ODM And Its Services

To access the DataOps platform functionalities directly from the Blindata UI, you must first configure the Blindata Agent to function as a reverse proxy. To do so, follow the tutorial below.

Access the Agent detail page and go to schedule.

Click on the FAB button, select ODM_PROXY as the Job Type and then follow the steps to create the new Job.

Each service in the job configuration should define the endpoint at which it can be accessed. Additionally, you can use servicesConfigurations to configure the following:

• Stage Filtering: A regex pattern can be configured to match specific DataOps stages, ensuring that DevOps actions are permitted only for stages that satisfy the regex pattern
• Authentication: When executors require authentication or authorization to process tasks effectively, they can be specified along with the necessary secrets that must be provided
• Registry: The Registry (V1 or V2 / Builder) can be configured with API version, descriptor pointer (default branch and path for the descriptor file), and git providers—including authentication methods and secret mapping for secure access to git-backed resources

{
  "servicesURLs": {
    "blueprint": "<>",
    "devops": "<>",
    "registry": "<>",
    "policy": "<>",
    "notification": "<>",
    "marketplace": "<>"
  },
  "servicesConfigurations": {
    "devops": {
      "canDeployOnlyStage": "<>",
      "executors": {
        "gitlab": {
          "secrets": [
            { "secretType": "gitlab-token" }
          ]
        }
      }
    }
  }
}

Example with Registry V2 (Builder)

When the registry service URL is set, you must provide a full servicesConfigurations.registry block. The following example shows both DevOps and Registry V2 configuration:

{
  "servicesURLs": {
    "blueprint": "<>",
    "devops": "<>",
    "registry": "<>",
    "policy": "<>",
    "notification": "<>",
    "marketplace": "<>"
  },
  "servicesConfigurations": {
    "devops": {
      "canDeployOnlyStage": "<>",
      "executors": {
        "gitlab": {
          "secrets": [
            { "secretType": "gitlab-token" }
          ]
        }
      }
    },
    "registry": {
      "apiVersion": "v2",
      "descriptorPointer": {
        "defaultBranchName": "main",
        "defaultPath": "src/main/descriptor.json"
      },
      "gitProviders": {
        "github": {
          "baseUrl": "https://api.github.com",
          "authenticationMethods": [
            {
              "method": "PAT",
              "secrets": [
                {
                  "inputHeader": "github-pat",
                  "secretType": "github-pat",
                  "outputHeader": "gpauth-param-token"
                }
              ]
            }
          ]
        }
      ]
    }
  }
}

While navigating the DataOps platform sections in the Blindata UI, you can view the current configuration in use and, if multiple configurations are available, switch to your desired one.

Accessing ODM Services Programmatically via API

You can access ODM services via API by targeting the agent endpoint with the configured context path. The authentication methods are the same of Blindata API.

API Endpoint Template

Use the following template to construct your API requests:

{agent-url}/agent-api/v1/odm/proxy/{odm-config-uuid}/{service-name}/{odm-service-path}

Replace the placeholders with the appropriate values:

• {agent-url}: Base URL of the agent
• {odm-config-uuid}: Unique identifier of the job that configures the Blindata Agent as a reverse proxy for ODM
• {service-name}: Name of the ODM service
• {odm-service-path}: Specific API path of the ODM service

Example

For instance, an API request may look like this:

https://demo.blindata.io/agent-api/v1/odm/proxy/563b0cc8-345e-4594-bb72-e0b2f2f343c6/policy/api/v1/pp/policy/policy-engines

Personal Secrets Integration

When making API requests through the agent proxy, you can leverage personal secrets for secure authentication. If you have previously saved personal secrets using the /agent-api/v1/config/personal-secrets endpoint, you can include them in your request headers.

DevOps (executors)

For DevOps, use the following header pattern:

x-odm-ag-<ExecutorName>-executor-secret-<SecretType>: <SecretName>

where:

• <ExecutorName> is the name of the executor, as specified in the configuration
• <SecretType> is one of the types of secrets that the executor can handle, as specified in the configuration
• <SecretName> is the name you assigned to the secret when it was stored using the /agent-api/v1/config/personal-secrets endpoint.

The agent will automatically transform the header name to follow the pattern used by the ODM DevOps service and replace the secret name with the actual secret value, so that it can be properly forwarded to the executor. This results in the following pattern (note the removal of -ag- in the header name and the replacement of <SecretName> with <SecretValue>):

x-odm-<ExecutorName>-executor-secret-<SecretType>: <SecretValue>

Registry / Builder

For Registry (e.g. Registry V2 / Builder) endpoints, authentication can use headers of the form:

x-odm-ag-<inputHeader>: <SecretName>

where <inputHeader> is the value of inputHeader from your registry’s git provider secret configuration (under servicesConfigurations.registry.gitProviders). The agent looks up the matching secret configuration, resolves the secret by name and type via personal secrets, and forwards the request with a header x-odm-<outputHeader>: <SecretValue>, where <outputHeader> comes from the same secret configuration. For example, if the config has "inputHeader": "github-pat" and "outputHeader": "gpauth-param-token", sending x-odm-ag-github-pat: my-github-token-name results in the agent forwarding x-odm-gpauth-param-token: <resolved-secret-value>.

In cases where the secret is not saved through the agent and you have direct access to the secret value, you can send the ODM header pattern directly to the agent. The agent will forward it to the ODM service as-is, without modifying the header name or value.

Example with Personal Secret

POST /agent-api/v1/odm/proxy/{odm-config-uuid}/policy/api/v1/policies

x-odm-ag-gitlab-executor-secret-gitlab-token: my-gitlab-token-name
x-odm-azure-executor-secret-api-key: api-key-value
Content-Type: application/json

{"policy": "data"}

This request will be forwarded as:

x-odm-gitlab-executor-secret-gitlab-token: gitlab-token-value // Transformed to ODM pattern and secret name replaced by secret value
x-odm-azure-executor-secret-api-key: api-key-value // Forwarded unchanged
Content-Type: application/json

{"policy": "data"}

Defining specific constraints for Platform Services

DevOps

The DevOps service allows defining specific constraints within the ODM job configuration. The canDeployOnlyStage setting accepts a regex pattern that is used to validate operations performed by the devops service. Only operations targeting stages that match this regex will be permitted. For example, you can configure it to allow actions exclusively on the dev stage or only on the prod one .

{
  "servicesURLs": {
    "devops": "<>",
    [...]
  },
  "servicesConfigurations": {
    "devops": {
      "canDeployOnlyStage": "StageRegex"
    }
  }
}

Registry (V1 / V2)

If you set servicesURLs.registry, you must provide a corresponding servicesConfigurations.registry block. A full example that includes both DevOps and Registry V2 is shown in the Example with Registry V2 (Builder) above. This configuration is required for the Registry service (including Registry V2 / Builder) and defines how the agent resolves descriptor locations and authenticates with git providers.

• apiVersion (required): API version of the registry. Allowed values are v1 and v2. Use v2 for Registry V2 / Builder.

• descriptorPointer (required): Default location of the descriptor file (DPDS) in the repository.

  • defaultBranchName: Default git branch name (e.g. main, master).
  • defaultPath: Path to the descriptor file within the repo (e.g. src/main/descriptor.json). For descriptor format and usage, see the Data Product Builder .

• gitProviders (at least one required): Git providers used by the Registry (e.g. for Builder). When present, each provider must have a valid configuration. Supported provider keys: azure, bitbucket, github, gitlab. For each provider:

  • baseUrl (required): Base URL of the git provider (e.g. https://api.github.com).
  • authenticationMethods: Array of authentication method objects. Each object has:
    • method: Authentication method (e.g. PAT for Personal Access Token).
    • secrets: Array of secret mappings. Each entry has:
      • inputHeader: Unique identifier for the incoming request header (the part after the x-odm-ag- prefix). Used by the agent to match the header and look up this configuration. Must be unique across all git providers and authentication methods. Matching is case-insensitive.
      • secretType: Type used to look up the secret in the agent’s personal secrets (same as for DevOps).
      • outputHeader: Header name suffix sent to the ODM Registry service. The agent forwards the secret as x-odm- + outputHeader with the resolved secret value.
  • defaultSettings (optional): Reserved for future use; can be omitted.

Validation rules: If servicesURLs.registry is set, servicesConfigurations.registry must be present. When registry configuration is present, descriptorPointer is required and both defaultBranchName and defaultPath must be non-empty. The field apiVersion is required and must be v1 or v2. Every entry under gitProviders must have a non-empty baseUrl. Duplicate inputHeader values across all secret configurations are not allowed and will be rejected.