How to Develop Custom Jobs

The Generic Job functionality within the agent allows you to execute custom external developments by triggering them through the agent using the generic job configuration.

Custom Job Architecture

Overview

The Generic Job is designed to be flexible, allowing the user to specify job parameters and connections dynamically. The configuration is defined in JSON format and includes the endpoint of the job, the parameters required for the job, and the connections that the job needs to access external systems.

Configuration Format

Below is an example of the JSON configuration required to trigger a Generic Job:

{
  "jobUrl": "endpoint of the job",
  "params": {
    "param1": "param1 value",
    "param2": "param2 value",
    ...
  },
  "connections": {
    "key of the connection job side": "name of the connection Blindata side",
    ...
  },
  "type": "GENERIC_JOB"
}

All the following parameters are required:

  • jobUrl: The endpoint where the job will be triggered.
  • params: A key-value pair of parameters required for the job.
  • connections: A mapping of connection keys (as referenced in the job) to the actual connection names defined in Blindata.

Setup

Creating a generic job is very similar to creating any other type of job in Blindata. As with other jobs, you need to define a resource on which the job will act and specify the type as GENERIC_JOB, as shown in the figure.

Create Generic Job

Job Definition Example

This example specifies the necessary details for triggering a generic job in Blindata. It includes the endpoint where the job will be triggered (jobUrl), various parameters required for the job execution (params), and the connections needed (connections).

{
  "jobUrl": "http://example.com/trigger-job",
  "params": {
    "githubToken": "your-github-token",
    "queryText": "SELECT * FROM config",
    "githubUrl": "https://github.com/your-repo.git",
    "targetSystemUuid": "your-target-system-uuid",
    "tenantUuid": "your-tenant-uuid",
    "dryRun": "true",
    "subDirectory": "/sql/"
  },
  "connections": {
    "redshift": "Redshift Connection",
    "blindata": "Blindata Connection"
  }
}

Job Result

If the target job is triggered successfully, the final result will be as shown in the figure.

Success Job

In case of failure the error message will be specific depending on the target job implementation in use.