Definition of Probes

Within the Blindata platform, the quality probes are organized into projects. Each project constitutes a set of probes that can be logically grouped and which must be scheduled and executed in unison. The interface allows you to easily define projects and create all the necessary probes within each project. It is therefore necessary, first of all, to define a project. The pieces of information required for creating a new project are:

• Name (required): project name.
• Description: description of the project.

After having created one or more projects it is possible to proceed with the creation of quality probes. Each probe must be created in the context of a project, therefore first of all it is necessary to select the project within which the probe is to be created.

The form to fill out for the creation of a new probe allows you to define a typology, one or more queries and the relative connections to the source systems for each probe. In detail, the pieces of information required to create a probe are:

• Name (required): probe name
• Check (required): name and code of the quality check to which you want to associate the results extracted from the probe
• Type (required): type of probe - for further details see the next paragraph
• Connection* (required): name and type of connection you want to use; supported connection types are JDBC, Mongo or Salesforce
• Collection* (required): if you have specified a connection of type Mongo, name of the collection to query
• Result Field Name*: name of the field or alias of the query containing the value to be extracted
• Default Value*: if a connection of type Mongo has been specified, value to use if the query does not return any documents
• Query* (required): if a JDBC or Salesforce connection was specified, query text
• Pipelines* (required): if you specified a connection of type Mongo, one or more text elements each representing a stage of a Mongo pipeline

The fields identified with an asterisk can be repeated based on the type of probe chosen and also depend on the type of connection specified. However, the interface is guided and supports the user in compiling according to his choices.

Maintenance of Probes

To modify a probe, click on the “Modify” button in the probe’s detail page. A form similar to the one for creation will open where it will be possible to modify all the fields of interest.

Within the Blindata platform, every change to the probes is versioned; the history of the versions can be browsed and consulted on the probe detail page (list on the right of the page). Therefore, once you have confirmed the changes to a probe, you will see a new version appear in the list.

Note that when we refer generically to a quality probe within a project, we mean the most recent version of that probe.

Tags

Blindata platform allows you to mark certain points in a project’s history as significant, just like common version control systems do. Each project can be associated with tags, which allow it to identify and recover the snapshot of the project in a given instant of time, with each probe in its version at the time of tagging.

To create a new tag,you are required to fill in the following information:

• Name (required): tag name
• Description: description of the tag

As mentioned, all the probes currently part of the project in their current version will be associated with the new tag.

Each tag can be modified in its name and description and can be deleted.

The utility of tags is to be able to mark some release points, development or production versions, proceed with new developments without altering production configurations and, when necessary, roll back a tag to make hotfixes.

Let’s take the following use case as an example. After a series of developments, a satisfactory solution has been reached, which is desired to be taken into production. Therefore, the current project state is associated with a first tag, for example “V0.0.1”. After some executions in the production environment, some changes are made to the project’s probes, which are then modified and versioned. Unfortunately, it is discovered that one of the probes in production has a bug that must be immediately corrected. Therefore, the first step is to create a new tag to store the current project state (for example “rework”). The “V0.0.1” production tag can then be restored - meaning the state of all project probes is changed from the current state to the state captured at the time of tagging, the probes are modified as necessary, and the current state is tagged again with, for example, “V0.0.2”. At this point, you can go back to working on what is in development by restoring the “rework” tag again. The “rework” tag can now also be deleted.