Apache Hive
Catalog Hive metastore databases, tables, views and columns through HiveServer2.
Capabilities
| Capability | Support | Notes |
|---|---|---|
| Tables, views & columns | Supported | The JDBC strategy reads the metastore through the driver; no information_schema is needed. |
| Keys, constraints & indexes | Not available | |
| Procedures & functions | Not available | |
| Data lineage | Partial | |
| Lineage from query history | Not available | |
| Business glossary links | Supported | |
| Data quality probes | Supported | |
| Data profiling | Supported | |
| Data classification | Supported | |
| Chat with your data | Supported |
Connection settings
Create the connection on the Blindata Agent from Settings > Agents > Connections with the connection wizard, choosing Apache Hive in the technology step.
| Driver class | org.apache.hive.jdbc.HiveDriver |
|---|---|
| JDBC URL | jdbc:hive2://localhost:10000/default |
| Authentication |
Service principal (shared technical user) User principal (personal credentials, BYOC) Ephemeral credentials (asked at each run)
|
| Crawling strategy | JDBC |
The Blindata Agent does not include this driver: download the vendor JDBC driver and copy the .jar into the agent drivers folder, as described in
Add JDBC drivers.
Driver properties can be added under Advanced settings in the connection wizard.
Prerequisites & permissions
- A Blindata Agent installed with network access to Apache Hive.
- The vendor JDBC driver installed on the agent (see Add JDBC drivers).
- A user allowed to run
SHOW DATABASES/DESCRIBEon the databases to document (Ranger or Sentry policies).
Set up metadata crawling
- Create the connection to Apache Hive on the agent (see Connection settings) and use Test to verify it.
- Create or open the system that represents Apache Hive in the catalog and link it to the connection.
- In the CRAWLING tab add a crawling job with strategy JDBC and, if needed, replace the default queries with the ones suggested below.
- Run a TEST to check the number of extracted objects, then RUN or schedule the job.
The job options and the column aliases of every extraction query are described in the Generic JDBC reference.
Suggested crawling configuration
Use the JDBC strategy with the default queries: the agent reads databases, tables, views and columns from the Hive metastore through HiveServer2, without needing an INFORMATION_SCHEMA. To limit the extraction to some databases, add a where table_schema in (...) filter to the default Tables and Columns queries.