kyuubi

Kyuubi is an enhanced editon of Apache Spark's primordial Thrift JDBC/ODBC Server.

Kyuubi ACL Management Guide

Authorization Modes

Three primary modes for Kyuubi authorization are available by spark-authorizer:

Storage-Based Authorization

Enabling Storage Based Authorization in the Hive Metastore Server uses the HDFS permissions to act as the main source for verification and allows for consistent data and metadata authorization policy. This allows control over metadata access by verifying if the user has permission to access corresponding directories on the HDFS. Similar with HiveServer2, files and directories will be translated into hive metadata objects, such as dbs, tables, partitions, and be protected from end user’s queries through Kyuubi.

Storage-Based Authorization offers users with Database, Table and Partition-level coarse-gained access control.

Please refer to the Storage-Based Authorization Guide in the online documentation for an overview on how to configure Storage-Based Authorization for Kyuubi.

SQL-Standard Based Authorization

Enabling SQL-Standard Based Authorization gives users more fine-gained control over access comparing with Storage Based Authorization. Besides of the ability of Storage Based Authorization, SQL-Standard Based Authorization can improve it to Views and Column-level. Unfortunately, Spark SQL does not support grant/revoke statements which controls access, this might be done only through the HiveServer2. But it’s gratifying that spark-authorizer makes Spark SQL be able to understand this fine-grain access control granted or revoked by Hive.

With Kyuubi, the SQL-Standard Based Authorization is guaranteed for the security configurations, metadata, and storage information is preserved from end users.

Please refer to the SQL-Standard Based Authorization Guide in the online documentation for an overview on how to configure SQL-Standard Based Authorization for Spark SQL.

Ranger Security Support (Recommended)

Apache Ranger is a framework to enable, monitor and manage comprehensive data security across the Hadoop platform but end before Spark or Spark SQL. The spark-authorizer enables Kyuubi with control access ability reusing Ranger Plugin for Hive MetaStore . Apache Ranger makes the scope of existing SQL-Standard Based Authorization expanded but without supporting Spark SQL. spark-authorizer sticks them together.

Please refer to the Spark SQL Ranger Security Support Guide in the online documentation for an overview on how to configure Ranger for Spark SQL.

Additional Documentations

Building Kyuubi
Kyuubi Deployment Guide
Kyuubi Containerization Guide
High Availability Guide
Configuration Guide
Authentication/Security Guide
Kyuubi Architecture
Home Page