Overview
Atlas is a scalable and extensible set of core foundational governance services – enabling enterprises to effectively and efficiently meet their compliance requirements within Hadoop and allows integration with the whole enterprise data ecosystem.
Apache Atlas provides open metadata management and governance capabilities for organizations to build a catalog of their data assets, classify and govern these assets and provide collaboration capabilities around these data assets for data scientists, analysts and the data governance team.
Atlas 是一组可扩展和可扩展的核心基础治理服务——使企业能够有效和高效地满足其在 Hadoop 中的合规性要求,并允许与整个企业数据生态系统集成。
Apache Atlas 为组织提供开放的元数据管理和治理功能,以构建其数据资产的目录,对这些资产进行分类和治理,并为数据科学家、分析师和数据治理团队提供围绕这些数据资产的协作功能。
Features
Metadata types & instances
Pre-defined types for various Hadoop and non-Hadoop metadata
Ability to define new types for the metadata to be managed
Types can have primitive attributes, complex attributes, object references; can inherit from other types
Instances of types, called entities, capture metadata object details and their relationships
REST APIs to work with types and instances allow easier integration
元数据类型和实例
各种 Hadoop 和非 Hadoop 元数据的预定义类型
能够为要管理的元数据定义新类型
类型可以有原始属性、复杂属性、对象引用;可以从其他类型继承
类型的实例,称为实体,捕获元数据对象详细信息及其关系
用于处理类型和实例的 REST API 允许更轻松的集成
Classification
Ability to dynamically create classifications - like PII, EXPIRES_ON, DATA_QUALITY, SENSITIVE
Classifications can include attributes - like expiry_date attribute in EXPIRES_ON classification
Entities can be associated with multiple classifications, enabling easier discovery and security enforcement
Propagation of classifications via lineage - automatically ensures that classifications follow the data as it goes through various processing
能够动态创建分类 - 如 PII、EXPIRES_ON、DATA_QUALITY、SENSITIVE
分类可以包括属性 - 如 EXPIRES_ON 分类中的 expiry_date 属性
实体可以与多个分类相关联,从而更容易发现和安全实施
通过谱系传播分类 - 自动确保分类在经过各种处理时遵循数据
Lineage
Intuitive UI to view lineage of data as it moves through various processes
REST APIs to access and update lineage
直观的 UI 可在数据通过各种流程时查看数据的沿袭
用于访问和更新谱系的 REST API
Search/Discovery
Intuitive UI to search entities by type, classification, attribute value or free-text
Rich REST APIs to search by complex criteria
SQL like query language to search entities - Domain Specific Language (DSL)
直观的 UI,可按类型、分类、属性值或自由文本搜索实体
丰富的 REST API 可按复杂条件进行搜索
用于搜索实体的 SQL 之类的查询语言 - 领域特定语言 (DSL)
Security & Data Masking
Fine grained security for metadata access, enabling controls on access to entity instances and operations like add/update/remove classifications
Integration with Apache Ranger enables authorization/data-masking on data access based on classifications associated with entities in Apache Atlas. For example:
who can access data classified as PII, SENSITIVE
customer-service users can only see last 4 digits of columns classified as NATIONAL_ID
元数据访问的细粒度安全性,支持对实体实例的访问和添加/更新/删除分类等操作的控制
与 Apache Ranger 的集成支持基于与 Apache Atlas 中实体关联的分类对数据访问进行授权/数据屏蔽。例如:
谁可以访问归类为 PII、敏感的数据
客户服务用户只能看到归类为 NATIONAL_ID 的列的最后 4 位数字