高性能计算--HPCC--工作原理篇

原文： http://hpccsystems.com/Why-HPCC/How-it-works
翻译：那海蓝蓝，译文请见“ 【】”中的部分

The HPCC system architecture incorporates the Thor and Roxie clusters as well as common middleware components, an external communications layer, client interfaces which provide both end-user services and system management tools, and auxiliary components to support monitoring and to facilitate loading and storing of filesystem data from external sources. An HPCC environment can include only Thor clusters, or both Thor and Roxie clusters. Each of these cluster types is described in more detail in the following sections below the architecture diagram.【HPCC系统架构采用Thor和Roxie集群如同普通中间件组件一样，外部的通讯层，客户端接口，同时提供最终用户服务和系统管理工具，便利的组件用以支持监控和方便加载和存储来自外部源头的文件系统数据。 HPCC的环境可以包括只Thor尔集群，或Thor和Roxie集群。在下面的架构图以下各节详细描述这些类型的群集。】

High-Level HPCC Architecture

【 那海蓝蓝解读： 】
【 1：Thor和Roxie配合的时序关系不明。是Thor日常准备数据给Roxie，还是Roxie实时从Thor要求数据？ 】
【2：Thor之间是什么关系？Roxie之间是什么关系？ 】
【3： Thor和Roxie 都有分布式文件系统则可跨多个物理节点，那么，图中的Thor节点只是一个角色，而不是实际结构分布示意图 。问题：真实的物理拓扑图应该是什么样的？】

The diagram above illustrates a high level overview of the platform architecture and how the components all work together as a powerful solution for managing Big Data. A brief description on each component is detailed below. 【上面的图表说明了高层次的平台架构的概述和所有的组件是如何作为一个强大的解决方案，用于管理大数据一起完成所有工作。每个组件的一个简要说明细节如下：】

Thor (the Data Refinery Cluster) is responsible for consuming vast amounts of data, transforming, linking and indexing that data. It functions as a distributed file system with parallel processing power spread across the nodes. A cluster can scale from a single node to thousands of nodes. 【Thor（数据加工）是负责消耗大量的数据，转换，连接和索引数据。作为一个分布式文件系统，它的功能使用了跨节点的并行处理能力。一个集群可以从一个单一的节点扩展规模到上千个节点。】

• Single-threaded【那海蓝蓝解读：为什么是单线程？从示意图上看，Thor和Roxie关系不明，是多对多？还是单对多？按理，是多对多的关系，但是，这和单线程有什么关系呢？】
• Distributed parallel processing
• Distributed file system
• Powerful parallel processing programming language (ECL)
• Optimized for Extraction, Transformation, Loading, Sorting, Indexing and Linking
• Scales from 1-1000s of nodes

Roxie (the Query Cluster) provides separate high-performance online query processing and data warehouse capabilities. 【Roxie（查询集群）提供单独的高性能联机查询处理和数据据仓库能力。】

• Multi-threaded
• Distributed parallel processing
• Distributed file system
• Powerful parallel processing programming language (ECL)
• Optimized for concurrent query processing
• Scales from 1-1000s of nodes

ECL (Enterprise Control Language) is the powerful programming language that is ideally suited for the manipulation of Big Data. 【ECL（企业控制语言）是强有力的程序设计语言，理想化的适合大数据量的维护。】

• Transparent and implicitly parallel programming language
• Non-procedural and dataflow oriented
• Modular, reusable, extensible syntax
• Combines data representation and algorithm implementation
• Easily extend using C++ libraries
• ECL is compiled into optimized C++

ECL IDE is a modern IDE used to code, debug and monitor ECL programs. 【ECL 是模块化的编码。调试、监控ECL程序的集成开发环境】

• Access to shared source code repositories
• Complete development, debugging and testing environment for developing ECL dataflow programs
• Access to the ECLWatch tool is built-in, allowing developers to watch job graphs as they are executing
• Access to current and historical job workunits

ESP (Enterprise Services Platform) provides an easy to use interface to access ECL queries using XML, HTTP, SOAP and REST. 【ESP（企业服务平台）提供了一个易于使用的接口来访问使用XML，HTTP，SOAP和REST的ECL查询】

• Standards-based interface to access ECL functions
• Supports SOAP, XML, HTTP and REST
• Supports SAML and various security standards