hadoop知识点
Exam Sections and Blueprint
1. HDFS (17%)
Describe the function of HDFS daemons
描述hdfs的功能性deamon,
Describe the normal operation of an Apache Hadoop cluster, both in data storage and in data processing
描述hadoop集群在数据存储和数据处理方面正常的操作流程。
Identify current features of computing systems that motivate a system like Apache Hadoop
??
Classify major goals of HDFS Design
分类hdfs设计的主要目标
Given a scenario, identify appropriate use case for HDFS Federation
给定场景,确认hdfs federation合适的应用。
Identify components and daemon of an HDFS HA-Quorum cluster
确认hdfs ha-quorum集群的的组件和常驻程序。
Analyze the role of HDFS security (Kerberos)
分析hdfs的安全性,hdfs。
Determine the best data serialization choice for a given scenario
判断给定场景最好的数据序列号选择
Describe file read and write paths
描述文件读写路径
Identify the commands to manipulate files in the Hadoop File System Shell
确认hdfs 操作文件的命令行命令
2. YARN and MapReduce version 2 (MRv2) (17%)
Understand how upgrading a cluster from Hadoop 1 to Hadoop 2 affects cluster settings
理解将集群从hadoop1升级到hadoop2对集群配置的影响。
Understand how to deploy MapReduce v2 (MRv2 / YARN), including all YARN daemons
理解如何部署mapreduce2(mr2/yarn),包括yarn的常驻程序。
Understand basic design strategy for MapReduce v2 (MRv2)
理解mr2基本的设计策略。
Determine how YARN handles resource allocations
掌握yarn分配资源的方法。
Identify the workflow of MapReduce job running on YARN
了解yarn上mr任务的工作流程。
Determine which files you must change and how in order to migrate a cluster from MapReduce version 1 (MRv1) to MapReduce version 2 (MRv2) running on YARN
了解如何将集群从mr1升级到mr2,知道哪些文件必须修改。
3. Hadoop Cluster Planning (16%)
Principal points to consider in choosing the hardware and operating systems to host an Apache Hadoop cluster
构架hadoop集群时硬件和操作系统的选择要点。
Analyze the choices in selecting an OS
分析操作系统的不同选择。
Understand kernel tuning and disk swapping
理解内核调节和磁盘交换区。
Given a scenario and workload pattern, identify a hardware configuration appropriate to the scenario
给定一个负载模型和场景,说明适合该场景的硬件配置。
Given a scenario, determine the ecosystem components your cluster needs to run in order to fulfill the SLA
给定场景,判断满足指定延迟所需的集群生态组件。
Cluster sizing: given a scenario and frequency of execution, identify the specifics for the workload, including CPU, memory, storage, disk I/O
给定集群规模,给定执行频率,说明工作负载,包括cpu,内存,存储,磁盘I/O
Disk Sizing and Configuration, including JBOD versus RAID, SANs, virtualization, and disk sizing requirements in a cluster
磁盘大小及配置,包括JBOD(磁盘簇),RAID(磁盘阵列),SANs(局域网存储),虚拟化和磁盘大小需求。
Network Topologies: understand network usage in Hadoop (for both HDFS and MapReduce) and propose or identify key network design components for a given scenario
4. Hadoop Cluster Installation and Administration (25%)
Given a scenario, identify how the cluster will handle disk and machine failures’
给定情况,确认集群如何处理磁盘和机器失败
Analyze a logging configuration and logging configuration file format
分析log配置和log文件格式
Understand the basics of Hadoop metrics and cluster health monitoring
理解hadoopmetric基础和集群健康情况监控
Identify the function and purpose of available tools for cluster monitoring
确认集群监控工具的功能和目的
Be able to install all the ecoystme components in CDH 5, including (but not limited to): Impala, Flume, Oozie, Hue, Cloudera Manager, Sqoop, Hive, and Pig
能够安装cdh5生态的组件,包括但不限于impala,flume,oozie,hue,cloudera manager,sqoop,hive,pig
Identify the function and purpose of available tools for managing the Apache Hadoop file system
确认hadoop文件系统管理工具的功能和目的。
5. Resource Management (10%)
Understand the overall design goals of each of Hadoop schedulers
理解hadoop 各种scheduler的设计目标
Given a scenario, determine how the FIFO Scheduler allocates cluster resources
给定场景,判断fifo scheduler如何分配集群资源
Given a scenario, determine how the Fair Scheduler allocates cluster resources under YARN
给定场景,判断fair scheduler如何分配集群资源
Given a scenario, determine how the Capacity Scheduler allocates cluster resources
给定场景,判断capacity scheduler如何分配集群资源
6. Monitoring and Logging (15%)
Understand the functions and features of Hadoop’s metric collection abilities
理解hadoop metric收集的方法和特点
Analyze the NameNode and JobTracker Web UIs
分析namenode和jobtracker的web ui
Understand how to monitor cluster daemons
理解如何监控集群常驻程序。
Identify and monitor CPU usage on master nodes
识别并能监控主节点上的cpu使用情况
Describe how to monitor swap and memory allocation on all nodes
描述如何监控物理内存和交换内存的在节点上的分配情况。
Identify how to view and manage Hadoop’s log files
了解如何管理和监控hadoop日志文件。
Interpret a log file
理解日志文件。
Disclaimer: These exam preparation pages are intended to provide information about the objectives covered by each exam, related resources, and recommended reading and courses. The material contained within these pages is not intended to guarantee a passing score on any exam. Cloudera recommends that a candidate thoroughly understand the objectives for each exam and utilize the resources and training courses recommended on these pages to gain a thorough understand of the domain of knowledge related to the role the exam evaluates.