Hadoop, Google File System等相关名词解释

最新推荐文章于 2022-02-18 14:31:08 发布

zmycoco2

最新推荐文章于 2022-02-18 14:31:08 发布

阅读量1.3k

点赞数

分类专栏：分布式数据存储 Hadoop

本文链接：https://blog.csdn.net/michaelzhou224/article/details/9072823

版权

本文介绍了Google File System (GFS) 的设计目标和工作原理，以及Hadoop Distributed File System (HDFS)的特点。同时提到了BigTable作为分布式结构化数据存储系统的作用，以及HBase作为Hadoop生态中的NoSQL数据库。还简述了MapReduce、Hive和Pig等在大数据处理中的角色。

摘要由CSDN通过智能技术生成

GFS

Google File System(GFS or GoogleFS) is a proprietary distributed file system developed by Google Inc. for its own use. It is designed to provide efficient reliable access to data using large clusters of commodity hardware. A new version of the Goofle File System is codenamed Colossus.

GFS is optimized for Google’s core data storage and usage needs(primarily the search engine), which can generate enormous amounts of data that needs to be retained. GFS grew out of an earlier Google effort, “BigFiles”. Files are divided into fixed-size chunks of 64 megabytes, similar to clusters or sectors in regular file systems, which are only extremely rarely overwritten. Pr shrunk; files are usually appended to or read. It is also designed and optimized to run on Google’s computing clysers, dense nodes which consist