hadoop-hdfs 简介、中文文档、中英对照文档 下载

hadoop-hdfs 文档 下载链接(含jar包、源码、pom)

组件名称中文-文档-下载链接中英对照-文档-下载链接
hadoop-hdfs-2.5.1.jarhadoop-hdfs-2.5.1-API文档-中文版.ziphadoop-hdfs-2.5.1-API文档-中英对照版.zip
hadoop-hdfs-2.6.5.jarhadoop-hdfs-2.6.5-API文档-中文版.ziphadoop-hdfs-2.6.5-API文档-中英对照版.zip
hadoop-hdfs-2.7.3.jarhadoop-hdfs-2.7.3-API文档-中文版.ziphadoop-hdfs-2.7.3-API文档-中英对照版.zip
hadoop-hdfs-2.9.1.jarhadoop-hdfs-2.9.1-API文档-中文版.ziphadoop-hdfs-2.9.1-API文档-中英对照版.zip

hadoop-hdfs 简介

Apache Hadoop HDFS 。

在这里插入图片描述

hadoop-hdfs 中文文档、中英对照文档 说明

摘要:hadoop-hdfs、org.apache.hadoop、中文文档、中英对照文档、下载、包含jar包、原API文档、源代码、Maven依赖信息文件、翻译后的API文档、apache、hadoop、hdfs、中英对照文档、jar包、java;

使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。

人性化翻译,文档中的代码和结构保持不变,注释和说明精准翻译,请放心使用。

双语对照,边学技术、边学英语。

涉及的包(package)

org.apache.hadoop.hdfs

org.apache.hadoop.hdfs.net

org.apache.hadoop.hdfs.protocol

org.apache.hadoop.hdfs.protocol.datatransfer

org.apache.hadoop.hdfs.protocol.datatransfer.sasl

org.apache.hadoop.hdfs.protocolPB

org.apache.hadoop.hdfs.qjournal.client

org.apache.hadoop.hdfs.qjournal.protocol

org.apache.hadoop.hdfs.qjournal.protocolPB

org.apache.hadoop.hdfs.qjournal.server

org.apache.hadoop.hdfs.security.token.block

org.apache.hadoop.hdfs.security.token.delegation

org.apache.hadoop.hdfs.server.balancer

org.apache.hadoop.hdfs.server.blockmanagement

org.apache.hadoop.hdfs.server.common

org.apache.hadoop.hdfs.server.datanode

org.apache.hadoop.hdfs.server.datanode.fsdataset

org.apache.hadoop.hdfs.server.datanode.fsdataset.impl

org.apache.hadoop.hdfs.server.datanode.metrics

org.apache.hadoop.hdfs.server.datanode.web

org.apache.hadoop.hdfs.server.datanode.web.webhdfs

org.apache.hadoop.hdfs.server.mover

org.apache.hadoop.hdfs.server.namenode

org.apache.hadoop.hdfs.server.namenode.ha

org.apache.hadoop.hdfs.server.namenode.ha.proto

org.apache.hadoop.hdfs.server.namenode.metrics

org.apache.hadoop.hdfs.server.namenode.snapshot

org.apache.hadoop.hdfs.server.namenode.top

org.apache.hadoop.hdfs.server.namenode.top.metrics

org.apache.hadoop.hdfs.server.namenode.top.window

org.apache.hadoop.hdfs.server.namenode.web.resources

org.apache.hadoop.hdfs.server.protocol

org.apache.hadoop.hdfs.tools

org.apache.hadoop.hdfs.tools.offlineEditsViewer

org.apache.hadoop.hdfs.tools.offlineImageViewer

org.apache.hadoop.hdfs.tools.snapshot

org.apache.hadoop.hdfs.util

org.apache.hadoop.hdfs.web

org.apache.hadoop.hdfs.web.resources

涉及的类(class)

AbstractFuture

AclEntryStatusFormat

AdminHelper

AtomicFileOutputStream

AuditLogger

AuthFilter

AvailableSpaceBlockPlacementPolicy

AvailableSpaceVolumeChoosingPolicy

BalancerBandwidthCommand

BlockIdManager

BlockListAsLongs.Builder

BlockManagerFaultInjector

BlockPlacementPolicyWithNodeGroup

BlockPlacementStatusDefault

BlockPoolTokenSecretManager

BlockStatsMXBean

BlockStoragePolicySuite

BlockUnderConstructionFeature

BPServiceActorAction

BPServiceActorActionException

CacheManager.PersistState

CachePool.DirectiveList

CancelDelegationTokenServlet

ChunkChecksum

CombinedHostFileManager

Content

Content.Counts

Content.CountsMap

Content.CountsMap.Key

ContentCounts

ContentCounts.Builder

CorruptReplicasMap.Reason

DatanodeDescriptor.CachedBlocksList

DatanodeDescriptor.CachedBlocksList.Type

DatanodeHttpServer

DataNodeLayoutVersion.Feature

DataNodeMetricHelper

DatanodeProtocolServerSideTranslatorPB

DatanodeStatistics

DatanodeStorageInfo

DataNodeUGIProvider

DatasetVolumeChecker

DatasetVolumeChecker.Callback

DataTransferThrottler

DefaultINodeAttributesProvider

DelegationTokenSecretManager.SecretManagerState

DFSHAAdmin

DFSNetworkTopology

DFSTopologyNodeImpl

DFSUtil.ConfiguredNNAddress

DFSUtil.ServiceComparator

Diff

Diff.Container

Diff.Element

Diff.ListType

Diff.Processor

Diff.UndoInfo

DirectoryWithQuotaFeature

DirectoryWithQuotaFeature.Builder

DirectoryWithSnapshotFeature.DirectoryDiff

DirectoryWithSnapshotFeature.DirectoryDiffList

Dispatcher.DBlock

Dispatcher.DDatanode

Dispatcher.StorageGroupMap

EncryptionFaultInjector

EncryptionZoneManager

EnumCounters

EnumCounters.Factory

EnumCounters.Map

EnumDoubles

ErrorReportAction

ExceptionHandler

ExitStatus

FencedException

FileDiff

FileDiffList

FileIoProvider.OPERATION

FinalizedReplica

FSConstants

FsDatasetCache.PageRounder

FsDatasetFactory

FsDatasetSpi.Factory

FsDatasetSpi.FsVolumeReferences

FSDirAttrOp

FSDirectory.DirOp

FSEditLogLoader.PositionTrackingInputStream

FSEditLogOp.OpInstanceCache

FSEditLogOp.Reader

FSEditLogOp.Writer

FSImageFormat.Loader

FSImageFormatPBINode.Loader

FSImageFormatPBINode.Saver

FSImageFormatPBSnapshot.Loader

FSImageFormatPBSnapshot.Saver

FSImageFormatProtobuf.Loader

FSImageFormatProtobuf.LoaderContext

FSImageFormatProtobuf.Saver

FSImageFormatProtobuf.SaverContext

FSImageFormatProtobuf.SaverContext.DeduplicationMap

FSImageFormatProtobuf.SectionName

FsImageProto

FsImageProto.CacheManagerSection

FsImageProto.CacheManagerSection.Builder

FsImageProto.CacheManagerSectionOrBuilder

FsImageProto.FileSummary

FsImageProto.FileSummary.Builder

FsImageProto.FileSummary.Section

FsImageProto.FileSummary.Section.Builder

FsImageProto.FileSummary.SectionOrBuilder

FsImageProto.FileSummaryOrBuilder

FsImageProto.FilesUnderConstructionSection

FsImageProto.FilesUnderConstructionSection.Builder

FsImageProto.FilesUnderConstructionSection.FileUnderConstructionEntry

FsImageProto.FilesUnderConstructionSection.FileUnderConstructionEntry.Builder

FsImageProto.FilesUnderConstructionSection.FileUnderConstructionEntryOrBuilder

FsImageProto.FilesUnderConstructionSectionOrBuilder

FsImageProto.INodeDirectorySection

FsImageProto.INodeDirectorySection.Builder

FsImageProto.INodeDirectorySection.DirEntry

FsImageProto.INodeDirectorySection.DirEntry.Builder

FsImageProto.INodeDirectorySection.DirEntryOrBuilder

FsImageProto.INodeDirectorySectionOrBuilder

FsImageProto.INodeReferenceSection

FsImageProto.INodeReferenceSection.Builder

FsImageProto.INodeReferenceSection.INodeReference

FsImageProto.INodeReferenceSection.INodeReference.Builder

FsImageProto.INodeReferenceSection.INodeReferenceOrBuilder

FsImageProto.INodeReferenceSectionOrBuilder

FsImageProto.INodeSection

FsImageProto.INodeSection.AclFeatureProto

FsImageProto.INodeSection.AclFeatureProto.Builder

FsImageProto.INodeSection.AclFeatureProtoOrBuilder

FsImageProto.INodeSection.Builder

FsImageProto.INodeSection.FileUnderConstructionFeature

FsImageProto.INodeSection.FileUnderConstructionFeature.Builder

FsImageProto.INodeSection.FileUnderConstructionFeatureOrBuilder

FsImageProto.INodeSection.INode

FsImageProto.INodeSection.INode.Builder

FsImageProto.INodeSection.INode.Type

FsImageProto.INodeSection.INodeDirectory

FsImageProto.INodeSection.INodeDirectory.Builder

FsImageProto.INodeSection.INodeDirectoryOrBuilder

FsImageProto.INodeSection.INodeFile

FsImageProto.INodeSection.INodeFile.Builder

FsImageProto.INodeSection.INodeFileOrBuilder

FsImageProto.INodeSection.INodeOrBuilder

FsImageProto.INodeSection.INodeSymlink

FsImageProto.INodeSection.INodeSymlink.Builder

FsImageProto.INodeSection.INodeSymlinkOrBuilder

FsImageProto.INodeSection.QuotaByStorageTypeEntryProto

FsImageProto.INodeSection.QuotaByStorageTypeEntryProto.Builder

FsImageProto.INodeSection.QuotaByStorageTypeEntryProtoOrBuilder

FsImageProto.INodeSection.QuotaByStorageTypeFeatureProto

FsImageProto.INodeSection.QuotaByStorageTypeFeatureProto.Builder

FsImageProto.INodeSection.QuotaByStorageTypeFeatureProtoOrBuilder

FsImageProto.INodeSection.XAttrCompactProto

FsImageProto.INodeSection.XAttrCompactProto.Builder

FsImageProto.INodeSection.XAttrCompactProtoOrBuilder

FsImageProto.INodeSection.XAttrFeatureProto

FsImageProto.INodeSection.XAttrFeatureProto.Builder

FsImageProto.INodeSection.XAttrFeatureProtoOrBuilder

FsImageProto.INodeSectionOrBuilder

FsImageProto.NameSystemSection

FsImageProto.NameSystemSection.Builder

FsImageProto.NameSystemSectionOrBuilder

FsImageProto.SecretManagerSection

FsImageProto.SecretManagerSection.Builder

FsImageProto.SecretManagerSection.DelegationKey

FsImageProto.SecretManagerSection.DelegationKey.Builder

FsImageProto.SecretManagerSection.DelegationKeyOrBuilder

FsImageProto.SecretManagerSection.PersistToken

FsImageProto.SecretManagerSection.PersistToken.Builder

FsImageProto.SecretManagerSection.PersistTokenOrBuilder

FsImageProto.SecretManagerSectionOrBuilder

FsImageProto.SnapshotDiffSection

FsImageProto.SnapshotDiffSection.Builder

FsImageProto.SnapshotDiffSection.CreatedListEntry

FsImageProto.SnapshotDiffSection.CreatedListEntry.Builder

FsImageProto.SnapshotDiffSection.CreatedListEntryOrBuilder

FsImageProto.SnapshotDiffSection.DiffEntry

FsImageProto.SnapshotDiffSection.DiffEntry.Builder

FsImageProto.SnapshotDiffSection.DiffEntry.Type

FsImageProto.SnapshotDiffSection.DiffEntryOrBuilder

FsImageProto.SnapshotDiffSection.DirectoryDiff

FsImageProto.SnapshotDiffSection.DirectoryDiff.Builder

FsImageProto.SnapshotDiffSection.DirectoryDiffOrBuilder

FsImageProto.SnapshotDiffSection.FileDiff

FsImageProto.SnapshotDiffSection.FileDiff.Builder

FsImageProto.SnapshotDiffSection.FileDiffOrBuilder

FsImageProto.SnapshotDiffSectionOrBuilder

FsImageProto.SnapshotSection

FsImageProto.SnapshotSection.Builder

FsImageProto.SnapshotSection.Snapshot

FsImageProto.SnapshotSection.Snapshot.Builder

FsImageProto.SnapshotSection.SnapshotOrBuilder

FsImageProto.SnapshotSectionOrBuilder

FsImageProto.StringTableSection

FsImageProto.StringTableSection.Builder

FsImageProto.StringTableSection.Entry

FsImageProto.StringTableSection.Entry.Builder

FsImageProto.StringTableSection.EntryOrBuilder

FsImageProto.StringTableSectionOrBuilder

FSLimitException.MaxDirectoryItemsExceededException

FSLimitException.PathComponentTooLongException

FSPermissionChecker

FsVolumeReference

FsVolumeSpi

FsVolumeSpi.BlockIterator

FsVolumeSpi.VolumeCheckContext

GetConf

GetDelegationTokenServlet

HAZKInfoProtos

HAZKInfoProtos.ActiveNodeInfo

HAZKInfoProtos.ActiveNodeInfo.Builder

HAZKInfoProtos.ActiveNodeInfoOrBuilder

HdfsAuditLogger

HdfsServerConstants.BlockUCState

HdfsServerConstants.NamenodeRole

HdfsServerConstants.NodeType

HdfsServerConstants.ReplicaState

HdfsServerConstants.RollingUpgradeStartupOption

HdfsServerConstants.StartupOption

Holder

HostFileManager

HostSet

IgnoreSnapshotException

INode.BlocksMapUpdateInfo

INode.BlocksMapUpdateInfo.UpdatedReplicationInfo

INode.Feature

INode.QuotaDelta

INode.ReclaimContext

INodeAttributeProvider

INodeAttributeProvider.AccessControlEnforcer

INodeAttributes.SnapshotCopy

INodeDirectory

INodeDirectory.SnapshotAndINode

INodeDirectoryAttributes.CopyWithQuota

INodeDirectoryAttributes.SnapshotCopy

INodeFileAttributes.SnapshotCopy

INodeMap

INodeReference

INodeReference.DstReference

INodeReference.WithCount

INodeReference.WithName

INodesInPath

Journal

JournalManager.CorruptionException

JournalNodeMXBean

JournalSet

JsonUtil

LayoutVersion.Feature

LayoutVersion.FeatureInfo

LayoutVersion.LayoutFeature

LengthInputStream

LightWeightHashSet

LightWeightLinkedSet

Matcher

MD5FileUtils

MetaRecoveryContext.RequestStopException

MetricsLoggerTask

MovedBlocks

MovedBlocks.Locations

NameNode.OperationCategory

NamenodeAddressParam

NameNodeHAProxyFactory

NameNodeLayoutVersion.Feature

NameNodeMetrics

NamenodeProtocolServerSideTranslatorPB

NamenodeWebHdfsMethods

NamespaceInfo.Capability

NNStorage.NameNodeDirType

NNStorage.NameNodeFile

NNStorageRetentionManager

NNUpgradeUtil

NumberReplicas

OfflineEditsViewer.Flags

ParamFilter

PBHelper

PBImageDelimitedTextWriter

QJournalProtocolProtos

QJournalProtocolProtos.AcceptRecoveryRequestProto

QJournalProtocolProtos.AcceptRecoveryRequestProto.Builder

QJournalProtocolProtos.AcceptRecoveryRequestProtoOrBuilder

QJournalProtocolProtos.AcceptRecoveryResponseProto

QJournalProtocolProtos.AcceptRecoveryResponseProto.Builder

QJournalProtocolProtos.AcceptRecoveryResponseProtoOrBuilder

QJournalProtocolProtos.CanRollBackRequestProto

QJournalProtocolProtos.CanRollBackRequestProto.Builder

QJournalProtocolProtos.CanRollBackRequestProtoOrBuilder

QJournalProtocolProtos.CanRollBackResponseProto

QJournalProtocolProtos.CanRollBackResponseProto.Builder

QJournalProtocolProtos.CanRollBackResponseProtoOrBuilder

QJournalProtocolProtos.DiscardSegmentsRequestProto

QJournalProtocolProtos.DiscardSegmentsRequestProto.Builder

QJournalProtocolProtos.DiscardSegmentsRequestProtoOrBuilder

QJournalProtocolProtos.DiscardSegmentsResponseProto

QJournalProtocolProtos.DiscardSegmentsResponseProto.Builder

QJournalProtocolProtos.DiscardSegmentsResponseProtoOrBuilder

QJournalProtocolProtos.DoFinalizeRequestProto

QJournalProtocolProtos.DoFinalizeRequestProto.Builder

QJournalProtocolProtos.DoFinalizeRequestProtoOrBuilder

QJournalProtocolProtos.DoFinalizeResponseProto

QJournalProtocolProtos.DoFinalizeResponseProto.Builder

QJournalProtocolProtos.DoFinalizeResponseProtoOrBuilder

QJournalProtocolProtos.DoPreUpgradeRequestProto

QJournalProtocolProtos.DoPreUpgradeRequestProto.Builder

QJournalProtocolProtos.DoPreUpgradeRequestProtoOrBuilder

QJournalProtocolProtos.DoPreUpgradeResponseProto

QJournalProtocolProtos.DoPreUpgradeResponseProto.Builder

QJournalProtocolProtos.DoPreUpgradeResponseProtoOrBuilder

QJournalProtocolProtos.DoRollbackRequestProto

QJournalProtocolProtos.DoRollbackRequestProto.Builder

QJournalProtocolProtos.DoRollbackRequestProtoOrBuilder

QJournalProtocolProtos.DoRollbackResponseProto

QJournalProtocolProtos.DoRollbackResponseProto.Builder

QJournalProtocolProtos.DoRollbackResponseProtoOrBuilder

QJournalProtocolProtos.DoUpgradeRequestProto

QJournalProtocolProtos.DoUpgradeRequestProto.Builder

QJournalProtocolProtos.DoUpgradeRequestProtoOrBuilder

QJournalProtocolProtos.DoUpgradeResponseProto

QJournalProtocolProtos.DoUpgradeResponseProto.Builder

QJournalProtocolProtos.DoUpgradeResponseProtoOrBuilder

QJournalProtocolProtos.FinalizeLogSegmentRequestProto

QJournalProtocolProtos.FinalizeLogSegmentRequestProto.Builder

QJournalProtocolProtos.FinalizeLogSegmentRequestProtoOrBuilder

QJournalProtocolProtos.FinalizeLogSegmentResponseProto

QJournalProtocolProtos.FinalizeLogSegmentResponseProto.Builder

QJournalProtocolProtos.FinalizeLogSegmentResponseProtoOrBuilder

QJournalProtocolProtos.FormatRequestProto

QJournalProtocolProtos.FormatRequestProto.Builder

QJournalProtocolProtos.FormatRequestProtoOrBuilder

QJournalProtocolProtos.FormatResponseProto

QJournalProtocolProtos.FormatResponseProto.Builder

QJournalProtocolProtos.FormatResponseProtoOrBuilder

QJournalProtocolProtos.GetEditLogManifestRequestProto

QJournalProtocolProtos.GetEditLogManifestRequestProto.Builder

QJournalProtocolProtos.GetEditLogManifestRequestProtoOrBuilder

QJournalProtocolProtos.GetEditLogManifestResponseProto

QJournalProtocolProtos.GetEditLogManifestResponseProto.Builder

QJournalProtocolProtos.GetEditLogManifestResponseProtoOrBuilder

QJournalProtocolProtos.GetJournalCTimeRequestProto

QJournalProtocolProtos.GetJournalCTimeRequestProto.Builder

QJournalProtocolProtos.GetJournalCTimeRequestProtoOrBuilder

QJournalProtocolProtos.GetJournalCTimeResponseProto

QJournalProtocolProtos.GetJournalCTimeResponseProto.Builder

QJournalProtocolProtos.GetJournalCTimeResponseProtoOrBuilder

QJournalProtocolProtos.GetJournalStateRequestProto

QJournalProtocolProtos.GetJournalStateRequestProto.Builder

QJournalProtocolProtos.GetJournalStateRequestProtoOrBuilder

QJournalProtocolProtos.GetJournalStateResponseProto

QJournalProtocolProtos.GetJournalStateResponseProto.Builder

QJournalProtocolProtos.GetJournalStateResponseProtoOrBuilder

QJournalProtocolProtos.HeartbeatRequestProto

QJournalProtocolProtos.HeartbeatRequestProto.Builder

QJournalProtocolProtos.HeartbeatRequestProtoOrBuilder

QJournalProtocolProtos.HeartbeatResponseProto

QJournalProtocolProtos.HeartbeatResponseProto.Builder

QJournalProtocolProtos.HeartbeatResponseProtoOrBuilder

QJournalProtocolProtos.IsFormattedRequestProto

QJournalProtocolProtos.IsFormattedRequestProto.Builder

QJournalProtocolProtos.IsFormattedRequestProtoOrBuilder

QJournalProtocolProtos.IsFormattedResponseProto

QJournalProtocolProtos.IsFormattedResponseProto.Builder

QJournalProtocolProtos.IsFormattedResponseProtoOrBuilder

QJournalProtocolProtos.JournalIdProto

QJournalProtocolProtos.JournalIdProto.Builder

QJournalProtocolProtos.JournalIdProtoOrBuilder

QJournalProtocolProtos.JournalRequestProto

QJournalProtocolProtos.JournalRequestProto.Builder

QJournalProtocolProtos.JournalRequestProtoOrBuilder

QJournalProtocolProtos.JournalResponseProto

QJournalProtocolProtos.JournalResponseProto.Builder

QJournalProtocolProtos.JournalResponseProtoOrBuilder

QJournalProtocolProtos.NewEpochRequestProto

QJournalProtocolProtos.NewEpochRequestProto.Builder

QJournalProtocolProtos.NewEpochRequestProtoOrBuilder

QJournalProtocolProtos.NewEpochResponseProto

QJournalProtocolProtos.NewEpochResponseProto.Builder

QJournalProtocolProtos.NewEpochResponseProtoOrBuilder

QJournalProtocolProtos.PersistedRecoveryPaxosData

QJournalProtocolProtos.PersistedRecoveryPaxosData.Builder

QJournalProtocolProtos.PersistedRecoveryPaxosDataOrBuilder

QJournalProtocolProtos.PrepareRecoveryRequestProto

QJournalProtocolProtos.PrepareRecoveryRequestProto.Builder

QJournalProtocolProtos.PrepareRecoveryRequestProtoOrBuilder

QJournalProtocolProtos.PrepareRecoveryResponseProto

QJournalProtocolProtos.PrepareRecoveryResponseProto.Builder

QJournalProtocolProtos.PrepareRecoveryResponseProtoOrBuilder

QJournalProtocolProtos.PurgeLogsRequestProto

QJournalProtocolProtos.PurgeLogsRequestProto.Builder

QJournalProtocolProtos.PurgeLogsRequestProtoOrBuilder

QJournalProtocolProtos.PurgeLogsResponseProto

QJournalProtocolProtos.PurgeLogsResponseProto.Builder

QJournalProtocolProtos.PurgeLogsResponseProtoOrBuilder

QJournalProtocolProtos.QJournalProtocolService

QJournalProtocolProtos.QJournalProtocolService.BlockingInterface

QJournalProtocolProtos.QJournalProtocolService.Interface

QJournalProtocolProtos.QJournalProtocolService.Stub

QJournalProtocolProtos.RequestInfoProto

QJournalProtocolProtos.RequestInfoProto.Builder

QJournalProtocolProtos.RequestInfoProtoOrBuilder

QJournalProtocolProtos.SegmentStateProto

QJournalProtocolProtos.SegmentStateProto.Builder

QJournalProtocolProtos.SegmentStateProtoOrBuilder

QJournalProtocolProtos.StartLogSegmentRequestProto

QJournalProtocolProtos.StartLogSegmentRequestProto.Builder

QJournalProtocolProtos.StartLogSegmentRequestProtoOrBuilder

QJournalProtocolProtos.StartLogSegmentResponseProto

QJournalProtocolProtos.StartLogSegmentResponseProto.Builder

QJournalProtocolProtos.StartLogSegmentResponseProtoOrBuilder

Quota

Quota.Counts

QuotaByStorageTypeEntry

QuotaByStorageTypeEntry.Builder

QuotaCounts

QuotaCounts.Builder

ReadOnlyList.Util

ReceivedDeletedBlockInfo

ReceivedDeletedBlockInfo.BlockStatus

ReconfigurationProtocolServerSideTranslatorPB

ReconfigurationProtocolServerSideUtils

ReferenceCountMap.ReferenceCounter

RemoteEditLog

RemoteEditLogManifest

RenewDelegationTokenServlet

ReplicaAlreadyExistsException

ReplicaBeingWritten

ReplicaHandler

ReplicaInfo.ReplicaDirInfo

ReplicaInPipeline

ReplicaInPipelineInterface

ReplicaInputStreams

ReplicaOutputStreams

ReplicaUnderRecovery

ReplicaWaitingToBeRecovered

ReportBadBlockAction

RollingWindowManager.Op

RollingWindowManager.TopWindow

RollingWindowManager.User

RoundRobinVolumeChoosingPolicy

RwLock

SecureDataNodeStarter

SecureDataNodeStarter.SecureResources

ShortCircuitRegistry

ShortCircuitRegistry.NewShmInfo

ShortCircuitRegistry.RegisteredShm

ShortCircuitRegistry.Visitor

SlowDiskTracker.DiskLatency

SlowPeerTracker.ReportForJson

Snapshot.Root

SnapshotException

SnapshotFSImageFormat

SnapshotFSImageFormat.ReferenceMap

SnapshotInfo.Bean

SnapshotManager

SnapshotStatsMXBean

StartupProgress.Counter

Storage.StorageState

StorageBlockReport

StorageLocation.CheckContext

StoragePolicyAdmin

StoragePolicySummary

StorageReceivedDeletedBlocks

TeeOutputStream

TokenKindParam

TokenServiceParam

TransferFsImage.HttpGetFailedException

TransferFsImage.HttpPutFailedException

UnexpectedReplicaStateException

UnresolvedTopologyException

UriFsPathParam

UserProvider

VolumeFailureSummary

VolumeScanner

WebHdfsHandler

WebImageViewer

WhitelistBasedTrustedChannelResolver

XmlImageVisitor

XMLUtils.InvalidXmlException

XMLUtils.Stanza

XMLUtils.UnmanglingError

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 1
    评论
hadoop 中文文档index Prefetch chain 预处理链 Pre selector Precondition Enforcer Fetch chain 提取链 Fetch DNS Fetch Http Extractor chain抽取链 Extractor HTML Extractor JS Write Chain 写链 ARC Writer Processor Post process Chain Crawl State Updater Post selector 范围部件,边界部件,处理器链   预取链:主要是做一些准备工作,例如,对处理进行延迟和重新处理,否决随后的操作。 提取链:主要是获得资源,进行DNS转换,填写请求和响应表单   抽取链:当提取完成时,抽取感兴趣的HTML,JavaScript,通常那里有新的也适合的URI,此时URI仅仅被发现,不会被评估   写链:存储爬行结果,返回内容和抽取特性,过滤完存储。   提交链:做最后的维护,例如,测试那些不在范围内的,提交给边界部件 1. Pre fetch Chain 预处理链。 1.1 precondition Enforcer: 预处理。Enforcer 处理。确定、提供捉取所需要的基本信息。 例如:查询、请求一个DNS,或者是 请求、分析一个robots.txt文件。--这些都是在进行下一步之前所要做的事情。 1.2 Pre selector :预选择器。查看爬虫的域,看是否这个URL已经被处理(捉取)过了。如果被捉取过了,那么就把这个URL的状态置为 OUT_OF_SCOPE (超出了域的范围),并且直接跳到 post processor(后处理),更新该爬虫的信息。 1.3 Quota Enforcer :限额处理。这是一个简单的配额处理器。检查主机、服务器、边界组 和现在的捉取URL已经超过了配额的话,阻塞正在处理的URL,并且该URL的状态设置为 S_BLOCKED_BY_QUOTA。 1.4 Runtime Limit Enforcer :持续时间限制处理。这是在爬虫程序的时限控制处理器。 这个处理器是扩展和提高Heritrix 长时间运行能力所必不可少的一个类。设置允许的最大允许时间,如果对一个URL的处理超过了这个最大的时间,那么处理器可以停止或暂停这个URL的处理。 1.4.1 暂停工作-暂停爬虫。一个修改(增加)持续时间可以重新恢复爬虫的工作。如果试图恢复爬虫的运行,而不修改运行允许的持续时间的话,爬虫会再次被停止。 1.5 Runtime Limit Enforcer . Operation:状态变量枚举。包含处理状态的常量。暂停、停止、阻塞。 2. Fetch Chain 捉取处理链。 2.1 Fetch DNS :捉取DNS。处理器去分析 和处理以dns:开始的URL. 2.2 Fetch HTTP :捉取HTTP。使用了Apache 的 HttpClient类库。处理以http:开头的URL。 2.3 Fetch FTP : 捉取FTP。捉取FTP的目录和文档,远程的FTP服务器必须支持 NLIST 命令。现在大多数的FTP服务器支持。 2.4 Fetch Stats :捉取的主机、端口、或者是边检组的一个统计。 2.5 BDB Cookie Storage :缓存器。使用BDB嵌入数据库引擎实现的缓存器,存储容量大。 2.6 Default Server Cache :服务器默认缓存器。 2.7 Heritrix Http Method Retry Handler : http 方法重请求帮助类。 2.8 Heritrix Protocol Socket Factory : Hreitrix的标准协议端口工厂。 2.9 Heritrix SSL Protocol Socket Factory :heritrix的SSL 协议端口工厂。 2.10 Simple Cookie Storage : 简单缓存存储器。 3. Extractor Chain 链接提取处理链。 3.1 Extractor 处理捉取的URL链接,这个类提供错误帮助当分析文档内容的时候。当然可以用它的子类 ContentExtactor 来替代。 3.2 Extractor HTML 从HTML 内容里提取链接。 3.3 Extractor CSS 从 CSS 里面提取链接。 3.4 Extractor JS 从 JS 里面提取链接。 3.5 Extractor PDF 从 PDF里面提起链接。 3.6 Extractor SWF 从 Flash里面提取链接。 3.7 Extractor Universal 最后一个链接提取器,通过提取任何看起来像链接的。如果使用的话,那么一般把它放在链接提取链的最后一个。 3.8 Extractor URI 从URI 里面提取链接。如果该URI含有其他链接的话。 3.9 Extractor XML 从XML里面提取 链接。 3.10 Link Context 链接字典的内容。 4. Write Chain 写处理链。 4.1 ARC Writer Processor 这是一个把捉取到的结果进行写写的处理,存储为 ARC(网络档案文件格式)的文档。每个 Heritrix实例只有一个这样的些处理线程在运行。 4.2 Default Metadata Provider 一个合适爬虫元数据的标准集。 4.3 Mirror Writer Processor 镜像 写处理。把捉取到的结果以镜像的形式存储。 4.4 Writer Pool Processor 写处理池。这是一个虚拟的文件池处理的实现。是ARC Writer Processor的父类。 5. Post Process Chain 后处理链。 5.1 Crawl State Updater :爬虫状态更新程序。 这是在处理了一个URL后的一步,用来更新已经被处理的URL的信息。这些信息包括IP、robots等信息。 5.2 Frontier Scheduler :定制边界URL的查询调度表。增加想要的或者是链接到其他地的链接。 5.3 Links Scope r 链接范围。确定哪个链接在范围内。把链接转换成Crawl URL类型,然后去查看该Crawl URL是不是在范围内。 5.4 Low Disk Pause Processor 当系统的磁盘很少了的时候,暂停处理。 5.5 Supplementary Links Scope r 做完普通的处理后再追加一些补充的处理的类。 5. Frontier 边界,下一个URL决定器。 5.1 Adaptive Revisit Frontier 重新检索边界程序。这是一个重复访问指定URL的边界程序。等待时间是可以设置的。 5.2 Adaptive Revisit Host Queue 是Crawl URL的一个主要的队列,每个队列应该代表一个主机(虽然这不是必须的)。 5.3 Adaptive Revisit Queue List 管理着Adaptive Revisit Host Queue 组成的列表。 5.4 BDB Frontier BDB数据库边界。使用 Berkeley DB 来维护自己知道的主机。 5.5 BDB Multiple Work Queues 是一个用嵌入式数据库实现的。 5.6 BDB Work Queue 一个独立的,有着相同class Key的列表。 6. Toe Pool 线程池。 6.1 Toe Pool 是一个Toe Thread线程池。 6.2 Toe Thread 是一个线程。 7 Crawl Controller Implement 主控制程序。 该类集合了所有的类,它可以协调执行爬虫和提供高水平的界面给运行的爬虫。
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

寒水馨

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值