2021SC@SDUSC hbase源码分析(五) HLog分析
目录
2021SC@SDUSC2021SC@SDUSC
2021SC@SDUSC2021SC@SDUSC
2021SC@SDUSC2021SC@SDUSC
HLog
默认情况下,所有写入操作(写入、更新以及删除)的数据都先以追加的形式写入HLog,再写入MemStore。大多数情况下,HLog并不会被读取,但如果RegionServer在某些异常情况下发生宕机,此时已经写入MemStore中但尚未f lush到磁盘的数据就会丢失,需要回放HLog补救丢失的数据。此外,HBase主从复制需要主集群将HLog日志发送给从集群,从集群在本地执行回放操作,完成集群之间的数据复制。
HLog结构
HLog日志格式
完整的WAL记录分WALKey
和KeyValue
两部分组成。
@InterfaceAudience.Private
protected void init(final byte[] encodedRegionName,
final TableName tablename,
long logSeqNum,
final long now,
List<UUID> clusterIds,
long nonceGroup,
long nonce,
MultiVersionConcurrencyControl mvcc,
NavigableMap<byte[], Integer> replicationScope,
Map<String, byte[]> extendedAttributes) {
this.sequenceId = logSeqNum;
this.writeTime = now;
this.clusterIds = clusterIds;
this.encodedRegionName = encodedRegionName;
this.tablename = tablename;
this.nonceGroup = nonceGroup;
this.nonce = nonce;
this.mvcc = mvcc;
if (logSeqNum != NO_SEQUENCE_ID) {
setSequenceId(logSeqNum);
}
this.replicationScope = replicationScope;
this.extendedAttributes = extendedAttributes;
}
以上为WalKeyImpl类中的初始化日志的方法。
HLog生命周期
(1)HLog构建
HBase的任何写入(更新、删除)操作都会先将记录追加写入到HLog文件中。
(2)HLog滚动
regionserver启动一个线程定期(由参数’hbase.regionserver.logroll.period’决定,默认1小时)对hlog进行滚动,重新创建一个新的hlog.,这样,regionserver上就会产生很多小的hlog 文件,hbase这样做的原因是当hbase的数据越来越多时,hlog的大小就会越来越大。当memorystore 刷写到磁盘时,过期的hlog 并没有作用便可删除,一个个小的hlog文件便于删除。
protected static final String WAL_ROLL_PERIOD_KEY = "hbase.regionserver.logroll.period";
this.rollPeriod = conf.getLong(WAL_ROLL_PERIOD_KEY, 3600000);
static MasterRegionWALRoller create(String name, Configuration conf, Abortable abortable,
FileSystem fs, Path walRootDir, Path globalWALRootDir, String archivedWALSuffix,
long rollPeriodMs, long flushSize) {
// we can not run with wal disabled, so force set it to true.
conf.setBoolean(WALFactory.WAL_ENABLED, true);
// we do not need this feature, so force disable it.
conf.setBoolean(AbstractFSWALProvider.SEPARATE_OLDLOGDIR, false);
conf.setLong(WAL_ROLL_PERIOD_KEY, rollPeriodMs);
// make the roll size the same with the flush size, as we only have one region here
conf.setLong(WALUtil.WAL_BLOCK_SIZE, flushSize * 2);
conf.setFloat(AbstractFSWAL.WAL_ROLL_MULTIPLIER, 0.5f);
return new MasterRegionWALRoller(name, conf, abortable, fs, walRootDir, globalWALRootDir,
archivedWALSuffix);
}
其中这段代码是对我们HLog进行处理,并加入调度定时执行:
@Override
public void run() {
updateTimeTrackingBeforeRun();
if (missedStartTime() && isScheduled()) {
onChoreMissedStartTime();
LOG.info("Chore: {} missed its start time", getName());
} else if (stopper.isStopped() || !isScheduled()) {
cancel(false);
cleanup();
LOG.info("Chore: {} was stopped", getName());
} else {
try {
long start = 0;
if (LOG.isDebugEnabled()) {
start = System.nanoTime();
}
if (!initialChoreComplete) {
initialChoreComplete = initialChore();
} else {
chore();
}
if (LOG.isDebugEnabled() && start > 0) {
long end = System.nanoTime();
LOG.debug("{} execution time: {} ms.", getName(),
TimeUnit.NANOSECONDS.toMillis(end - start));
}
} catch (Throwable t) {
LOG.error("Caught error", t);
if (this.stopper.isStopped()) {
cancel(false);
cleanup();
}
}
}
}
加入调度后会周期性执行 LogCleaner.chore() 方法
protected synchronized void chore() {
if (isDisabled() || isRunning()) {
LOG.warn("hbckChore is either disabled or is already running. Can't run the chore");
return;
}
regionInfoMap.clear();
disabledTableRegions.clear();
splitParentRegions.clear();
orphanRegionsOnRS.clear();
orphanRegionsOnFS.clear();
inconsistentRegions.clear();
checkingStartTimestamp = EnvironmentEdgeManager.currentTime();
running = true;
try {
loadRegionsFromInMemoryState();
loadRegionsFromRSReport();
try {
loadRegionsFromFS(scanForMergedParentRegions());
} catch (IOException e) {
LOG.warn("Failed to load the regions from filesystem", e);
}
saveCheckResultToSnapshot();
} catch (Throwable t) {
LOG.warn("Unexpected", t);
}
running = false;
}
(3)HLog失效
写入数据一旦从MemStore中落盘,对应的日志数据就会失效。为了方便处理,HBase中日志失效删除总是以文件为单位执行。查看某个HLog文件是否失效只需确认该HLog文件中所有日志记录对应的数据是否已经完成落盘,如果日志中所有日志记录已经落盘,则可以认为该日志文件失效。一旦日志文件失效,就会从WALs文件夹移动到oldWALs文件夹。注意此时HLog并没有被系统删除。
(4)HLog删除
Master后台会启动一个线程,每隔一段时间(参数’hbase.master.cleaner. interval’,默认1分钟)检查一次文件夹oldWALs下的所有失效日志文件,确认是否可以删除,确认可以删除之后执行删除操作。确认条件主要有两个:
•该HLog文件是否还在参与主从复制。对于使用HLog进行主从复制的业务,需要继续确认是否该HLog还在应用于主从复制。
•该HLog文件是否已经在OldWALs目录中存在10分钟。为了更加灵活地管理HLog生命周期,系统提供了参数设置日志文件的TTL(参数’hbase.master.logcleaner.ttl’,默认10分钟),默认情况下oldWALs里面的HLog文件最多可以再保存10分钟。
CleanerChore中的检查并删除方法(这里获取了所有oldWALs目录下的文件,并进行选择性删除):
private void traverseAndDelete(Path dir, boolean root, CompletableFuture<Boolean> result) {
try {
// 获取了所有oldWALs目录下的文件
List<FileStatus> allPaths = Arrays.asList(fs.listStatus(dir));
List<FileStatus> subDirs =
allPaths.stream().filter(FileStatus::isDirectory).collect(Collectors.toList());
List<FileStatus> files =
allPaths.stream().filter(FileStatus::isFile).collect(Collectors.toList());
// 调用checkAndDeleteFiles方法检查
boolean allFilesDeleted =
files.isEmpty() || deleteAction(() -> checkAndDeleteFiles(files), "files", dir);
//开始有条件的删除
List<CompletableFuture<Boolean>> futures = new ArrayList<>();
if (!subDirs.isEmpty()) {
sortByConsumedSpace(subDirs);
subDirs.forEach(subDir -> {
CompletableFuture<Boolean> subFuture = new CompletableFuture<>();
pool.execute(() -> traverseAndDelete(subDir.getPath(), false, subFuture));
futures.add(subFuture);
});
}
FutureUtils.addListener(
CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()])),
(voidObj, e) -> {
if (e != null) {
result.completeExceptionally(e);
return;
}
try {
boolean allSubDirsDeleted = futures.stream().allMatch(CompletableFuture::join);
boolean deleted = allFilesDeleted && allSubDirsDeleted && isEmptyDirDeletable(dir);
if (deleted && !root) {
deleted = deleteAction(() -> fs.delete(dir, false), "dir", dir);
}
result.complete(deleted);
} catch (Exception ie) {
result.completeExceptionally(ie);
}
});
} catch (Exception e) {
LOG.debug("Failed to traverse and delete the path: {}", dir, e);
result.completeExceptionally(e);
}
}
上面代码调用了 checkAndDeleteFiles(files) 方法,该方法的作用是:通过每个清理程序运行给定的文件,以查看是否应删除该文件,并在必要时将其删除。输入参数是所有oldWALs目录下的文件:
private boolean checkAndDeleteFiles(List<FileStatus> files) {
if (files == null) {
return true;
}
// first check to see if the path is valid
List<FileStatus> validFiles = Lists.newArrayListWithCapacity(files.size());
List<FileStatus> invalidFiles = Lists.newArrayList();
for (FileStatus file : files) {
if (validate(file.getPath())) {
validFiles.add(file);
} else {
LOG.warn("Found a wrongly formatted file: " + file.getPath() + " - will delete it.");
invalidFiles.add(file);
}
}
Iterable<FileStatus> deletableValidFiles = validFiles;
// check each of the cleaners for the valid files
for (T cleaner : cleanersChain) {
if (cleaner.isStopped() || this.getStopper().isStopped()) {
LOG.warn("A file cleaner" + this.getName() + " is stopped, won't delete any more files in:"
+ this.oldFileDir);
return false;
}
Iterable<FileStatus> filteredFiles = cleaner.getDeletableFiles(deletableValidFiles);
// trace which cleaner is holding on to each file
if (LOG.isTraceEnabled()) {
ImmutableSet<FileStatus> filteredFileSet = ImmutableSet.copyOf(filteredFiles);
for (FileStatus file : deletableValidFiles) {
if (!filteredFileSet.contains(file)) {
LOG.trace(file.getPath() + " is not deletable according to:" + cleaner);
}
}
}
deletableValidFiles = filteredFiles;
}
Iterable<FileStatus> filesToDelete = Iterables.concat(invalidFiles, deletableValidFiles);
return deleteFiles(filesToDelete) == files.size();
}
删除文件方法:
protected int deleteFiles(Iterable<FileStatus> filesToDelete) {
int deletedFileCount = 0;
for (FileStatus file : filesToDelete) {
Path filePath = file.getPath();
LOG.trace("Removing {} from archive", filePath);
try {
boolean success = this.fs.delete(filePath, false);
if (success) {
deletedFileCount++;
} else {
LOG.warn("Attempted to delete:" + filePath
+ ", but couldn't. Run cleaner chain and attempt to delete on next pass.");
}
} catch (IOException e) {
e = e instanceof RemoteException ?
((RemoteException)e).unwrapRemoteException() : e;
LOG.warn("Error while deleting: " + filePath, e);
}
}
return deletedFileCount;
}
总结
我们可以发现,删除的过程就是定期执行删除文件线程,并且属于有条件的删除。
HLog在写流程中文件生成后并不会永久的存储在系统中,它的使命完成后,文件会失效删除。