2021SC@SDUSC hbase源码分析（五）HLog分析

2021SC@SDUSC hbase源码分析（五） HLog分析

2021SC@SDUSC2021SC@SDUSC
2021SC@SDUSC2021SC@SDUSC
2021SC@SDUSC2021SC@SDUSC

HLog

默认情况下，所有写入操作（写入、更新以及删除）的数据都先以追加的形式写入HLog，再写入MemStore。大多数情况下，HLog并不会被读取，但如果RegionServer在某些异常情况下发生宕机，此时已经写入MemStore中但尚未f lush到磁盘的数据就会丢失，需要回放HLog补救丢失的数据。此外，HBase主从复制需要主集群将HLog日志发送给从集群，从集群在本地执行回放操作，完成集群之间的数据复制。

HLog结构

HLog日志格式

完整的WAL记录分WALKey和KeyValue两部分组成。

@InterfaceAudience.Private
protected void init(final byte[] encodedRegionName,
                    final TableName tablename,
                    long logSeqNum,
                    final long now,
                    List<UUID> clusterIds,
                    long nonceGroup,
                    long nonce,
                    MultiVersionConcurrencyControl mvcc,
                    NavigableMap<byte[], Integer> replicationScope,
                    Map<String, byte[]> extendedAttributes) {
  this.sequenceId = logSeqNum;
  this.writeTime = now;
  this.clusterIds = clusterIds;
  this.encodedRegionName = encodedRegionName;
  this.tablename = tablename;
  this.nonceGroup = nonceGroup;
  this.nonce = nonce;
  this.mvcc = mvcc;
  if (logSeqNum != NO_SEQUENCE_ID) {
    setSequenceId(logSeqNum);
  }
  this.replicationScope = replicationScope;
  this.extendedAttributes = extendedAttributes;
}

以上为WalKeyImpl类中的初始化日志的方法。

HLog生命周期

（1）HLog构建

HBase的任何写入（更新、删除）操作都会先将记录追加写入到HLog文件中。

（2）HLog滚动

regionserver启动一个线程定期（由参数’hbase.regionserver.logroll.period’决定，默认1小时）对hlog进行滚动，重新创建一个新的hlog.,这样，regionserver上就会产生很多小的hlog 文件，hbase这样做的原因是当hbase的数据越来越多时，hlog的大小就会越来越大。当memorystore 刷写到磁盘时，过期的hlog 并没有作用便可删除，一个个小的hlog文件便于删除。

protected static final String WAL_ROLL_PERIOD_KEY = "hbase.regionserver.logroll.period";

this.rollPeriod = conf.getLong(WAL_ROLL_PERIOD_KEY, 3600000);

static MasterRegionWALRoller create(String name, Configuration conf, Abortable abortable,
  FileSystem fs, Path walRootDir, Path globalWALRootDir, String archivedWALSuffix,
  long rollPeriodMs, long flushSize) {
  // we can not run with wal disabled, so force set it to true.
  conf.setBoolean(WALFactory.WAL_ENABLED, true);
  // we do not need this feature, so force disable it.
  conf.setBoolean(AbstractFSWALProvider.SEPARATE_OLDLOGDIR, false);
  conf.setLong(WAL_ROLL_PERIOD_KEY, rollPeriodMs);
  // make the roll size the same with the flush size, as we only have one region here
  conf.setLong(WALUtil.WAL_BLOCK_SIZE, flushSize * 2);
  conf.setFloat(AbstractFSWAL.WAL_ROLL_MULTIPLIER, 0.5f);
  return new MasterRegionWALRoller(name, conf, abortable, fs, walRootDir, globalWALRootDir,
    archivedWALSuffix);
}

其中这段代码是对我们HLog进行处理，并加入调度定时执行：

@Override
public void run() {
  updateTimeTrackingBeforeRun();
  if (missedStartTime() && isScheduled()) {
    onChoreMissedStartTime();
    LOG.info("Chore: {} missed its start time", getName());
  } else if (stopper.isStopped() || !isScheduled()) {
    cancel(false);
    cleanup();
    LOG.info("Chore: {} was stopped", getName());
  } else {
    try {
      long start = 0;
      if (LOG.isDebugEnabled()) {
        start = System.nanoTime();
      }
      if (!initialChoreComplete) {
        initialChoreComplete = initialChore();
      } else {
        chore();
      }
      if (LOG.isDebugEnabled() && start > 0) {
        long end = System.nanoTime();
        LOG.debug("{} execution time: {} ms.", getName(),
          TimeUnit.NANOSECONDS.toMillis(end - start));
      }
    } catch (Throwable t) {
      LOG.error("Caught error", t);
      if (this.stopper.isStopped()) {
        cancel(false);
        cleanup();
      }
    }
  }
}

加入调度后会周期性执行 LogCleaner.chore() 方法

protected synchronized void chore() {
  if (isDisabled() || isRunning()) {
    LOG.warn("hbckChore is either disabled or is already running. Can't run the chore");
    return;
  }
  regionInfoMap.clear();
  disabledTableRegions.clear();
  splitParentRegions.clear();
  orphanRegionsOnRS.clear();
  orphanRegionsOnFS.clear();
  inconsistentRegions.clear();
  checkingStartTimestamp = EnvironmentEdgeManager.currentTime();
  running = true;
  try {
    loadRegionsFromInMemoryState();
    loadRegionsFromRSReport();
    try {
      loadRegionsFromFS(scanForMergedParentRegions());
    } catch (IOException e) {
      LOG.warn("Failed to load the regions from filesystem", e);
    }
    saveCheckResultToSnapshot();
  } catch (Throwable t) {
    LOG.warn("Unexpected", t);
  }
  running = false;
}

（3）HLog失效

写入数据一旦从MemStore中落盘，对应的日志数据就会失效。为了方便处理，HBase中日志失效删除总是以文件为单位执行。查看某个HLog文件是否失效只需确认该HLog文件中所有日志记录对应的数据是否已经完成落盘，如果日志中所有日志记录已经落盘，则可以认为该日志文件失效。一旦日志文件失效，就会从WALs文件夹移动到oldWALs文件夹。注意此时HLog并没有被系统删除。

（4）HLog删除

Master后台会启动一个线程，每隔一段时间（参数’hbase.master.cleaner. interval’，默认1分钟）检查一次文件夹oldWALs下的所有失效日志文件，确认是否可以删除，确认可以删除之后执行删除操作。确认条件主要有两个：

•该HLog文件是否还在参与主从复制。对于使用HLog进行主从复制的业务，需要继续确认是否该HLog还在应用于主从复制。

•该HLog文件是否已经在OldWALs目录中存在10分钟。为了更加灵活地管理HLog生命周期，系统提供了参数设置日志文件的TTL（参数’hbase.master.logcleaner.ttl’，默认10分钟），默认情况下oldWALs里面的HLog文件最多可以再保存10分钟。

CleanerChore中的检查并删除方法（这里获取了所有oldWALs目录下的文件，并进行选择性删除）：

private void traverseAndDelete(Path dir, boolean root, CompletableFuture<Boolean> result) {
  try {
    // 获取了所有oldWALs目录下的文件
    List<FileStatus> allPaths = Arrays.asList(fs.listStatus(dir));
    List<FileStatus> subDirs =
        allPaths.stream().filter(FileStatus::isDirectory).collect(Collectors.toList());
    List<FileStatus> files =
        allPaths.stream().filter(FileStatus::isFile).collect(Collectors.toList());
    // 调用checkAndDeleteFiles方法检查
    boolean allFilesDeleted =
        files.isEmpty() || deleteAction(() -> checkAndDeleteFiles(files), "files", dir);

    //开始有条件的删除
    List<CompletableFuture<Boolean>> futures = new ArrayList<>();
    if (!subDirs.isEmpty()) {
      sortByConsumedSpace(subDirs);
      subDirs.forEach(subDir -> {
        CompletableFuture<Boolean> subFuture = new CompletableFuture<>();
        pool.execute(() -> traverseAndDelete(subDir.getPath(), false, subFuture));
        futures.add(subFuture);
      });
    }
    FutureUtils.addListener(
      CompletableFuture.allOf(futures.toArray(new CompletableFuture[futures.size()])),
      (voidObj, e) -> {
        if (e != null) {
          result.completeExceptionally(e);
          return;
        }
        try {
          boolean allSubDirsDeleted = futures.stream().allMatch(CompletableFuture::join);
          boolean deleted = allFilesDeleted && allSubDirsDeleted && isEmptyDirDeletable(dir);
          if (deleted && !root) {
            deleted = deleteAction(() -> fs.delete(dir, false), "dir", dir);
          }
          result.complete(deleted);
        } catch (Exception ie) {
          result.completeExceptionally(ie);
        }
      });
  } catch (Exception e) {
    LOG.debug("Failed to traverse and delete the path: {}", dir, e);
    result.completeExceptionally(e);
  }
}

上面代码调用了 checkAndDeleteFiles(files) 方法，该方法的作用是：通过每个清理程序运行给定的文件，以查看是否应删除该文件，并在必要时将其删除。输入参数是所有oldWALs目录下的文件：

private boolean checkAndDeleteFiles(List<FileStatus> files) {
  if (files == null) {
    return true;
  }

  // first check to see if the path is valid
  List<FileStatus> validFiles = Lists.newArrayListWithCapacity(files.size());
  List<FileStatus> invalidFiles = Lists.newArrayList();
  for (FileStatus file : files) {
    if (validate(file.getPath())) {
      validFiles.add(file);
    } else {
      LOG.warn("Found a wrongly formatted file: " + file.getPath() + " - will delete it.");
      invalidFiles.add(file);
    }
  }

  Iterable<FileStatus> deletableValidFiles = validFiles;
  // check each of the cleaners for the valid files
  for (T cleaner : cleanersChain) {
    if (cleaner.isStopped() || this.getStopper().isStopped()) {
      LOG.warn("A file cleaner" + this.getName() + " is stopped, won't delete any more files in:"
          + this.oldFileDir);
      return false;
    }

    Iterable<FileStatus> filteredFiles = cleaner.getDeletableFiles(deletableValidFiles);

    // trace which cleaner is holding on to each file
    if (LOG.isTraceEnabled()) {
      ImmutableSet<FileStatus> filteredFileSet = ImmutableSet.copyOf(filteredFiles);
      for (FileStatus file : deletableValidFiles) {
        if (!filteredFileSet.contains(file)) {
          LOG.trace(file.getPath() + " is not deletable according to:" + cleaner);
        }
      }
    }

    deletableValidFiles = filteredFiles;
  }

  Iterable<FileStatus> filesToDelete = Iterables.concat(invalidFiles, deletableValidFiles);
  return deleteFiles(filesToDelete) == files.size();
}

删除文件方法：

protected int deleteFiles(Iterable<FileStatus> filesToDelete) {
  int deletedFileCount = 0;
  for (FileStatus file : filesToDelete) {
    Path filePath = file.getPath();
    LOG.trace("Removing {} from archive", filePath);
    try {
      boolean success = this.fs.delete(filePath, false);
      if (success) {
        deletedFileCount++;
      } else {
        LOG.warn("Attempted to delete:" + filePath
            + ", but couldn't. Run cleaner chain and attempt to delete on next pass.");
      }
    } catch (IOException e) {
      e = e instanceof RemoteException ?
                ((RemoteException)e).unwrapRemoteException() : e;
      LOG.warn("Error while deleting: " + filePath, e);
    }
  }
  return deletedFileCount;
}

总结

我们可以发现，删除的过程就是定期执行删除文件线程，并且属于有条件的删除。

HLog在写流程中文件生成后并不会永久的存储在系统中，它的使命完成后，文件会失效删除。