filestatus java,Java FileSystem.getFileStatus方法代码示例

这篇博客详细介绍了Apache Flink如何从指定路径加载Savepoint,并使用指定的类加载器解析其元数据。在检查路径无误后,通过FileSystem API获取文件状态,寻找元数据文件并进行读取。在读取过程中,检查魔术数字以确认文件类型,并根据版本号使用相应的序列化器进行反序列化。最后,构建并返回Savepoint及其元数据的流处理句柄。
摘要由CSDN通过智能技术生成

import org.apache.flink.core.fs.FileSystem; //导入方法依赖的package包/类

/**

* Loads the savepoint at the specified path. This methods returns the savepoint, as well as the

* handle to the metadata.

*

* @param savepointFileOrDirectory Path to the parent savepoint directory or the meta data file.

* @param classLoader The class loader used to resolve serialized classes from legacy savepoint formats.

* @return The loaded savepoint

*

* @throws IOException Failures during load are forwarded

*/

public static Tuple2 loadSavepointWithHandle(

String savepointFileOrDirectory,

ClassLoader classLoader) throws IOException {

checkNotNull(savepointFileOrDirectory, "savepointFileOrDirectory");

checkNotNull(classLoader, "classLoader");

Path path = new Path(savepointFileOrDirectory);

LOG.info("Loading savepoint from {}", path);

FileSystem fs = FileSystem.get(path.toUri());

FileStatus status = fs.getFileStatus(path);

// If this is a directory, we need to find the meta data file

if (status.isDir()) {

Path candidatePath = new Path(path, SAVEPOINT_METADATA_FILE);

if (fs.exists(candidatePath)) {

path = candidatePath;

LOG.info("Using savepoint file in {}", path);

} else {

throw new IOException("Cannot find meta data file in directory " + path

+ ". Please try to load the savepoint directly from the meta data file "

+ "instead of the directory.");

}

}

// load the savepoint

final Savepoint savepoint;

try (DataInputStream dis = new DataInputViewStreamWrapper(fs.open(path))) {

int magicNumber = dis.readInt();

if (magicNumber == MAGIC_NUMBER) {

int version = dis.readInt();

SavepointSerializer> serializer = SavepointSerializers.getSerializer(version);

savepoint = serializer.deserialize(dis, classLoader);

} else {

throw new RuntimeException("Unexpected magic number. This can have multiple reasons: " +

"(1) You are trying to load a Flink 1.0 savepoint, which is not supported by this " +

"version of Flink. (2) The file you were pointing to is not a savepoint at all. " +

"(3) The savepoint file has been corrupted.");

}

}

// construct the stream handle to the metadata file

// we get the size best-effort

long size = 0;

try {

size = fs.getFileStatus(path).getLen();

}

catch (Exception ignored) {

// we don't know the size, but we don't want to fail the savepoint loading for that

}

StreamStateHandle metadataHandle = new FileStateHandle(path, size);

return new Tuple2<>(savepoint, metadataHandle);

}

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值