HDFS Client源码阅读
FileSystem的创建过程
完整代码流程如图
![image-20210825160846948](https://12img.oss-cn-hangzhou.aliyuncs.com/typeroa/image-20210825160846948.png)
fs = FileSystem.get(uri, conf, user);
通过FileSystem.get(uri,conf,user)方法获取FileSystem实例。
- get(uri,conf,user)方法首先验证Kerberos认证
- 获取用户组信息
- 调用FileSystem.get(uri,conf)获取FileSystem实例
![image-20210825145607408](https://12img.oss-cn-hangzhou.aliyuncs.com/typeroa/image-20210825145607408.png)
FileSystem.get(uri,conf);
- 获取schema和授权
- schema和授权为null时,返回默认FS(default.fs)
- 无授权时,如果schema符合默认schema,返回默认FS
- schema和授权都不为null
- 判断fs cache是否开启:
fs.%s.impl.disable.cache
- 未开启则调用createFileSystem
- 开启则调用CACHE.get
- 判断fs cache是否开启:
![image-20210825145615337](https://12img.oss-cn-hangzhou.aliyuncs.com/typeroa/image-20210825145615337.png)
![image-20210825145620261](https://12img.oss-cn-hangzhou.aliyuncs.com/typeroa/image-20210825145620261.png)
createFileSystem(uri,conf);
通过反射获取FileSystem实例,并完成初始化。
![image-20210825155919717](https://12img.oss-cn-hangzhou.aliyuncs.com/typeroa/image-20210825155919717.png)
CACHE.get(uri,conf);
return getInternal(uri, conf, key);
- 判断FS Cache中是否存在可复用FS
- 存在即返回
- 不存在创建一个新的FS
- 再次判断Cache中是否存在可复用FS(避免多线程创建多次)
- 无可复用FS时,将新创建的FS加入Cache,并返回
![image-20210825160419320](https://12img.oss-cn-hangzhou.aliyuncs.com/typeroa/image-20210825160419320.png)
![image-20210825160647548](https://12img.oss-cn-hangzhou.aliyuncs.com/typeroa/image-20210825160647548.png)