上一篇博客项目 & 多文件云传输自平衡(1) & 资源文件操作已经将整个关于资源信息,文件信息,片段头信息,文件读写传输操作等基本介绍完。接下来进入资源请求者想要获取资源的步骤过程了。
对于一个资源,其资源编号是由谁来确定呢?是资源发送者?还是资源接收者?
不,应该是资源初始拥有者。
资源初始拥有者为该资源确定资源编号,后面的普通拥有者没有权利修改这个资源编号。
资源通信流程
1.资源拥有者注册资源
如果没有资源,那么资源请求者,获取什么资源呢?
所以第一步建立一个资源拥有者身份。
资源拥有者首先需要和资源注册中心建立联系。
/**
* 资源发送者的上一层,资源拥有者,通过RMI与对端的接收者通信
*/
public class ResourceHolder {
private static RMIServer resourceHolderServer;//资源拥有者服务器
private static ResourceHolderResourcePool resourcePool;//资源拥有者的资源池,pojo类,curd.所以不放代码了
private static IResourceRegistryCenterAction action;//往资源注册中心的操作
private static Node me;//资源注册中心标记的我的位置
static {
//资源拥有者相对于资源注册中心,作为一个客户端,建立与注册中心的联系
RMIServerScanner.scanInterface("/src/main/resources/holder.mapping.xml");
try {
PropertiesParser.loadProperties("/src/main/resources/rmi.cfg.properties");
} catch (Exception e) {
e.printStackTrace();
}
resourcePool = new ResourceHolderResourcePool();
me = new Node();
try {
me.setIp(new String(InetAddress.getLocalHost().getAddress()));
me.setPort(Integer.parseInt(PropertiesParser.getValue("rmi_server_port")));
resourceHolderServer = new RMIServer();
RMIInitializer.init(resourceHolderServer, "/src/main/resources/rmi.cfg.properties");
String resourceRegistryCenterIp = PropertiesParser
.getValue("resource_registry_center_ip");
int resourceRegistryCenterPort = Integer.valueOf(PropertiesParser
.getValue("resource_registry_center_port"));
action = new RMIProxy(
new RMIClient(resourceRegistryCenterPort, resourceRegistryCenterIp))
.getRmiProxy(IResourceRegistryCenterAction.class);
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (PropertiesFileNotFoundException e) {
e.printStackTrace();
}
}
/**
* 获得资源注册中心得操作
* @return
*/
static IResourceRegistryCenterAction getResourceRegistryCenterAction() {
return action;
}
/**
* 通过资源编号,查找资源
* @param resourceId
* @return
*/
public static ResourceInfo getResourceInfoById(int resourceId) {
return resourcePool.getResourceInfo(resourceId);
}
/**
* 设置结点类型,初始拥有者 or 普通拥有者
* @param type
*/
public static void setNodeType(int type){
me.setType(type);
}
public static void addListener(IListener listener){
resourceHolderServer.addListener(listener);
}
public static Node getNode(){
return me;
}
/**
* 注册某资源
* @param resourceId
*/
public static void registryResource(int resourceId) {
action.registry(resourceId, me);
}
/**
* 注销某资源
* @param resourceId
*/
public static void logoutResource(int resourceId) {
action.logout(resourceId, me);
}
/**
* 注销所有资源
*/
public static void logoutResource() {
action.logout(me);
}
public static void startup() throws IOException {
if (resourceHolderServer.isStartup()) {
return;
}
resourceHolderServer.startup();
}
public static void shutdown() {
if (!resourceHolderServer.isStartup()) {
return;
}
resourceHolderServer.shutdown();
}
}
提供资源拥有者与资源注册中心,直接交互的操作
这里提供资源拥有向资源注册中心,注册或删除资源,
所以第一步,资源拥有者(已生成资源信息)向资源注册中心注册资源。
2.资源请求者请求资源
准确来说,一个完整的资源请求过程,确实是从资源请求者发起。有需要,才让注册的服务有用武之地。
2.1资源请求者获取资源拥有者列表
资源请求者,也就是资源接收者
/**
* 资源请求的开始
*/
public class ResourceReceiver {
private static final int DEFAULT_MAX_SENDER_COUNT = 5;
private static final int DEFAULT_RECEIVER_SERVER_PORT = 54192;
private static IResourceRegistryCenterAction resourceRegistryCenterAction;
static {
//同样资源请求者,也需要和资源注册中心建立联系
try {
PropertiesParser.loadProperties("/src/main/resources/rmi.cfg.properties");
String resourceRegistryCenterIp = PropertiesParser
.getValue("resource_registry_center_ip");
int resourceRegistryCenterPort = Integer.valueOf(PropertiesParser
.getValue("resource_registry_center_port"));
resourceRegistryCenterAction = new RMIProxy(
new RMIClient(resourceRegistryCenterPort, resourceRegistryCenterIp))
.getRmiProxy(IResourceRegistryCenterAction.class);
} catch (PropertiesFileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
private RMIClient rmiClient;
private IResourceHolderAction resourceHolderAction;
private ReceiverServer receiverServer;
private ResourceReceiverAddress me;
private ISenderSelectedStrategy senderSelectedStrategy;
private int maxSenderCount;
public ResourceReceiver() throws UnknownHostException {
this.senderSelectedStrategy = new DefaultSenderSelectedStrategy();
this.maxSenderCount = DEFAULT_MAX_SENDER_COUNT;
this.rmiClient.setIp(InetAddress.getLocalHost().getHostAddress());
this.rmiClient.setPort(DEFAULT_RECEIVER_SERVER_PORT);
this.rmiClient = new RMIClient();
this.rmiClient.setServerAddressPool(null);
this.resourceHolderAction = new RMIProxy(this.rmiClient)
.getRmiProxy(IResourceHolderAction.class);//资源拥有者工作的代理对象
}
2.2选择发送端 — 负载均衡实现点
每个节点拥有一个健康值,健康值(心跳值)越低,说明其状态越好,越高说明其负载过大。
资源拥有者向资源注册中心获取资源拥有者列表,通过接口选择不同策略,筛选出最终的资源发送者
其他负载均衡实现手段各异。
我只拥有健康值(心跳值)来判断发送端负载情况。
可以模仿Dubbo。建立轮询机制,
基于权重的负载均衡机制
dubbo默认使用,策略参数是“random”
数量足够时,按照权重分配比例。
基于权重的轮询负载均衡机制
轮询,固定化顺序。通过权重分配,固定化一轮中提供者的选取顺序。
等等。
2.3资源请求者切割资源,建立资源接收端链接资源发送端
3.资源拥有者发送资源
3.1资源拥有者建立资源发送端
/**
* 发送端
*/
public class SenderClient {
private static final String DEFAULT_RECEIVE_IP = "127.0.0.1";
private static final int DEFAULT_RECEIVE_PORT = 52946;
private String receiveIp;//接收者IP
private int receivePort;//接收者Port
private Socket socket;
private ResourceInfo resourceInfo;//资源拥有者,所以有该类
private List<SectionHeader> sectionList;//被分配的任务
private ISendAfterAction sendAfterAction;
public SenderClient(ResourceInfo resourceInfo, List<SectionHeader> sectionList) {
this.receiveIp = DEFAULT_RECEIVE_IP;
this.receivePort = DEFAULT_RECEIVE_PORT;
this.resourceInfo = resourceInfo;
this.sectionList = sectionList;
}
public SenderClient setReceiveIp(String receiveIp) {
this.receiveIp = receiveIp;
return this;
}
public SenderClient setSendAfterAction(ISendAfterAction sendAfterAction) {
this.sendAfterAction = sendAfterAction;
return this;
}
public SenderClient setReceivePort(int receivePort) {
this.receivePort = receivePort;
return this;
}
public void send() throws IOException {
this.socket = new Socket(this.receiveIp, this.receivePort);
SenderThread senderThread = new SenderThread(this.socket, this.resourceInfo, this.sectionList);
senderThread.startSend(sendAfterAction);//sendAfterAction可在上一层设置
}
}
3.2发送端建立发送线程
public class SenderThread implements Runnable{
private Socket socket;
private ResourceInfo resourceInfo;
private List<SectionHeader> headerList;
private DataOutputStream dos;
private SectionTransfer sectionTransfer;
private RandomAccessFilePool rafPool;
private ISendAfterAction afterSendAction;
public SenderThread(Socket socket, ResourceInfo resourceInfo,
List<SectionHeader> sectionList) throws IOException {
this.socket = socket;
this.resourceInfo = resourceInfo;
this.headerList = sectionList;
this.rafPool = new RandomAccessFilePool(this.resourceInfo);
this.dos = new DataOutputStream(this.socket.getOutputStream());
this.sectionTransfer = new SectionTransfer();
}
/**
* 开始发送
* @param iSendAfterAction
*/
public void startSend(ISendAfterAction iSendAfterAction) {
/**
* 排序的目的是,将同一个文件的片段放在一起,这样可以避免文件读写通道的频繁切换
*/
this.headerList.sort(new Comparator<SectionHeader>() {
@Override
public int compare(SectionHeader o1, SectionHeader o2) {
int res = o1.getFileId() - o2.getFileId();
if (res > 0) return 1;
if (res < 0) return -1;
return 0;
}
});
this.afterSendAction = iSendAfterAction;
this.afterSendAction.afterSend();
}
@Override
public void run() {
int oldFileId = -1;
for (SectionHeader section : this.headerList){
int fileId = section.getFileId();
long offset = section.getOffset();
long len = section.getLen();
try {
if (oldFileId != -1 && oldFileId != fileId){
this.rafPool.close(oldFileId);
}
RandomAccessFile raf = null;
raf = this.rafPool.getRandomAccessFile(fileId, "r");
byte[] buffer = FileAccessor.readSection(raf, offset, (int) len);//读出内容
this.sectionTransfer.sendSection(this.dos, section, buffer);
} catch (IOException e) {
e.printStackTrace();
}finally {
oldFileId = fileId;
}
}
try {
this.sectionTransfer.sendEND(this.dos);
//指定片段头发送完成
} catch (IOException e) {
e.printStackTrace();
}finally {
try {
this.rafPool.close(oldFileId);//关不读写通道
} catch (IOException e) {
e.printStackTrace();
}
close();
}
}
private void close() {
try {
if (this.dos != null){
this.dos.close();
}
} catch (IOException e) {
e.printStackTrace();
}finally {
this.dos = null;
}
if (this.socket != null || !this.socket.isClosed()){
try {
this.socket.close();
} catch (IOException e) {
e.printStackTrace();
}finally {
this.socket = null;
}
}
}
}
4.资源接收者接收资源
4.1资源接收者建立资源接收端
准确来说,在指定发送端发送信息后,就已经建立资源接收端。
/**
* 资源接收端
*/
public class ReceiverServer implements Runnable{
public static final int DEFAULT_RECEIVE_SERVER_PORT = 52946;
private ServerSocket receiverServer;
private int receiverPort;
private boolean goon;
private int senderCount;
private RandomAccessFilePool accessFilePool;
private UnreceiveSectionPool unreceivePool;
public ReceiverServer(ResourceInfo resourceInfo, int senderCount) {
this.receiverPort = DEFAULT_RECEIVE_SERVER_PORT;
this.senderCount = senderCount;
this.accessFilePool = new RandomAccessFilePool(resourceInfo);
this.unreceivePool = initReceiveSectionPool(resourceInfo);
}
public ReceiverServer setReceiverPort(int receiverPort) {
this.receiverPort = receiverPort;
return this;
}
private UnreceiveSectionPool initReceiveSectionPool(ResourceInfo resourceInfo) {
UnreceiveSectionPool pool = new UnreceiveSectionPool();
FileInfoAccessor fileInfoAcs = new FileInfoAccessor(resourceInfo.getFileInfoPool());
while (fileInfoAcs.hasNext()){
FileInfo file = fileInfoAcs.next();
pool.addTargetFile(file.getId(), file.getSize());
}
return pool;
}
@Override
public void run() {
int linkedSenderCount = 0;
// TODO 这里可以考虑:某一个发送端在发送前,已连接连接时,
// 就已经“崩溃”了的情况。
// 处理方法提示:进行超时处理即可。
while (linkedSenderCount < this.senderCount) {
try {
Socket socket = this.receiverServer.accept();
new ReceiverThread(socket, this.accessFilePool, this.unreceivePool);
linkedSenderCount++;
} catch (IOException e) {
this.goon = false;
}
}
}
public void startup() throws IOException {
if (this.goon){
return;
}
this.receiverServer = new ServerSocket(this.receiverPort);
this.goon = true;
new Thread(this).start();
}
public void shutdown() {
if (this.goon == false) {
return;
}
close();
}
private void close() {
try {
if (this.receiverServer != null && !this.receiverServer.isClosed()) {
this.receiverServer.close();
}
} catch (IOException e) {
e.printStackTrace();
} finally {
this.receiverServer = null;
}
}
}
4.2资源接收端建立资源接收线程,接收资源,完成读写
/**
* 资源接收线程,接收资源,进行资源读写
*/
public class ReceiverThread implements Runnable{
private Socket socket;
private DataInputStream dis;
private SectionTransfer sectionTransfer;
private volatile boolean goon;
public ReceiverThread(Socket socket, RandomAccessFilePool accessFilePool,
UnreceiveSectionPool unreceivePool) {
this.socket = socket;
this.sectionTransfer = new SectionTransfer();
this.sectionTransfer.setSectionProcessor(
new SectionProcessor(accessFilePool, unreceivePool));
this.goon = true;
new Thread(this).start();
}
@Override
public void run() {
while (this.goon) {
try {
this.sectionTransfer.receiveSection(this.dis);
} catch (Exception e) {
if (this.goon == false) {
break;
}
// TODO 接收失败,应该考虑包括断点续传在内的一些逻辑处理
this.goon = false;
}
}
close();
}
class SectionProcessor implements ISectionProcessor {
private RandomAccessFilePool accessFilePool;
private UnreceiveSectionPool unreceivePool;
public SectionProcessor(RandomAccessFilePool accessFilePool,
UnreceiveSectionPool unreceivePool) {
this.accessFilePool = accessFilePool;
this.unreceivePool = unreceivePool;
}
@Override
public void processSection(SectionHeader sectionHeader, byte[] context) throws Exception {
int fileId = sectionHeader.getFileId();
if (fileId == SectionTransfer.END){
ReceiverThread.this.close();
}
RandomAccessFile raf = this.accessFilePool.getRandomAccessFile(fileId, "rw");
//文件读写
FileAccessor.writeSection(raf, sectionHeader.getOffset(), context);
UnreceiveSection unreceiveSection = this.unreceivePool.getUnreceiveSection(fileId);
boolean isAllReceive = unreceiveSection.receiveSection(sectionHeader);
if (isAllReceive){
this.accessFilePool.close(fileId);
}
}
}
private void close() {
this.goon = false;
try {
if (this.dis != null) {
this.dis.close();
}
} catch (IOException e) {
} finally {
this.dis = null;
}
try {
if (this.socket != null && !this.socket.isClosed()) {
this.socket.close();
}
} catch (IOException e) {
} finally {
this.socket = null;
}
}
}
5.成为新的资源拥有者,向资源注册中心注册
此时注册的是,普通拥有者
小结
到这里,项目的核心步骤已经基本上完成了。或许会有疑问,资源接收者的资源信息类,应该从哪里获取?
个人的想法,有这些
- 可以在资源注册中心那里,建立一个资源信息类池,当资源初始拥有者注册资源时,同将资源放入池中。资源请求者获取资源服务时,注册中心,将该资源信息类发给资源接收者。资源普通拥有者根据类型,无需上传资源。缺点是,加剧了资源注册中心的负担。
- 由于拥有资源初始拥有者与普通拥有者,可以选择一个发送者,先向获得资源信息类。在指定资源信息发送。缺点,如果资源信息过大,无法完整发送。
- APP层建立资源信息模块,因为使用者接触的就是资源列表。由APP层传递在于资源信息,在传递给底层。
持续改进中