Namenode是hdfs的名字节点,保存着文件系统的元数据,响应客户端与DataNode的请求。
有如下线程为Namenode进行服务:
1、HeartbeatMonitor:心跳守护线程,该线程一直在进行与DataNode之间的心跳检测,默认是EXPIRE_INTERVAL = 10 * 60 * 1000 (10分钟)后,如果DataNode还没有发来心跳信号,就进行如下处理:
1)将该DataNode从心跳列表中剔除;
2)文件系统总的容量也响应减少;
3)将该DataNode的死块进行处理
synchronized void heartbeatCheck() {
synchronized (heartbeats) {
DatanodeInfo nodeInfo = null;
while ((heartbeats.size() > 0) &&
((nodeInfo = (DatanodeInfo) heartbeats.first()) != null) &&
(nodeInfo.lastUpdate() < System.currentTimeMillis() - EXPIRE_INTERVAL)) {
LOG.info("Lost heartbeat for " + nodeInfo.getName());
heartbeats.remove(nodeInfo);
synchronized (datanodeMap) {
datanodeMap.remove(nodeInfo.getName());
}
totalCapacity -= nodeInfo.getCapacity();
totalRemaining -= nodeInfo.getRemaining();
Block deadblocks[] = nodeInfo.getBlocks();
if (deadblocks != null) {
for (int i = 0; i < deadblocks.length; i++) {
removeStoredBlock(deadblocks[i], nodeInfo);
}
}
if (heartbeats.size() > 0) {
nodeInfo = (DatanodeInfo) heartbeats.first();
}
}
}
}
2)LeaseMonitor:租约守护线程,该线程一直进行租约的检查;
class LeaseMonitor implements Runnable {
public void run() {
while (fsRunning) {
synchronized (FSNamesystem.this) {
synchronized (leases) {
Lease top;
while ((sortedLeases.size() > 0) &&
((top = (Lease) sortedLeases.first()) != null)) {
if (top.expired()) {
top.releaseLocks();
leases.remove(top.holder);
LOG.info("Removing lease " + top + ", leases remaining: " + sortedLeases.size());
if (!sortedLeases.remove(top)) {
LOG.info("Unknown failure trying to remove " + top + " from lease set.");
}
} else {
break;
}
}
}
}
try {
Thread.sleep(2000);
} catch (InterruptedException ie) {
}
}
}
}
3、Listener:Socket监听线程,当监听到一个请求时,创建一个服务端的Connection线程;
public void run() {
LOG.info(getName() + ": starting");
while (running) {
try {
new Connection(socket.accept()).start(); // start a new connection
} catch (SocketTimeoutException e) { // ignore timeouts
} catch (Exception e) { // log all other exceptions
LOG.log(Level.INFO, getName() + " caught: " + e, e);
}
}
try {
socket.close();
} catch (IOException e) {
LOG.info(getName() + ": e=" + e);
}
LOG.info(getName() + ": exiting");
}
4、Connection:服务端connection线程,做如下几件事:
1)获取调用参数param;
2)new一个call对象,并放入调用队列callQueue;
3)唤醒调用队列callQueue,交给后面的Handler去处理;
while (running) {
int id;
try {
id = in.readInt(); // try to read an id
} catch (SocketTimeoutException e) {
continue;
}
if (LOG.isLoggable(Level.FINE))
LOG.fine(getName() + " got #" + id);
Writable param = makeParam(); // read param
param.readFields(in);
Call call = new Call(id, param, this);
synchronized (callQueue) {
callQueue.addLast(call); // queue the call
callQueue.notify(); // wake up a waiting handler
}
while (running && callQueue.size() >= maxQueuedCalls) {
synchronized (callDequeued) { // queue is full
callDequeued.wait(timeout); // wait for a dequeue
}
}
}
5、Handler:处理请求队列的守护线程,默认会起10个,当上述connection中请求队列被唤醒时,对队列中call真正进行处理;
Call call;
synchronized (callQueue) {
while (running && callQueue.size()==0) { // wait for a call
callQueue.wait(timeout);
}
if (!running) break;
call = (Call)callQueue.removeFirst(); // pop the queue
}
synchronized (callDequeued) { // tell others we've dequeued
callDequeued.notify();
}
if (LOG.isLoggable(Level.FINE))
LOG.fine(getName() + ": has #" + call.id + " from " +
call.connection.socket.getInetAddress().getHostAddress());
String error = null;
Writable value = null;
try {
value = call(call.param); // make the call
} catch (IOException e) {
LOG.log(Level.INFO, getName() + " call error: " + e, e);
error = getStackTrace(e);
} catch (Exception e) {
LOG.log(Level.INFO, getName() + " call error: " + e, e);
error = getStackTrace(e);
}
DataOutputStream out = call.connection.out;
synchronized (out) {
out.writeInt(call.id); // write call id
out.writeBoolean(error!=null); // write error flag
if (error != null)
value = new UTF8(error);
value.write(out); // write value
out.flush();
}