先上源码 https://github.com/sougou1323/ServerCat
觉得还可以的话,给个start吧(斜眼笑)。
1. 架构
整个系统分为两个部分,server和server mc。
其中server负责网络通信部分和协议的解析,主要使用Java NIO 编写。
server mc将mvc中的v去掉了,个人感觉现在都是前后端结偶,还提出了大前端的概念(主要是懒的实现视图渲染了,哈哈),所以系统中没有提供对视图渲染的支持,只提供了json格式的数据交互。
2. server
server中模仿了tomcat,使用Lifecycle类对对象的声明周期进行统一管理,后面单独写文章来讲。
server使用Java NIO编写 ,使用了reactor线程模型,核心类主要有三个
- EndPoint
- Acceptor
- Poller
其中,Acceptor负责监听连接,当有新的连接到达后,将SocketChannel包装成PollerEvent,注册到Poller的event队列中,Poller检测到数组加入了新的Poller事件,将SocketChannel注册到selector,监听事件。
下面我们看一下EndPoint 和 Acceptor
public class NioEndPoint extends AbstractEndPoint {
private Acceptor acceptor;
private int pollerCount = Math.min(2, Runtime.getRuntime().availableProcessors());
private AtomicInteger pollerRotate = new AtomicInteger(0);
private Poller[] pollers;
private Executor pool;
private ServerSocketChannel ssChannel;
public NioEndPoint() {
this(8080);
}
public NioEndPoint(int port) {
this.port = port;
}
@Override
public void execute(Wrapper attachment) {
pool.execute(new NioProcessor(attachment));
}
public void registerToPoller(SocketChannel client) {
new Thread(() -> getPoller().register(client)).start();
}
private Poller getPoller() {
int id = Math.abs(pollerRotate.incrementAndGet() % pollerCount);
return pollers[id];
}
@Override
protected void initInternal() {
pool = new ThreadPoolExecutor(
100,
120,
1,
TimeUnit.SECONDS,
new ArrayBlockingQueue<>(200),
new ThreadFactory() {
private int count;
@Override
public Thread newThread(Runnable r) {
return new Thread(r, "workerThread-" + count++);
}
},
new ThreadPoolExecutor.CallerRunsPolicy());
try {
this.ssChannel = ServerSocketChannel.open();
ssChannel.bind(new InetSocketAddress(port));
ssChannel.configureBlocking(true);
} catch (IOException e) {
e.printStackTrace();
log.error("ssChannel IOException");
}
}
@Override
protected void startInternal() {
initAcceptor();
initPoller();
}
@Override
protected void stopInternal() {
}
/**
* 初始化Acceptor
*/
private void initAcceptor() {
String acceptorName = "NIOAcceptor";
acceptor = new Acceptor(acceptorName);
Thread acceptorThread = new Thread(acceptor, acceptorName);
acceptorThread.setDaemon(true);
acceptorThread.start();
}
/**
* 初始化Poller
*/
private void initPoller() {
pollers = new Poller[pollerCount];
for (int i = 0; i < pollerCount; i++) {
String pollerName = "NIOPoller-" + i;
Poller nioPoller = null;
try {
nioPoller = new Poller(this, pollerName);
} catch (IOException e) {
e.printStackTrace();
}
Thread pollerThread = new Thread(nioPoller, pollerName);
pollerThread.setDaemon(true);
pollerThread.start();
pollers[i] = nioPoller;
}
}
class Acceptor implements Runnable {
private String name;
public Acceptor(String name) {
this.name = name;
}
@Override
public void run() {
log.info(Thread.currentThread().getName() + "[Acceptor:" + name + "]开始监听, 端口" + NioEndPoint.this.getPort());
while (getState() == LifecycleState.STARTED || getState() == LifecycleState.STARTING_PREP) {
SocketChannel client;
try {
client = ssChannel.accept();
log.info("Acceptor[" + name + "]接到请求...[" + client + "]");
NioEndPoint.this.registerToPoller(client);
} catch (IOException e) {
log.error("Acceptor[" + name + "] IOException");
}
}
}
}
}
在Endpoint的start()方法中,对Acceptor和Poller进行了初始化。
@Override
protected void startInternal() {
initAcceptor();
initPoller();
}
Acceptor初始化很简单,就是创建一个线程,运行起来。
Poller的初始化也很简单,EndPoint中持有一个Poller数组,首先要对Poller数组进行初始化,之后遍历Poller数组将所有的Poller运行起来。
private void initAcceptor() {
String acceptorName = "NIOAcceptor";
acceptor = new Acceptor(acceptorName);
Thread acceptorThread = new Thread(acceptor, acceptorName);
acceptorThread.setDaemon(true);
acceptorThread.start();
}
private void initPoller() {
pollers = new Poller[pollerCount];
for (int i = 0; i < pollerCount; i++) {
String pollerName = "NIOPoller-" + i;
Poller nioPoller = null;
try {
nioPoller = new Poller(this, pollerName);
} catch (IOException e) {
e.printStackTrace();
}
Thread pollerThread = new Thread(nioPoller, pollerName);
pollerThread.setDaemon(true);
pollerThread.start();
pollers[i] = nioPoller;
}
}
在Endpoint的初始化方法中,监听端口。
Acceptor的run方法中,对ServerSocketChannel循环查询,当有新的连接进来后,注册到Poller的event队列中。
try {
this.ssChannel = ServerSocketChannel.open();
ssChannel.bind(new InetSocketAddress(port));
ssChannel.configureBlocking(true);
} catch (IOException e) {
e.printStackTrace();
log.error("ssChannel IOException");
}
public void run() {
log.info(Thread.currentThread().getName() + "[Acceptor:" + name + "]开始监听, 端口" + NioEndPoint.this.getPort());
while (getState() == LifecycleState.STARTED || getState() == LifecycleState.STARTING_PREP) {
SocketChannel client;
try {
client = ssChannel.accept();
log.info("Acceptor[" + name + "]接到请求...[" + client + "]");
NioEndPoint.this.registerToPoller(client);
} catch (IOException e) {
log.error("Acceptor[" + name + "] IOException");
}
}
}
下面再看一下Poller
public class Poller implements Runnable{
private EndPoint server;
private String name;
private Queue<PollerEvent> events;
private Selector selector;
private final Lock LOCK = new ReentrantLock();
private final Condition NOT_EMPTY = LOCK.newCondition();
public Poller(EndPoint endPoint, String name) throws IOException {
this.server = endPoint;
this.name = name;
events = new ConcurrentLinkedQueue<>();
selector = Selector.open();
}
public void register(SocketChannel client) {
LOCK.lock();
Wrapper wrapper = new NioSocketWrapper(server, client, this);
events.offer(new PollerEvent(wrapper));
NOT_EMPTY.signalAll();
LOCK.unlock();
}
@Override
public void run() {
log.info("Poller[" + name + "] 等待连接");
while (server.getState() == LifecycleState.STARTING_PREP || server.getState() == LifecycleState.STARTED) {
LOCK.lock();
try {
while (events.size() == 0) {
NOT_EMPTY.await();
}
registerEvent();
log.info("PollerEvent[" + name + "]注册完毕");
selector.select();
Iterator<SelectionKey> selectionKeyIterator = selector.selectedKeys().iterator();
while (selectionKeyIterator.hasNext()) {
SelectionKey key = selectionKeyIterator.next();
if (key.isReadable()) {
Wrapper attachment = (Wrapper) key.attachment();
if (attachment != null) {
processSocket(attachment);
}
}
selectionKeyIterator.remove();
}
} catch (InterruptedException | IOException e) {
e.printStackTrace();
} finally {
LOCK.unlock();
}
}
}
private void processSocket(Wrapper attachment) {
server.execute(attachment);
}
private void registerEvent() {
if (events.size() != 0) {
for (int i = 0; i < events.size(); i++) {
events.poll().run();
}
}
}
public Selector getSelector() {
return selector;
}
}
下面是Poller的register方法,将PollerEvent放入event队列后,将会唤醒在run方法中阻塞的线程,进行处理。
public void register(SocketChannel client) {
LOCK.lock();
Wrapper wrapper = new NioSocketWrapper(server, client, this);
events.offer(new PollerEvent(wrapper));
NOT_EMPTY.signalAll();
LOCK.unlock();
}
下面是Poller的run方法,当线程被唤醒后,循环查询。当有读事件发生,将事件的内容包装成Processor丢到线程池中运行(Processor继承了Runnable接口)。
public void run() {
log.info("Poller[" + name + "] 等待连接");
while (server.getState() == LifecycleState.STARTING_PREP || server.getState() == LifecycleState.STARTED) {
LOCK.lock();
try {
while (events.size() == 0) {
NOT_EMPTY.await();
}
registerEvent();
log.info("PollerEvent[" + name + "]注册完毕");
selector.select();
Iterator<SelectionKey> selectionKeyIterator = selector.selectedKeys().iterator();
while (selectionKeyIterator.hasNext()) {
SelectionKey key = selectionKeyIterator.next();
if (key.isReadable()) {
Wrapper attachment = (Wrapper) key.attachment();
if (attachment != null) {
processSocket(attachment);
}
}
selectionKeyIterator.remove();
}
} catch (InterruptedException | IOException e) {
e.printStackTrace();
} finally {
LOCK.unlock();
}
}
}
private void processSocket(Wrapper attachment) {
server.execute(attachment);
}
Processor的run方法中解析了http协议,并调用adapter适配器。
public void run() {
NioSocketWrapper nioSocketWrapper = (NioSocketWrapper) wrapper;
// 解析http协议
Http11Processor httpProcessor = new Http11Processor();
Request request = httpProcessor.process(nioSocketWrapper);
Response response = new Response();
new Adapter(nioSocketWrapper).service(request, response);
}
adapter来调用DispatcherServlet,连接部分处理完毕,进入server mc部分。
public void service(Request request, Response response) {
// 调用DispatcherServlet
new DispatcherServlet().service(request, response);
try {
ByteBuffer[] buffer = response.getResponseByteBuffer();
wrapper.getClient().write(buffer);
} catch (IOException e) {
e.printStackTrace();
}
}
3. server mc
先看一下DispatcherServlet
@WebServlet(urlPatterns = "/*")
public class DispatcherServlet implements MyServlet {
@Override
public void init() {
HelpLoader.init();
}
@Override
public void service(Request request, Response response) {
String method = request.getMethod().name().toUpperCase();
String path = request.getPath();
Handler handler = ControllerHelper.getHandler(method, path);
if (handler != null) {
// 获取对象实例
Class<?> controllerClass = handler.getControllerClass();
Object controllerBean = BeanHelper.getBean(controllerClass);
Method actionMethod = handler.getControllerMethod();
Object result = ReflectionUtil.invokeMethod(controllerBean, actionMethod);
handleDataResult(new Data(result), response);
}
}
/**
* 返回json数据
* @param data 数据
* @param response response
*/
@Override
public void handleDataResult(Data data, Response response) {
Object model = data.getModel();
if (model != null) {
response.setContentType("application/json");
String json = JSON.toJSON(model).toString();
response.setBody(json.getBytes(StandardCharsets.UTF_8));
}
}
}
@WebServlet 是servlet规范的注解,学过servlet的同学肯定不陌生了。
DispatcherServlet主要做了三件事
- 初始化容器
- 通过调用相应的业务方法
- 处理结果
首先,DispatcherServlet的init方法调用了HelpLoader的init方法
@Override
public void init() {
HelpLoader.init();
}
public class HelpLoader {
public static void init() {
Class<?> [] classes = {ClassHelper.class, BeanHelper.class, AopHelper.class, IocHelper.class, ControllerHelper.class};
for (Class<?> cls: classes) {
ClassUtil.loadClass(cls.getName());
}
}
}
我们可以看到,HelpLoader的init方法加载了一些类,我们主要看一下BeanHelper和ControllerHelper
首先是BeanHelper
public class BeanHelper {
private static final Map<Class<?>, Object> BEAN_MAP = new HashMap<>();
static {
Set<Class<?>> beanClassSet = ClassHelper.getBeanClassSet();
for (Class<?> beanClass : beanClassSet) {
Object obj = ReflectionUtil.newInstance(beanClass);
BEAN_MAP.put(beanClass, obj);
}
}
public static Map<Class<?>, Object> getBeanMap() {
return BEAN_MAP;
}
public static <T> T getBean(Class<?> cls) {
if (!BEAN_MAP.containsKey(cls)) {
throw new RuntimeException("can not get bean by class : " + cls);
}
return (T) BEAN_MAP.get(cls);
}
public static void setBean(Class<?> cls, Object obj) {
BEAN_MAP.put(cls, obj);
}
}
在BeanHelper的静态代码块中,首先使用ClassHelper获取了标记@Controller和@Service注解的类,具体实现如下。
public class ClassHelper {
private static final Set<Class<?>> CLASS_SET ;
static {
String basePackage = ConfigHelper.getAppBasePackage();
CLASS_SET = ClassUtil.getClassSet(basePackage);
}
public static Set<Class<?>> getBeanClassSet() {
Set<Class<?>> beanClassSet = new HashSet<>();
beanClassSet.addAll(getServiceClassSet());
beanClassSet.addAll(getControllerClassSet());
return beanClassSet;
}
private static Collection<? extends Class<?>> getServiceClassSet() {
Set<Class<?>> classSet = new HashSet<>();
for (Class<?> cls : CLASS_SET) {
if (cls.isAnnotationPresent(Service.class)) {
classSet.add(cls);
}
}
return classSet;
}
public static Set<Class<?>> getControllerClassSet() {
Set<Class<?>> classSet = new HashSet<>();
for (Class<?> cls :CLASS_SET) {
if (cls.isAnnotationPresent(Controller.class)) {
classSet.add(cls);
}
}
return classSet;
}
}
之后,实例化对象,保存到BEAN_MAP中。
ControllerHelper在静态代码块中将request和handler保存在REQUEST_MAP中,方便后面按照路径进行映射。其中,request封装了请求的method(指的是GET、POST。。。后面统称method)和请求路径。handler封装了标记@Controller注解的类的Class对象和请求路径对应的方法。
public class ControllerHelper {
private static final Map<Request, Handler> REQUEST_MAP = new HashMap<>();
static {
// 获取 controller class set
Set<Class<?>> controllerClassSet = ClassHelper.getControllerClassSet();
if (CollectionUtils.isNotEmpty(controllerClassSet)) {
for (Class<?> controllerClass : controllerClassSet) {
// 遍历每一个controller class
Method[] methods = controllerClass.getDeclaredMethods();
if (ArrayUtils.isNotEmpty(methods)) {
for (Method method : methods) {
if (method.isAnnotationPresent(RequestMapping.class)) {
// 获取controller class 的每一个 有RequestMapping注解的 方法
RequestMapping requestMapping = method.getAnnotation(RequestMapping.class);
String requestPath = requestMapping.value();
String requestMethod = requestMapping.method().name();
// 封装请求路径path、请求方法method 保存到 REQUEST_MAP 中
Request request = new Request(requestMethod, requestPath);
Handler handler = new Handler(controllerClass, method);
REQUEST_MAP.put(request, handler);
}
}
}
}
}
}
public static Handler getHandler(String method, String path) {
Request request = new Request(method, path);
return REQUEST_MAP.get(request);
}
}
初始化容器说完了,我们再来看DispatcherServlet的service方法
@Override
public void service(Request request, Response response) {
String method = request.getMethod().name().toUpperCase();
String path = request.getPath();
Handler handler = ControllerHelper.getHandler(method, path);
if (handler != null) {
// 获取对象实例
Class<?> controllerClass = handler.getControllerClass();
Object controllerBean = BeanHelper.getBean(controllerClass);
Method actionMethod = handler.getControllerMethod();
Object result = ReflectionUtil.invokeMethod(controllerBean, actionMethod);
handleDataResult(new Data(result), response);
}
}
首先获取请求的method和请求路径。
之后通过ControllerHelper的getHandler方法取出对应的handler,我们刚刚分析完,ControllerHelper在静态代码块中将request和handler保存在REQUEST_MAP中,方便后面按照路径进行映射。所以我们现在得到的handler就是请求对应类和方法的封装。
之后我们通过BeanHelper的getBean方法,得到请求对应类的实例,这个在上面也讲了,不记得的同学往上面翻一翻。
有了对象实例和对应方法,我们就可以进行调用了。invokeMethod方法就是对invoke进行了简单的封装
public static Object invokeMethod(Object obj, Method method, Object... args) {
Object result = null;
try {
method.setAccessible(true);
result = method.invoke(obj, args);
} catch (Exception e) {
throw new RuntimeException(e);
}
return result;
}
最后,我们调用handleDataResult方法处理调用的结果。将json字符串保存在response中。
public void handleDataResult(Data data, Response response) {
Object model = data.getModel();
if (model != null) {
response.setContentType("application/json");
String json = JSON.toJSON(model).toString();
response.setBody(json.getBytes(StandardCharsets.UTF_8));
}
}
到此,DispatcherServlet的调用就结束了。大家可能比较疑惑,请求是如何返回的呢?
细心的同学可能发现了,在adapter的service方法中,调用DispatcherServlet后,就是返回请求的部分。
public void service(Request request, Response response) {
// 调用DispatcherServlet
new DispatcherServlet().service(request, response);
try {
ByteBuffer[] buffer = response.getResponseByteBuffer();
wrapper.getClient().write(buffer);
} catch (IOException e) {
e.printStackTrace();
}
}
这里我们只是简要分析了一些主要代码,剩下的代码大家可以去Github上面看。
由于本人能力有限,难免会出现问题,或者对某个问题没有采用更优的解决方案,希望大家多多提意见。
后面主要想从以下几个方面优化项目:
- 优化线程池的使用,为了开发效率,很多地方都是直接创建线程,创建销毁线程的成本很高
- 实现热加载
- 优化日志系统
- 优化性能