文章目录
目前网上大多数关于Zuul的相关讨论大都是
Spring-Cloud-Zuul
相关的,但是笔者认为Spring Cloud Zuul 有利也有弊。好处就是开发起来比较方便,一个注解就可以启用一个网关,并且增加了一些默认的过滤器组件;但是Spring Cloud 为了开发者的快速接入,也对Netflix Zuul 的一些功能做了阉割。
上图中可以看出,Netflix Zuul 主要包含了三大模块:Zuul Core模块
、过滤器加载模块
、过滤器管理模块
。其中核心模块和过滤器加载模块,在Spring Cloud Zuul 的定制版本中,都没有做太大的改动。
-
核心模块:解决了,请求接收然后通过
pre
、route
、post
三个主要类型的过滤器。当然了,还有另外2 中上图中没有展现,就是error
和custom
类型。error 是处理异常时的处理动作,custom 是自定义处理请求的过滤流程。上面各个过滤器之间是通过RequestContext
(Map类型,ThreadLocal 变量) 来做上下文传递的。ZuulFilterRunner 组织了各种类型的过滤器的执行逻辑。 -
过滤器加载模块:这个模块主要是给Core 模块的FilterRunner 提供服务的,也就是提供编译好的Filter组件。其中的FilterFileManager 是以轮询的方式,不断的从包含groovy 脚本的目录中加载文件,提供给 FilterLoader 编译和管理。
-
过滤器管理模块:这个模块是往往被我们忽略的模块,但是这个模块对于我们运维和做一些功能增强时,非常有帮助。大多数情况下,都需要对其做定制开发,从而适应各个公司不同的业务需求。
Netflix Zuul 官方的代码目录如下所示:
上面代码一共有4 个模块,下面分别介绍一下:
-
zuul-core:核心模块,对应上面架构图的 GateWay Core。
-
zuul-netflix:该模块依赖于 zuul-core 模块,包含了 过滤器加载模块和过滤器管理模块的逻辑(管理模块的配置和页面在下面项目)。
-
zuul-netflix-webapp:包含了 Zuul 网关以 Web 的方式启用的启动配置。
-
zuul-simple-webapp:官方提供的一个简单的页面Demo。
核心模块
ZuulServlet
ZuulServlet 是Netflix Zuul 的全局入口,在 zuul-netflix-webapp 的web.xml 中的配置如下:
<servlet>
<servlet-name>ZuulServlet</servlet-name>
<servlet-class>com.moguhu.zuul.http.ZuulServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>ZuulServlet</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
ZuulServlet 的主要逻辑如下:
@Override
public void service(javax.servlet.ServletRequest servletRequest, javax.servlet.ServletResponse servletResponse) throws ServletException, IOException {
try {
init((HttpServletRequest) servletRequest, (HttpServletResponse) servletResponse);
// Marks this request as having passed through the "Zuul engine", as opposed to servlets
// explicitly bound in web.xml, for which requests will not have the same data attached
RequestContext context = RequestContext.getCurrentContext();
context.setZuulEngineRan();
try {
preRoute();
} catch (ZuulException e) {
error(e);
postRoute();
return;
}
try {
route();
} catch (ZuulException e) {
error(e);
postRoute();
return;
}
try {
postRoute();
} catch (ZuulException e) {
error(e);
return;
}
} catch (Throwable e) {
error(new ZuulException(e, 500, "UNHANDLED_EXCEPTION_" + e.getClass().getName()));
} finally {
RequestContext.getCurrentContext().unset();
}
}
上面 service() 方法中可以看出,ZuulServlet 依次执行了 pre过滤器、route过滤器、post过滤器;当产生错误时,会执行error 过滤器。
以上各种类型的 Filter 会委派给ZuulRunner
去执行。
ZuulRunner
ZuulRunner 本身也没什么逻辑,而是使用了单例的处理器 FilterProcessor 来处理。
/**
* executes "post" filterType ZuulFilters
*/
public void postRoute() throws ZuulException {
FilterProcessor.getInstance().postRoute();
}
/**
* executes "route" filterType ZuulFilters
*/
public void route() throws ZuulException {
FilterProcessor.getInstance().route();
}
/**
* executes "pre" filterType ZuulFilters
*/
public void preRoute() throws ZuulException {
FilterProcessor.getInstance().preRoute();
}
/**
* executes "error" filterType ZuulFilters
*/
public void error() {
FilterProcessor.getInstance().error();
}
FilterProcessor
FilterProcessor 回去从 FilterLoader 中获取各个类型的过滤器,并且顺序执行。主要逻辑如下:
/**
* runs "post" filters which are called after "route" filters. ZuulExceptions from ZuulFilters are thrown.
* Any other Throwables are caught and a ZuulException is thrown out with a 500 status code
*/
public void postRoute() throws ZuulException {
try {
runFilters("post");
} catch (ZuulException e) {
throw e;
} catch (Throwable e) {
throw new ZuulException(e, 500, "UNCAUGHT_EXCEPTION_IN_POST_FILTER_" + e.getClass().getName());
}
}
/**
* runs all "error" filters. These are called only if an exception occurs. Exceptions from this are swallowed and logged so as not to bubble up.
*/
public void error() {
try {
runFilters("error");
} catch (Throwable e) {
logger.error(e.getMessage(), e);
}
}
/**
* Runs all "route" filters. These filters route calls to an origin.
*/
public void route() throws ZuulException {
try {
runFilters("route");
} catch (ZuulException e) {
throw e;
} catch (Throwable e) {
throw new ZuulException(e, 500, "UNCAUGHT_EXCEPTION_IN_ROUTE_FILTER_" + e.getClass().getName());
}
}
/**
* runs all "pre" filters. These filters are run before routing to the orgin.
*/
public void preRoute() throws ZuulException {
try {
runFilters("pre");
} catch (ZuulException e) {
throw e;
} catch (Throwable e) {
throw new ZuulException(e, 500, "UNCAUGHT_EXCEPTION_IN_PRE_FILTER_" + e.getClass().getName());
}
}
/**
* runs all filters of the filterType sType/ Use this method within filters to run custom filters by type
*/
public Object runFilters(String sType) throws Throwable {
if (RequestContext.getCurrentContext().debugRouting()) {
Debug.addRoutingDebug("Invoking {" + sType + "} type filters");
}
boolean bResult = false;
List<ZuulFilter> list = FilterLoader.getInstance().getFiltersByType(sType);
if (list != null) {
for (int i = 0; i < list.size(); i++) {
ZuulFilter zuulFilter = list.get(i);
Object result = processZuulFilter(zuulFilter);
if (result != null && result instanceof Boolean) {
bResult |= ((Boolean) result);
}
}
}
return bResult;
}
RequestContext
在各个 Filter 执行的过程中,参数的传递依赖是通过 线程本地变量 RequestContext 来实现的。
public class RequestContext extends ConcurrentHashMap<String, Object> {
private static final Logger LOG = LoggerFactory.getLogger(RequestContext.class);
protected static Class<? extends RequestContext> contextClass = RequestContext.class;
private static RequestContext testContext = null;
protected static final ThreadLocal<? extends RequestContext> threadLocal = new ThreadLocal<RequestContext>() {
@Override
protected RequestContext initialValue() {
try {
return contextClass.newInstance();
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
};
}
过滤器加载模块
过滤器加载模块主要提供给 Core 模块,从磁盘(或者网络等其他介质)中加载Filter ,并完成实例化的过程,以备Core 模块的使用。其中FilterFileManager 负责从磁盘中加载数据,然后调用FilterLoader 的单例,去实例化Filter以备用。
FilterFileManager
FilterFileManager 中会启用一个守护线程,在后台不断地循环管理Filter 文件(源码中每次循环后,会休息1秒钟)。
void startPoller() {
poller = new Thread("GroovyFilterFileManagerPoller") {
public void run() {
while (bRunning) {
try {
sleep(pollingIntervalSeconds * 1000);
manageFiles();
} catch (Exception e) {
e.printStackTrace();
}
}
}
};
poller.setDaemon(true);
poller.start();
}
void manageFiles() throws Exception {
List<File> aFiles = getFiles();
processGroovyFiles(aFiles);
}
/**
* Returns a List<File> of all Files from all polled directories
*/
List<File> getFiles() {
List<File> list = new ArrayList<>();
for (String sDirectory : aDirectories) {
if (sDirectory != null) {
File directory = getDirectory(sDirectory);
File[] aFiles = directory.listFiles(FILENAME_FILTER);
if (aFiles != null) {
list.addAll(Arrays.asList(aFiles));
}
}
}
return list;
}
/**
* puts files into the FilterLoader. The FilterLoader will only addd new or changed filters
*/
void processGroovyFiles(List<File> aFiles) throws Exception {
for (File file : aFiles) {
FilterLoader.getInstance().putFilter(file);
}
}
循环每个Filter 都会调用 FilterLoader.getInstance().putFilter() 将其放入FilterLoader的Map中。
FilterLoader
public class FilterLoader {
final static FilterLoader INSTANCE = new FilterLoader();
private final ConcurrentHashMap<String, Long> filterClassLastModified = new ConcurrentHashMap<String, Long>();
private final ConcurrentHashMap<String, String> filterClassCode = new ConcurrentHashMap<String, String>();
private final ConcurrentHashMap<String, String> filterCheck = new ConcurrentHashMap<String, String>();
private final ConcurrentHashMap<String, List<ZuulFilter>> hashFiltersByType = new ConcurrentHashMap<String, List<ZuulFilter>>();
private FilterRegistry filterRegistry = FilterRegistry.instance();
static DynamicCodeCompiler COMPILER;
static FilterFactory FILTER_FACTORY = new DefaultFilterFactory();
...
/**
* Given source and name will compile and store the filter if it detects that the filter code has changed or
* the filter doesn't exist. Otherwise it will return an instance of the requested ZuulFilter
*/
public ZuulFilter getFilter(String sCode, String sName) throws Exception {
if (filterCheck.get(sName) == null) {
filterCheck.putIfAbsent(sName, sName);
if (!sCode.equals(filterClassCode.get(sName))) {
LOG.info("reloading code " + sName);
filterRegistry.remove(sName);
}
}
ZuulFilter filter = filterRegistry.get(sName);
if (filter == null) {
Class clazz = COMPILER.compile(sCode, sName);
if (!Modifier.isAbstract(clazz.getModifiers())) {
filter = (ZuulFilter) FILTER_FACTORY.newInstance(clazz);
}
}
return filter;
}
/**
* From a file this will read the ZuulFilter source code, compile it, and add it to the list of current filters
* a true response means that it was successful.
*/
public boolean putFilter(File file) throws Exception {
String sName = file.getAbsolutePath() + file.getName();
if (filterClassLastModified.get(sName) != null && (file.lastModified() != filterClassLastModified.get(sName))) {
LOG.debug("reloading filter " + sName);
filterRegistry.remove(sName);
}
ZuulFilter filter = filterRegistry.get(sName);
if (filter == null) {
Class clazz = COMPILER.compile(file);
if (!Modifier.isAbstract(clazz.getModifiers())) {
filter = (ZuulFilter) FILTER_FACTORY.newInstance(clazz);
List<ZuulFilter> list = hashFiltersByType.get(filter.filterType());
if (list != null) {
hashFiltersByType.remove(filter.filterType()); //rebuild this list
}
filterRegistry.put(file.getAbsolutePath() + file.getName(), filter);
filterClassLastModified.put(sName, file.lastModified());
return true;
}
}
return false;
}
/**
* Returns a list of filters by the filterType specified
*/
public List<ZuulFilter> getFiltersByType(String filterType) {
List<ZuulFilter> list = hashFiltersByType.get(filterType);
if (list != null) return list;
list = new ArrayList<ZuulFilter>();
Collection<ZuulFilter> filters = filterRegistry.getAllFilters();
for (Iterator<ZuulFilter> iterator = filters.iterator(); iterator.hasNext(); ) {
ZuulFilter filter = iterator.next();
if (filter.filterType().equals(filterType)) {
list.add(filter);
}
}
Collections.sort(list); // sort by priority
hashFiltersByType.putIfAbsent(filterType, list);
return list;
}
}
过滤器管理模块
上面的过滤器加载模块最终交互的对象是磁盘文件,那么过滤器管理模块就相对是个独立的东西了。相关的操作就是Filter 文件。
过滤器管理模块直接操作文件的类是:ZuulFilterPoller 。它会起一个线程,不断的循环去抓取Filter的元数据(源码中由ZuulFilterDAO提供)。
ZuulFilterDAO
public interface ZuulFilterDao {
/**
* @return a list of all filterIds
*/
List<String> getAllFilterIds() throws Exception;
/**
* returns all filter revisions for the given filterId
*/
List<FilterInfo> getZuulFilters(String filterId) throws Exception;
/**
* returns a specific revision for a filter
*/
FilterInfo getFilter(String filterId, int revision) throws Exception;
...
}
官方给的默认的实现是 Cassandra,也就是Filter的元数据是从 Cassandra 数据库中获取的。但是这个数据库在国内用的相对较少,如果要定制的话,通常会改成 MySQL 或者 Http接口的方式去实现。
ZuulFilterPoller
ZuulFilterPoller 中起了一个循环线程,不断的去获取元数据的信息,从而更新Filter。代码如下:
private Thread checkerThread = new Thread("ZuulFilterPoller") {
@Override
public void run() {
while (running) {
try {
if (canary.get()) {
HashMap<String, ComponentDto> setFilters = new HashMap<>();
List<ComponentDto> activeScripts = ZuulFilterDAOFactory.getZuulFilterDao().getAllActiveFilters();
if (activeScripts != null) {
for (ComponentDto newComponent : activeScripts) {
setFilters.put(String.valueOf(newComponent.getCompId()), newComponent);
}
}
List<ComponentDto> canaryScripts = ZuulFilterDAOFactory.getZuulFilterDao().getAllCanaryFilters();
if (canaryScripts != null) {
for (ComponentDto newComponent : canaryScripts) {
setFilters.put(String.valueOf(newComponent.getCompId()), newComponent);
}
}
for (ComponentDto next : setFilters.values()) {
doCompCheck(next);
}
} else if (active.get()) {
List<ComponentDto> newComponents = ZuulFilterDAOFactory.getZuulFilterDao().getAllActiveFilters();
if (CollectionUtils.isEmpty(newComponents)) {
logger.warn("ZuulFilterPoller Warnning: There has NO active Component!");
return;
}
for (ComponentDto component : newComponents) {
doCompCheck(component);
}
}
} catch (Throwable e) {
logger.error("ZuulFilterPoller run error! {}", e);
}
try {
sleep(pollerInterval.get());
} catch (InterruptedException e) {
logger.error("ZuulFilterPoller sleep error! {}", e);
}
}
}
};
private static void doCompCheck(ComponentDto newComponent) throws IOException {
...
writeCompToDisk(newComponent);
}
private static void writeCompToDisk(ComponentDto newComponent) throws IOException {
...
}
定制开发的建议
对于Zuul 网关的使用,有2 大方式
:一种就是快速便捷的使用 Spring Cloud Zuul
的定制版本;另外一种就是基于Netflix Zuul 原生进行定制开发。Spring Cloud Zuul 的优点就是:开发方便,一个注解就可以开启一个网关节点;但是缺点也是很明显的,那就是不易于扩展,没有管理界面。那么相反的就是 原生的Netflix Zuul
的优缺点了。但是笔者认为,如果想要获取Zuul 的更多的定制特性,那么就需要基于原生的Zuul 进行定制开发。
通常情况下,需要对架构中的过滤器管理模块进行增强。可以考虑的功能点有:API 分组管理、API 管理、Filter 管理、服务管理等等。当然,生产中管理模块最好是一个独立的系统。Filter的管理通过网络去完成,所以由此看来工作量还是不小的。下面给出笔者的参考架构: