3.6、Eureka Server 源码分析
上一篇文章简单介绍了 Eureka 的一些概念，今天咱们来看看其原理和源码，首先先看 Eureka Server 的原理。
3.6.1、Eureka Server启动原理
本文章使用的 Eureka Server 依赖为:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
学习并使用过 Eureka Server 的朋友都知道，需要在启动类上加上 @EnableEurekaServer 注解，所以需要看Eureka Server 的启动原理的话，需要从 @EnableEurekaServer 注解开始。
/**
* Annotation to activate Eureka Server related configuration.
* {@link EurekaServerAutoConfiguration}
*
* @author Dave Syer
* @author Biju Kunjummen
*
*/
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Import(EurekaServerMarkerConfiguration.class)
public @interface EnableEurekaServer {}
将配置类 EurekaServerMarkerConfiguration 加入Spring 容器。
@Configuration(proxyBeanMethods = false)
public class EurekaServerMarkerConfiguration {
@Bean
public Marker eurekaServerMarkerBean() {
return new Marker();
}
class Marker {}
}
其实 EurekaServerMarkerConfiguration是个空类，没有任何实现，从 @EnableEurekaServer 的注释中可以看到 EurekaServerMarkerConfiguration 是个激活类，用于激活 EurekaServerAutoConfiguration 自动配置类 ，所以真正的配置信息都在 EurekaServerAutoConfiguration 中。
@Configuration(proxyBeanMethods = false)
@Import(EurekaServerInitializerConfiguration.class)
@ConditionalOnBean(EurekaServerMarkerConfiguration.Marker.class)
@EnableConfigurationProperties({ EurekaDashboardProperties.class,
InstanceRegistryProperties.class })
@PropertySource("classpath:/eureka/server.properties")
public class EurekaServerAutoConfiguration implements WebMvcConfigurer {
//other...
// 用于服务注册的核心Bean
@Bean
public PeerAwareInstanceRegistry peerAwareInstanceRegistry(
ServerCodecs serverCodecs) {
this.eurekaClient.getApplications(); // force initialization
return new InstanceRegistry(this.eurekaServerConfig, this.eurekaClientConfig,
serverCodecs, this.eurekaClient,
this.instanceRegistryProperties.getExpectedNumberOfClientsSendingRenews(),
this.instanceRegistryProperties.getDefaultOpenForTrafficCount());
}
// 用于管理集群服务节点的Bean
@Bean
@ConditionalOnMissingBean
public PeerEurekaNodes peerEurekaNodes(PeerAwareInstanceRegistry registry,
ServerCodecs serverCodecs,
ReplicationClientAdditionalFilters replicationClientAdditionalFilters) {
return new RefreshablePeerEurekaNodes(registry, this.eurekaServerConfig,
this.eurekaClientConfig, serverCodecs, this.applicationInfoManager,
replicationClientAdditionalFilters);
}
// Eureka Server上下文
@Bean
@ConditionalOnMissingBean
public EurekaServerContext eurekaServerContext(ServerCodecs serverCodecs,
PeerAwareInstanceRegistry registry, PeerEurekaNodes peerEurekaNodes) {
return new DefaultEurekaServerContext(this.eurekaServerConfig, serverCodecs,
registry, peerEurekaNodes, this.applicationInfoManager);
}
// Eureka Server的Bootstrap启动类
@Bean
public EurekaServerBootstrap eurekaServerBootstrap(PeerAwareInstanceRegistry registry,
EurekaServerContext serverContext) {
return new EurekaServerBootstrap(this.applicationInfoManager,
this.eurekaClientConfig, this.eurekaServerConfig, registry,
serverContext);
}
@Configuration(proxyBeanMethods = false)
protected static class EurekaServerConfigBeanConfiguration {
// 根据eureka.server相关配置类
@Bean
@ConditionalOnMissingBean
public EurekaServerConfig eurekaServerConfig(EurekaClientConfig clientConfig) {
EurekaServerConfigBean server = new EurekaServerConfigBean();
if (clientConfig.shouldRegisterWithEureka()) {
// Set a sensible default if we are supposed to replicate
server.setRegistrySyncRetries(5);
}
return server;
}
}
//other...
}
注意到在该类上有个 @Import 注解，用于导入其他的配置信息 EurekaServerInitializerConfiguration ，该类实现了 ServletContextAware 和 SmartLifecycle 两个接口，其中 ServletContextAware 用于获取 ServletContext 容器上下文，而 SmartLifecycle 接口是可以在Spring的生命周期中会调用相关的方法，所以在 EurekaServerInitializerConfiguration 中详细重写了start 方法：
//EurekaServerInitializerConfiguration#start
@Override
public void start() {
new Thread(() -> {
try {
// TODO: is this class even needed now?
//EurekaServerBootstrap初始化Context
eurekaServerBootstrap.contextInitialized(
EurekaServerInitializerConfiguration.this.servletContext);
log.info("Started Eureka Server");
//发布EurekaRegistryAvailableEvent事件
publish(new EurekaRegistryAvailableEvent(getEurekaServerConfig()));
EurekaServerInitializerConfiguration.this.running = true;
//发布EurekaServerStartedEvent事件
publish(new EurekaServerStartedEvent(getEurekaServerConfig()));
}
catch (Exception ex) {
// Help!
log.error("Could not initialize Eureka servlet context", ex);
}
}).start();
}
在start方法中新建了一个线程，用于初始化 EurekaServerBootstrap 并启动Eureka Server，同时又向外发布了 EurekaRegistryAvailableEvent 和 EurekaServerStartedEvent 事件，如果需要可以订阅该事件进行一些具体的处理操作，此处不做太多叙述。回到上面，查看 contextInitialized 方法：
// EurekaServerBootstrap#contextInitialized
public void contextInitialized(ServletContext context) {
try {
//初始化Eureka环境
initEurekaEnvironment();
//初始化Context上下文
initEurekaServerContext();
context.setAttribute(EurekaServerContext.class.getName(), this.serverContext);
}
catch (Throwable e) {
log.error("Cannot bootstrap eureka server :", e);
throw new RuntimeException("Cannot bootstrap eureka server :", e);
}
}
这个方法调用了initEurekaEnvironment()，初始化Eureka运行环境；调用了initEurekaServerContext()，初始化Eureka运行上下文。
//EurekaServerBootstrap#initEurekaEnvironment
protected void initEurekaEnvironment() throws Exception {
log.info("Setting the eureka configuration..");
// 设置数据中心
String dataCenter = ConfigurationManager.getConfigInstance()
.getString(EUREKA_DATACENTER);
if (dataCenter == null) {
log.info(
"Eureka data center value eureka.datacenter is not set, defaulting to default");
ConfigurationManager.getConfigInstance()
.setProperty(ARCHAIUS_DEPLOYMENT_DATACENTER, DEFAULT);
}
else {
ConfigurationManager.getConfigInstance()
.setProperty(ARCHAIUS_DEPLOYMENT_DATACENTER, dataCenter);
}
//设置 Eureka 环境
String environment = ConfigurationManager.getConfigInstance()
.getString(EUREKA_ENVIRONMENT);
if (environment == null) {
ConfigurationManager.getConfigInstance()
.setProperty(ARCHAIUS_DEPLOYMENT_ENVIRONMENT, TEST);
log.info(
"Eureka environment value eureka.environment is not set, defaulting to test");
}
else {
ConfigurationManager.getConfigInstance()
.setProperty(ARCHAIUS_DEPLOYMENT_ENVIRONMENT, environment);
}
}
//EurekaServerBootstrap#initEurekaServerContext
protected void initEurekaServerContext() throws Exception {
// For backward compatibility
JsonXStream.getInstance().registerConverter(new V1AwareInstanceInfoConverter(),
XStream.PRIORITY_VERY_HIGH);
XmlXStream.getInstance().registerConverter(new V1AwareInstanceInfoConverter(),
XStream.PRIORITY_VERY_HIGH);
if (isAws(this.applicationInfoManager.getInfo())) {
this.awsBinder = new AwsBinderDelegate(this.eurekaServerConfig,
this.eurekaClientConfig, this.registry, this.applicationInfoManager);
this.awsBinder.start();
}
//正式初始化
EurekaServerContextHolder.initialize(this.serverContext);
log.info("Initialized server context");
// Copy registry from neighboring eureka node
// 从邻近的Eureka节点复制注册表-即集群节点同步
int registryCount = this.registry.syncUp();
this.registry.openForTraffic(this.applicationInfoManager, registryCount);
// Register all monitoring statistics.
// 注册所有监控统计信息
EurekaMonitors.registerAllStats();
}
至此，Eureka Server启动完毕。
3.6.2、功能点
Eureka Server作为一个服务注册中心，则提供了如下几个功能，用于满足与Eureka Client的交互需求：
- 服务注册
- 服务心跳/续约
- 服务剔除
- 服务下线
- 集群同步
- 获取注册表中服务实例信息
上一小节介绍了 Eureka Server 的启动原理之后，其中注册了多个Bean会在这些功能点处起到关键性作用。
3.6.3、核心接口：InstanceRegistry
InstanceRegistry 是 Eureka Server 注册表的最核心接口，其职责是在内存中管理注册到 Eureka Server 中的服务实例信息。
public interface InstanceRegistry extends LeaseManager<InstanceInfo>, LookupService<String> {}
其中 LookupService 是提供对服务实例进行检索的最基本功能。
public interface LookupService<T> {
Application getApplication(String appName);
Applications getApplications();
List<InstanceInfo> getInstancesById(String id);
InstanceInfo getNextServerFromEureka(String virtualHostname, boolean secure);
}
而 LeaseManager 接口则是对注册到 Eureka Server 中的服务实例租约进行管理，分别是服务注册、服务下线、服务租约更新以及服务剔除，具体功能实现都由 AbstractInstanceRegistry 提供。
public interface LeaseManager<T> {
// 服务注册
void register(T r, int leaseDuration, boolean isReplication);
// 服务下线
boolean cancel(String appName, String id, boolean isReplication);
// 服务续约
boolean renew(String appName, String id, boolean isReplication);
// 服务剔除
void evict();
}
其余的两个功能：1）、服务集群同步由 PeerAwareInstanceRegistry 接口（实现类为 PeerAwareInstanceRegistryImpl） 类提供功能。
# PeerAwareInstanceRegistry.java
public int syncUp() {}//集群同步
2）、获取服务信息的功能则由 AbstractInstanceRegistry 抽象类实现。
# AbstractInstanceRegistry.java
// 全量式拉取注册表信息
public Applications getApplicationsFromMultipleRegions(String[] remoteRegions) {}
// 增量式拉取注册表信息
public Applications getApplicationDeltasFromMultipleRegions(String[] remoteRegions) {}
3.6.4、服务注册
Eureka Client 在发起服务注册时会将自身的服务实例元数据封装在 InstanceInfo 数据结构中，然后将 InstanceInfo 发送到 Eureka Server。Eureka Server 在接收到 Eureka Client 发送的 InstanceInfo 后将会尝试将其放到本地注册表中以供其他 Eureka Client 进行服务发现。
首先先来看服务注册的功能，它主要由PeerAwareInstanceRegistryImpl#register方法实现，代码如下：
/**
* Registers the information about the {@link InstanceInfo} and replicates
* this information to all peer eureka nodes. If this is replication event
* from other replica nodes then it is not replicated.
*
* @param info
* the {@link InstanceInfo} to be registered and replicated.
* @param isReplication
* true if this is a replication event from other replica nodes,
* false otherwise.
*/
// PeerAwareInstanceRegistryImpl#register
@Override
public void register(final InstanceInfo info, final boolean isReplication) {
int leaseDuration = Lease.DEFAULT_DURATION_IN_SECS;
if (info.getLeaseInfo() != null && info.getLeaseInfo().getDurationInSecs() > 0) {
leaseDuration = info.getLeaseInfo().getDurationInSecs();
}
//调用父类
super.register(info, leaseDuration, isReplication);
//通过replicateToPeers方法复制对应的行为到其他节点
replicateToPeers(Action.Register, info.getAppName(), info.getId(), info, null, isReplication);
}
其中replicateToPeers方法会贯穿整个集群服务节点发生变更的时候，后期会讲述，此处就不累赘。
在看父类 AbstractInstanceRegistry 的register方法之前，需要理清几个概念：
-
InstanceInfo 保存的是服务的基本信息，包括服务名，实例ID，IP地址等；
-
Lease 描述了泛型T的基于时间的属性，包括服务注册时间、服务启动时间、服务最后更新时间、服务下线时间、周期，而且还定义了三种行为，分别为Register, Cancel, Renew。其实说到底就是对泛型T的一种补充，内部会持有泛型T的引用。
public class Lease<T> { enum Action { Register, Cancel, Renew }; public static final int DEFAULT_DURATION_IN_SECS = 90; private T holder;//泛型T的引用 private long evictionTimestamp; private long registrationTimestamp; private long serviceUpTimestamp; // Make it volatile so that the expiration task would see this quicker private volatile long lastUpdateTimestamp; private long duration; //other... }
回到主题，查看AbstractInstanceRegistry的register方法：
// AbstractInstanceRegistry#register
public void register(InstanceInfo registrant, int leaseDuration, boolean isReplication) {
try {
read.lock();
//首先根据appName获取服务实例对象集群，
Map<String, Lease<InstanceInfo>> gMap = registry.get(registrant.getAppName());
REGISTER.increment(isReplication);
if (gMap == null) {
//如果为Null, 则新创建一个Map，并把当前的注册应用程序信息添加到此Map当中
final ConcurrentHashMap<String, Lease<InstanceInfo>> gNewMap = new ConcurrentHashMap<String, Lease<InstanceInfo>>();
gMap = registry.putIfAbsent(registrant.getAppName(), gNewMap);
if (gMap == null) {
gMap = gNewMap;
}
}
Lease<InstanceInfo> existingLease = gMap.get(registrant.getId());
// Retain the last dirty timestamp without overwriting it, if there is already a lease
//根据当前存在节点的触碰时间和注册节点的触碰时间比较
if (existingLease != null && (existingLease.getHolder() != null)) {
Long existingLastDirtyTimestamp = existingLease.getHolder().getLastDirtyTimestamp();
Long registrationLastDirtyTimestamp = registrant.getLastDirtyTimestamp();
logger.debug("Existing lease found (existing={}, provided={}", existingLastDirtyTimestamp, registrationLastDirtyTimestamp);
// this is a > instead of a >= because if the timestamps are equal, we still take the remote transmitted
// InstanceInfo instead of the server local copy.
//如果前者的时间晚于后者的时间，那么当前注册的实例就以已存在的实例为准
if (existingLastDirtyTimestamp > registrationLastDirtyTimestamp) {
logger.warn("There is an existing lease and the existing lease's dirty timestamp {} is greater" +
" than the one that is being registered {}", existingLastDirtyTimestamp, registrationLastDirtyTimestamp);
logger.warn("Using the existing instanceInfo instead of the new instanceInfo as the registrant");
registrant = existingLease.getHolder();
}
} else {
// The lease does not exist and hence it is a new registration
synchronized (lock) {
//更新其每分钟期望的续约数量及其阈值
if (this.expectedNumberOfClientsSendingRenews > 0) {
// Since the client wants to register it, increase the number of clients sending renews
this.expectedNumberOfClientsSendingRenews = this.expectedNumberOfClientsSendingRenews + 1;
updateRenewsPerMinThreshold();
}
}
logger.debug("No previous lease information found; it is new registration");
}
//创建新的租约信息
Lease<InstanceInfo> lease = new Lease<InstanceInfo>(registrant, leaseDuration);
if (existingLease != null) {
//如果租约存在，则继承已存在的租约服务上线时间
lease.setServiceUpTimestamp(existingLease.getServiceUpTimestamp());
}
//将当前的注册节点的租约信息保存到map当中
gMap.put(registrant.getId(), lease);
//保存到最近注册的队列中recentRegisteredQueue
recentRegisteredQueue.add(new Pair<Long, String>(
System.currentTimeMillis(),
registrant.getAppName() + "(" + registrant.getId() + ")"));
// This is where the initial state transfer of overridden status happens
if (!InstanceStatus.UNKNOWN.equals(registrant.getOverriddenStatus())) {
logger.debug("Found overridden status {} for instance {}. Checking to see if needs to be add to the "
+ "overrides", registrant.getOverriddenStatus(), registrant.getId());
if (!overriddenInstanceStatusMap.containsKey(registrant.getId())) {
logger.info("Not found overridden id {} and hence adding it", registrant.getId());
overriddenInstanceStatusMap.put(registrant.getId(), registrant.getOverriddenStatus());
}
}
InstanceStatus overriddenStatusFromMap = overriddenInstanceStatusMap.get(registrant.getId());
if (overriddenStatusFromMap != null) {
logger.info("Storing overridden status {} from map", overriddenStatusFromMap);
registrant.setOverriddenStatus(overriddenStatusFromMap);
}
// Set the status based on the overridden status rules
// 根据覆盖状态规则InstanceStatusOverrideRule得到服务实例的最终状态，并设置服务实例的当前状态
InstanceStatus overriddenInstanceStatus = getOverriddenInstanceStatus(registrant, existingLease, isReplication);
registrant.setStatusWithoutDirty(overriddenInstanceStatus);
// If the lease is registered with UP status, set lease service up timestamp
// 如果服务的状态为UP，则设置租约的服务上线时间为当前时间；
if (InstanceStatus.UP.equals(registrant.getStatus())) {
lease.serviceUp();
}
//添加最近租约变更记录队列，标识ActionType为ADDED；
registrant.setActionType(ActionType.ADDED);
//将当前Lease对象包装成状态变更对象，并添加到recentlyChangedQueue，用于增量式获取注册表信息
recentlyChangedQueue.add(new RecentlyChangedItem(lease));
//设置服务实例信息更新时间
registrant.setLastUpdatedTimestamp();
//失效readWriteCacheMap缓存，用于全量式获取注册表信息
invalidateCache(registrant.getAppName(), registrant.getVIPAddress(), registrant.getSecureVipAddress());
logger.info("Registered instance {}/{} with status {} (replication={})",
registrant.getAppName(), registrant.getId(), registrant.getStatus(), isReplication);
} finally {
read.unlock();
}
}
简单梳理一下服务注册的过程：
- 这里的
registry是new ConcurrentHashMap<String, Map<String, Lease<InstanceInfo>>>()对象，它会根据appName对服务实例进行分类，同时又根据 InstanceInfo 的服务实例instanceId（可以通过eureka.instance.instance-id进行配置）进行映射到Lease 对象； - 首先根据appName获取服务实例对象集群，如果为Null, 则新创建一个Map，并把当前的注册应用程序信息添加到此Map当中；
- 根据当前注册的ID，如果能在map中取到则做以下操作：
- 根据当前存在节点的触碰时间和注册节点的触碰时间比较，如果前者的时间晚于后者的时间，那么当前注册的实例就以已存在的实例为准；
- 否则创建一个新的租约，并更新其每分钟期望的续约数量及其阈值；
- 将当前的注册节点的租约信息保存到map当中；
- 并保存到最近注册的队列中
recentRegisteredQueue； - 根据覆盖状态规则 InstanceStatusOverrideRule 得到服务实例的最终状态，并设置服务实例的当前状态；
- 如果服务的状态为
UP，则设置租约的服务上线时间为当前时间； - 添加最近租约变更记录队列，标识
ActionType为ADDED； - 将当前Lease对象包装成状态变更对象，并添加到
recentlyChangedQueue，用于Eureka Client 进行增量式获取注册表信息； - 每次注册的时候，失效 Eureka Client 的
LoadingCache<Key, Value> readWriteCacheMap缓存，这将会导致 Eureka Client 在同步的时候进行全量式获取注册信息表。
3.6.5、服务心跳
在 Eureka Client 完成服务注册之后，需要定时向 Eureka Server 发送心跳请求（默认30s一次），维持自己在 Eureka Server 中租约的有效性。
接受服务心跳的功能，它主要由PeerAwareInstanceRegistryImpl#renew方法实现，代码如下：
//PeerAwareInstanceRegistryImpl#renew
public boolean renew(final String appName, final String id, final boolean isReplication) {
if (super.renew(appName, id, isReplication)) {
replicateToPeers(Action.Heartbeat, appName, id, null, null, isReplication);
return true;
}
return false;
}
同样看父类 AbstractInstanceRegistry 的renew方法：
/**
* Marks the given instance of the given app name as renewed, and also marks whether it originated from
* replication.
*
* @see com.netflix.eureka.lease.LeaseManager#renew(java.lang.String, java.lang.String, boolean)
*/
//AbstractInstanceRegistry#renew
public boolean renew(String appName, String id, boolean isReplication) {
RENEW.increment(isReplication);
//根据appName获取服务集群中的租约集合
Map<String, Lease<InstanceInfo>> gMap = registry.get(appName);
Lease<InstanceInfo> leaseToRenew = null;
if (gMap != null) {
leaseToRenew = gMap.get(id);
}
//如果租约不存在，则直接返回false
if (leaseToRenew == null) {
RENEW_NOT_FOUND.increment(isReplication);
logger.warn("DS: Registry: lease doesn't exist, registering resource: {} - {}", appName, id);
return false;
} else {
InstanceInfo instanceInfo = leaseToRenew.getHolder();
if (instanceInfo != null) {
// touchASGCache(instanceInfo.getASGName());
//如果租约存在，则同样根据覆盖状态规则InstanceStatusOverrideRule得到服务实例的最终状态；
InstanceStatus overriddenInstanceStatus = this.getOverriddenInstanceStatus(
instanceInfo, leaseToRenew, isReplication);
//如果得到的服务实例最后状态为UNKNOWN，则取消续约
if (overriddenInstanceStatus == InstanceStatus.UNKNOWN) {
logger.info("Instance status UNKNOWN possibly due to deleted override for instance {}"
+ "; re-register required", instanceInfo.getId());
RENEW_NOT_FOUND.increment(isReplication);
return false;
}
//如果租约服务的状态与最终状态不一致，则将当前租约服务的状态更新为由覆盖状态规则获取的最终状态
if (!instanceInfo.getStatus().equals(overriddenInstanceStatus)) {
logger.info(
"The instance status {} is different from overridden instance status {} for instance {}. "
+ "Hence setting the status to overridden status", instanceInfo.getStatus().name(),
instanceInfo.getOverriddenStatus().name(),
instanceInfo.getId());
instanceInfo.setStatusWithoutDirty(overriddenInstanceStatus);
}
}
renewsLastMin.increment();
//更新租约中的有效时间
leaseToRenew.renew();
return true;
}
}
服务心跳流程：
- 根据appName获取服务集群中的租约集合；
- 如果租约不存在，则直接返回false；
- 如果租约存在，则同样根据覆盖状态规则 InstanceStatusOverrideRule 得到服务实例的最终状态；
- 如果得到的服务实例最后状态为
UNKNOWN，则取消续约； - 如果租约服务的状态与最终状态不一致，则将当前租约服务的状态更新为由覆盖状态规则获取的最终状态；
- 更新租约中的有效时间。
3.6.6、服务剔除
如果 Eureka Client 在注册后，既没有续约，也没有下线（服务崩溃或者网络异常等原因造成服务不稳定），那么服务的状态就处于不可知的状态，不能保证能够从服务实例中获取到回馈，所以需要服务剔除定时AbstractlnstanceRegistry 的 evict清理那些不稳定的服务，该方法会批量将注册表中所有过期租约剔除，代码如下：
//AbstractInstanceRegistry#evict
public void evict() {
evict(0l);
}
//AbstractInstanceRegistry#evict
public void evict(long additionalLeaseMs) {
logger.debug("Running the evict task");
// 判断Eureka Server是否处于自我保护模式下，如果处于自我保护模式，则直接返回
if (!isLeaseExpirationEnabled()) {
logger.debug("DS: lease expiration is currently disabled.");
return;
}
// We collect first all expired items, to evict them in random order. For large eviction sets,
// if we do not that, we might wipe out whole apps before self preservation kicks in. By randomizing it,
// the impact should be evenly distributed across all applications.
//服务剔除过程将会遍历registry注册表，找出其中所有的过期租约，添加到需要剔除的集合中；
List<Lease<InstanceInfo>> expiredLeases = new ArrayList<>();
for (Entry<String, Map<String, Lease<InstanceInfo>>> groupEntry : registry.entrySet()) {
Map<String, Lease<InstanceInfo>> leaseMap = groupEntry.getValue();
if (leaseMap != null) {
for (Entry<String, Lease<InstanceInfo>> leaseEntry : leaseMap.entrySet()) {
Lease<InstanceInfo> lease = leaseEntry.getValue();
if (lease.isExpired(additionalLeaseMs) && lease.getHolder() != null) {
expiredLeases.add(lease);
}
}
}
}
// To compensate for GC pauses or drifting local time, we need to use current registry size as a base for
// triggering self-preservation. Without that we would wipe out full registry.
//根据配置文件中的租约百分比阈值和当前注册表中的租约总量计算初最大允许的剔除租约数量（当前注册表中租约总数量减去当前注册表租约阈值）
int registrySize = (int) getLocalRegistrySize();
int registrySizeThreshold = (int) (registrySize * serverConfig.getRenewalPercentThreshold());
//获取允许剔除租约的最大数量
int evictionLimit = registrySize - registrySizeThreshold;
//取允许剔除的最大数量和当前需要剔除租约的集合大小中的小值
int toEvict = Math.min(expiredLeases.size(), evictionLimit);
if (toEvict > 0) {
logger.info("Evicting {} items (expired={}, evictionLimit={})", toEvict, expiredLeases.size(), evictionLimit);
//随机剔除过期的服务实例租约信息
Random random = new Random(System.currentTimeMillis());
for (int i = 0; i < toEvict; i++) {
// Pick a random item (Knuth shuffle algorithm)
int next = i + random.nextInt(expiredLeases.size() - i);
Collections.swap(expiredLeases, i, next);
Lease<InstanceInfo> lease = expiredLeases.get(i);
String appName = lease.getHolder().getAppName();
String id = lease.getHolder().getId();
EXPIRED.increment();
logger.warn("DS: Registry: expired lease for {}/{}", appName, id);
//调用internalCancel服务下线的方法将其从注册表中清理除去
internalCancel(appName, id, false);
}
}
}
服务剔除流程:
- 判断Eureka Server是否处于自我保护模式下，如果处于自我保护模式，则直接返回；
- 服务剔除过程将会遍历registry注册表，找出其中所有的过期租约，添加到需要剔除的集合中；
- 根据配置文件中的租约百分比阈值（默认为0.85，可以通过
eureka.server.renewal-percent-threshold=0.85进行配置）和当前注册表中的租约总量计算出最大允许的剔除租约数量，即当前注册表中租约总数量减去当前注册表租约阈值； - 取允许剔除的最大数量和当前需要剔除租约的集合大小中的小数值；
- 根据数量按照
Knuth shuffle algorithm算法随机剔除租约过期的服务信息，关于Knuth洗牌算法可以查看博客Knuth洗牌算法； - 最后调用
internalCancel服务下线的方法将其从注册表中清理除去（internalCancel方法可以详见下一章节）。
其实在服务剔除过程中，由很多限制，都是为了保证 Eureka Server 的可用性：
- 处于自我保护模式时期不能进行服务剔除操作；
- 过期服务的剔除操作是分批进行的；
- 服务剔除是随机逐个删除，剔除均匀分布在所有需要剔除的应用中，防止在同一时刻内同一服务集群中的过期服务全部被删除，以致大量发生服务剔除时，在未自我保护前提下促使程序崩溃或者不可用。
服务剔除的操作是一个定时任务，由 AbstractInstanceRegistry 中的内部类 EvictionTask 用于定时执行服务剔除，默认为60秒（可以通过eureka.server.eviction-interval-timer-in-ms进行配置）。服务剔除的示例可以在后面的章节中阅读。
3.6.7、服务下线
Eureka Server 在应用销毁时，会向 Eureka Server 发送服务下线请求时，清除注册表中关于本应用的租约，避免无效的服务调用。在服务剔除的过程中，也是通过服务下线的逻辑完成对单个服务实例过期租约的清除工作。
服务下线的主要实现代码位于 AbstractInstanceRegistry的cancel方法中，仅需要服务实例的服务名和服务实例id即可完成服务下线：
//AbstractInstanceRegistry#cancel
public boolean cancel(String appName, String id, boolean isReplication) {
return internalCancel(appName, id, isReplication);
}
/**
* {@link #cancel(String, String, boolean)} method is overridden by {@link PeerAwareInstanceRegistry}, so each
* cancel request is replicated to the peers. This is however not desired for expires which would be counted
* in the remote peers as valid cancellations, so self preservation mode would not kick-in.
*/
//AbstractInstanceRegistry#internalCancel
protected boolean internalCancel(String appName, String id, boolean isReplication) {
try {
read.lock();
CANCEL.increment(isReplication);
//通过registry根据服务名和服务实例id查询关于服务实例的租约Lease是否存在；
Map<String, Lease<InstanceInfo>> gMap = registry.get(appName);
Lease<InstanceInfo> leaseToCancel = null;
if (gMap != null) {
leaseToCancel = gMap.remove(id);
}
//将其加入到最近下线的服务队列中
recentCanceledQueue.add(new Pair<Long, String>(System.currentTimeMillis(), appName + "(" + id + ")"));
InstanceStatus instanceStatus = overriddenInstanceStatusMap.remove(id);
if (instanceStatus != null) {
logger.debug("Removed instance id {} from the overridden map which has value {}", id, instanceStatus.name());
}
//如果租约不存在，返回下线失败false
if (leaseToCancel == null) {
CANCEL_NOT_FOUND.increment(isReplication);
logger.warn("DS: Registry: cancel failed because Lease is not registered for: {}/{}", appName, id);
return false;
} else {
//如果租约存在，则从registry注册表中移除，并设置服务下线时间
leaseToCancel.cancel();
InstanceInfo instanceInfo = leaseToCancel.getHolder();
String vip = null;
String svip = null;
if (instanceInfo != null) {
//标识服务行为为DELETED
instanceInfo.setActionType(ActionType.DELETED);
//在最近租约变更记录队列中添加新的下线记录，以用于Eureka Client增量式获取注册表
recentlyChangedQueue.add(new RecentlyChangedItem(leaseToCancel));
instanceInfo.setLastUpdatedTimestamp();
vip = instanceInfo.getVIPAddress();
svip = instanceInfo.getSecureVipAddress();
}
invalidateCache(appName, vip, svip);
logger.info("Cancelled instance {}/{} (replication={})", appName, id, isReplication);
}
} finally {
read.unlock();
}
synchronized (lock) {
if (this.expectedNumberOfClientsSendingRenews > 0) {
// Since the client wants to cancel it, reduce the number of clients to send renews.
this.expectedNumberOfClientsSendingRenews = this.expectedNumberOfClientsSendingRenews - 1;
updateRenewsPerMinThreshold();
}
}
return true;
}
服务下线流程与服务注册类似：
- 通过registry根据服务名和服务实例id查询关于服务实例的租约Lease是否存在；
- 将需要下线的服务加入到最近下线的服务队列中；
- 如果租约不存在，则返回false，表示下线失败；
- 如果租约存在，则从registry注册表中移除，并设置服务下线时间；
- 标识服务行为为
DELETED； - 在最近租约变更记录队列中添加新的下线记录，以用于 Eureka Client 增量式获取注册表；
- 最后失效 Eureka Client 的
LoadingCache<Key, Value> readWriteCacheMap缓存，这将会导致 Eureka Client 在同步的时候进行全量式获取注册信息表。
3.6.8、集群同步
如果 Eureka Server 是通过集群的方式进行部署，那么为了维护整个集群中的 Eureka Server 注册表中数据的一致性，则需要一个机制同步 Eureka Server 集群中的注册表数据。
Eureka Server 集群同步包含两个部分，一部分是Eureka Server 在启动过程中从它的peer节点中拉取注册表信息，并将这些服务实例的信息注册到本地注册表中；另一部分是 Eureka Server 每次对本地注册表进行操作时，同时会将操作同步到它的peer节点中，达到集群注册表数据的一致性目的。
3.6.8.1、Eureka Server初始化本地注册表信息
之前在Eureka Server启动原理章节中，EurekaServerBootstrap 的contextInitialized方法中的initEurekaServerContext方法最后有以下一行代码：
int registryCount = this.registry.syncUp();
之前注释中也解释了其功能，即为集群同步，咱们来看具体的实现 PeerAwareInstanceRegistryImpl的syncUp方法：
//PeerAwareInstanceRegistryImpl#syncUp
public int syncUp() {
// Copy entire entry from neighboring DS node
int count = 0;
for (int i = 0; ((i < serverConfig.getRegistrySyncRetries()) && (count == 0)); i++) {
if (i > 0) {
try {
Thread.sleep(serverConfig.getRegistrySyncRetryWaitMs());
} catch (InterruptedException e) {
logger.warn("Interrupted during registry transfer..");
break;
}
}
Applications apps = eurekaClient.getApplications();
for (Application app : apps.getRegisteredApplications()) {
for (InstanceInfo instance : app.getInstances()) {
try {
if (isRegisterable(instance)) {
register(instance, instance.getLeaseInfo().getDurationInSecs(), true);
count++;
}
} catch (Throwable t) {
logger.error("During DS init copy", t);
}
}
}
}
return count;
}
集群同步的流程如下：
- 首先根据配置文件中的
eureka.server.registry-sync-retries的配置属性获取注册表同步重试次数； - 线程等待
eureka.server.registry-sync-retry-wait-ms时间； - Eureka Server 在集群中也是一个 Eureka Client ，所以在启动的时候也会进行 DiscoveryClient 的初始化，会从其对应的 Eureka Server 中拉取全量的注册表信息。在 Eureka Server 集群部署的情况下，Eureka Server 从它的peer节点中拉取到注册表信息；
- 获取后，将遍历整个 Applications ，将所有的服务实例通过 AbstractInstanceRegistry的
registry方法注册到自身注册表中。
在syncUp同步方法完成之后，还有一行代码：
registry.openForTraffic(applicationInfoManager, registryCount);
在syncUp初始化本地注册表时，Eureka Server 并不会接受来自 Eureka Client 的通信请求（如服务注册、或者获取注册表信息等请求）。在同步注册表信息结束后会通过 PeerAwarelnstanceRegistrylmpl 的openForTraffic方法允许该Server接受流量，代码如下：
//PeerAwareInstanceRegistryImpl#openForTraffic
public void openForTraffic(ApplicationInfoManager applicationInfoManager, int count) {
// Renewals happen every 30 seconds and for a minute it should be a factor of 2.
this.expectedNumberOfClientsSendingRenews = count;
updateRenewsPerMinThreshold();
logger.info("Got {} instances from neighboring DS node", count);
logger.info("Renew threshold is: {}", numberOfRenewsPerMinThreshold);
this.startupTime = System.currentTimeMillis();
//如果同步的应用实例数量为0 ，将在一段时间内拒绝Client获取注册信息
if (count > 0) {
this.peerInstancesTransferEmptyOnStartup = false;
}
DataCenterInfo.Name selfName = applicationInfoManager.getInfo().getDataCenterInfo().getName();
boolean isAws = Name.Amazon == selfName;
if (isAws && serverConfig.shouldPrimeAwsReplicaConnections()) {
logger.info("Priming AWS connections for all replicas..");
primeAwsReplicas(applicationInfoManager);
}
logger.info("Changing status to UP");
//修改服务实例的状态为健康上线UP，并可以接受流量
applicationInfoManager.setInstanceStatus(InstanceStatus.UP);
super.postInit();
}
在 Eureka Server 中有个 StatusFilter 过滤器，用于检查 Eureka Server 的状态，当 Eureka Server 的状态不为UP的时候，将拒绝所有的请求。
public class StatusFilter implements Filter {
private static final int SC_TEMPORARY_REDIRECT = 307;
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
InstanceInfo myInfo = ApplicationInfoManager.getInstance().getInfo();
InstanceStatus status = myInfo.getStatus();
//判断当前服务的健康状态，如果不为UP，则直接通过response返回307状态码
if (status != InstanceStatus.UP && response instanceof HttpServletResponse) {
HttpServletResponse httpRespone = (HttpServletResponse) response;
httpRespone.sendError(SC_TEMPORARY_REDIRECT,
"Current node is currently not ready to serve requests -- current status: "
+ status + " - try another DS node: ");
}
//放行
chain.doFilter(request, response);
}
//other...
}
在判断当前服务的健康状态不为UP的时候，会直接通过 Response 返回307状态码，否则可以接收流量。
//PeerAwareInstanceRegistryImpl#shouldAllowAccess
public boolean shouldAllowAccess(boolean remoteRegionRequired) {
if (this.peerInstancesTransferEmptyOnStartup) {
if (!(System.currentTimeMillis() > this.startupTime + serverConfig.getWaitTimeInMsWhenSyncEmpty())) {
return false;
}
}
if (remoteRegionRequired) {
for (RemoteRegionRegistry remoteRegionRegistry : this.regionNameVSRemoteRegistry.values()) {
if (!remoteRegionRegistry.isReadyForServingData()) {
return false;
}
}
}
return true;
}
//PeerAwareInstanceRegistryImpl#shouldAllowAccess
public boolean shouldAllowAccess() {
return shouldAllowAccess(true);
}
在 Eureka Client 请求获取注册表信息时，Eureka Server 会判断此时是否允许获取注册表信息。上述做法是为了避免 Eureka Server 在syncUp方法中没有获取到任何服务实例信息，Eureka Server 注册表中的信息影响到 Eureka Client 缓存的注册表中的信息。如果 Eureka Server 在syncUp方法中没有获取到任何实例信息，它将把peerInstancesTransferEmptyOnStartup设置为true，这时该 Eureka Server 在WaitTimeInMsWhenSyncEmpty（可以通过eureka.server.wait-time-in-ms-when-sync-empty进行设置，默认为5分钟）时间后才能被 Eureka Client 访问获取注册表信息。
3.6.8.2、Eureka Server 节点间注册表信息的同步复制
为了保证 Eureka Server 集群运行时注册表信息的一致性，每个 Eureka Server 在对本地注册表进行管理操作时，会将相应的操作同步到集群中的所有peer节点。
在PeerAwareInstanceRegistryImpl 的register、cancel、renew等方法都添加了同步到peer节点的操作，使 Eureka Server 集群中的注册表信息保持最终一致性。如下：
//PeerAwareInstanceRegistryImpl#register
public void register(final InstanceInfo info, final boolean isReplication) {
int leaseDuration = Lease.DEFAULT_DURATION_IN_SECS;
if (info.getLeaseInfo() != null && info.getLeaseInfo().getDurationInSecs() > 0) {
leaseDuration = info.getLeaseInfo().getDurationInSecs();
}
super.register(info, leaseDuration, isReplication);
replicateToPeers(Action.Register, info.getAppName(), info.getId(), info, null, isReplication);
}
//PeerAwareInstanceRegistryImpl#cancel
public boolean cancel(final String appName, final String id, final boolean isReplication) {
if (super.cancel(appName, id, isReplication)) {
replicateToPeers(Action.Cancel, appName, id, null, null, isReplication);
return true;
}
return false;
}
//PeerAwareInstanceRegistryImpl#renew
public boolean renew(final String appName, final String id, final boolean isReplication) {
if (super.renew(appName, id, isReplication)) {
replicateToPeers(Action.Heartbeat, appName, id, null, null, isReplication);
return true;
}
return false;
}
所以可以关注replicateToPeers方法，它将遍历 Eureka Server 中的peer节点，向每个peer节点发送同步请求。代码如下：
//PeerAwareInstanceRegistryImpl#replicateToPeers
private void replicateToPeers(Action action, String appName, String id,
InstanceInfo info /* optional */,
InstanceStatus newStatus /* optional */, boolean isReplication) {
Stopwatch tracer = action.getTimer().start();
try {
if (isReplication) {
numberOfReplicationsLastMin.increment();
}
// If it is a replication already, do not replicate again as this will create a poison replication
//如果peer集群为空，或者本来就是复制操作，则直接返回，不再复制
if (peerEurekaNodes == Collections.EMPTY_LIST || isReplication) {
return;
}
// 向peer集群中的每一个peer进行同步
for (final PeerEurekaNode node : peerEurekaNodes.getPeerEurekaNodes()) {
// If the url represents this host, do not replicate to yourself.
// 如果peer节点是自身的话，不进行复制
if (peerEurekaNodes.isThisMyUrl(node.getServiceUrl())) {
continue;
}
//根据Action调用不同的同步请求
replicateInstanceActionsToPeers(action, appName, id, info, newStatus, node);
}
} finally {
tracer.stop();
}
}
PeerEurekaNode 代表一个可同步共享数据的 Eureka Server 。在 PeerEurekaNode 中，具有 register、 cancel、heartbeat、statusUpdate 等诸多用于向 peer 点同步注册表信息的操作。
在replicatelnstanceActionsToPeers方法中将根据 Action 的不同，调用 PeerEurekaNode的不同方法进行同步复制，代码如下所示：
//PeerAwareInstanceRegistryImpl#replicateInstanceActionsToPeers
private void replicateInstanceActionsToPeers(Action action, String appName,
String id, InstanceInfo info, InstanceStatus newStatus,
PeerEurekaNode node) {
try {
InstanceInfo infoFromRegistry;
CurrentRequestVersion.set(Version.V2);
switch (action) {
case Cancel://同步下线
node.cancel(appName, id);
break;
case Heartbeat://同步心跳
InstanceStatus overriddenStatus = overriddenInstanceStatusMap.get(id);
infoFromRegistry = getInstanceByAppAndId(appName, id, false);
node.heartbeat(appName, id, infoFromRegistry, overriddenStatus, false);
break;
case Register://同步注册
node.register(info);
break;
case StatusUpdate://同步状态更新
infoFromRegistry = getInstanceByAppAndId(appName, id, false);
node.statusUpdate(appName, id, newStatus, infoFromRegistry);
break;
case DeleteStatusOverride://删除覆盖状态
infoFromRegistry = getInstanceByAppAndId(appName, id, false);
node.deleteStatusOverride(appName, id, infoFromRegistry);
break;
}
} catch (Throwable t) {
logger.error("Cannot replicate information to {} for action {}", node.getServiceUrl(), action.name(), t);
} finally {
CurrentRequestVersion.remove();
}
}
仔细查看 PeerEurekaNode 的cancel、heartbeat、register、statusUpdate、deleteStatusOverride方法中，都是启动了一个 InstanceReplicationTask 任务，通过 HttpReplicationClient的同名方法进行相关同步操作。
3.6.9、获取注册表中服务实例信息
Eureka Server 获取注册表的服务实例信息主要通过两个方法实现 AbstractlnstanceRegistry的getApplicationsFromMultipleRegions从多地区获取全量注册表数据， AbstractlnstanceRegistry getApplicationDeltasFromMultipleRegions 从多地区获取增量式注册表数据。
3.6.9.1、全量式获取注册表信息
//AbstractInstanceRegistry#getApplicationsFromMultipleRegions
public Applications getApplicationsFromMultipleRegions(String[] remoteRegions) {
boolean includeRemoteRegion = null != remoteRegions && remoteRegions.length != 0;
logger.debug("Fetching applications registry with remote regions: {}, Regions argument {}",
includeRemoteRegion, remoteRegions);
if (includeRemoteRegion) {
GET_ALL_WITH_REMOTE_REGIONS_CACHE_MISS.increment();
} else {
GET_ALL_CACHE_MISS.increment();
}
Applications apps = new Applications();
apps.setVersion(1L);
for (Entry<String, Map<String, Lease<InstanceInfo>>> entry : registry.entrySet()) {
Application app = null;
if (entry.getValue() != null) {
for (Entry<String, Lease<InstanceInfo>> stringLeaseEntry : entry.getValue().entrySet()) {
Lease<InstanceInfo> lease = stringLeaseEntry.getValue();
if (app == null) {
app = new Application(lease.getHolder().getAppName());
}
app.addInstance(decorateInstanceInfo(lease));
}
}
if (app != null) {
apps.addApplication(app);
}
}
if (includeRemoteRegion) {
for (String remoteRegion : remoteRegions) {
RemoteRegionRegistry remoteRegistry = regionNameVSRemoteRegistry.get(remoteRegion);
if (null != remoteRegistry) {
Applications remoteApps = remoteRegistry.getApplications();
for (Application application : remoteApps.getRegisteredApplications()) {
if (shouldFetchFromRemoteRegistry(application.getName(), remoteRegion)) {
logger.info("Application {} fetched from the remote region {}",
application.getName(), remoteRegion);
Application appInstanceTillNow = apps.getRegisteredApplications(application.getName());
if (appInstanceTillNow == null) {
appInstanceTillNow = new Application(application.getName());
apps.addApplication(appInstanceTillNow);
}
for (InstanceInfo instanceInfo : application.getInstances()) {
appInstanceTillNow.addInstance(instanceInfo);
}
} else {
logger.debug("Application {} not fetched from the remote region {} as there exists a "
+ "whitelist and this app is not in the whitelist.",
application.getName(), remoteRegion);
}
}
} else {
logger.warn("No remote registry available for the remote region {}", remoteRegion);
}
}
}
apps.setAppsHashCode(apps.getReconcileHashCode());
return apps;
}
3.6.9.2、增量式获取注册表信息
AbstractInstanceRegistry#getApplicationDeltasFromMultipleRegions
public Applications getApplicationDeltasFromMultipleRegions(String[] remoteRegions) {
if (null == remoteRegions) {
remoteRegions = allKnownRemoteRegions; // null means all remote regions.
}
boolean includeRemoteRegion = remoteRegions.length != 0;
if (includeRemoteRegion) {
GET_ALL_WITH_REMOTE_REGIONS_CACHE_MISS_DELTA.increment();
} else {
GET_ALL_CACHE_MISS_DELTA.increment();
}
Applications apps = new Applications();
apps.setVersion(responseCache.getVersionDeltaWithRegions().get());
Map<String, Application> applicationInstancesMap = new HashMap<String, Application>();
try {
write.lock();
Iterator<RecentlyChangedItem> iter = this.recentlyChangedQueue.iterator();
logger.debug("The number of elements in the delta queue is :{}", this.recentlyChangedQueue.size());
while (iter.hasNext()) {
Lease<InstanceInfo> lease = iter.next().getLeaseInfo();
InstanceInfo instanceInfo = lease.getHolder();
logger.debug("The instance id {} is found with status {} and actiontype {}",
instanceInfo.getId(), instanceInfo.getStatus().name(), instanceInfo.getActionType().name());
Application app = applicationInstancesMap.get(instanceInfo.getAppName());
if (app == null) {
app = new Application(instanceInfo.getAppName());
applicationInstancesMap.put(instanceInfo.getAppName(), app);
apps.addApplication(app);
}
app.addInstance(new InstanceInfo(decorateInstanceInfo(lease)));
}
if (includeRemoteRegion) {
for (String remoteRegion : remoteRegions) {
RemoteRegionRegistry remoteRegistry = regionNameVSRemoteRegistry.get(remoteRegion);
if (null != remoteRegistry) {
Applications remoteAppsDelta = remoteRegistry.getApplicationDeltas();
if (null != remoteAppsDelta) {
for (Application application : remoteAppsDelta.getRegisteredApplications()) {
if (shouldFetchFromRemoteRegistry(application.getName(), remoteRegion)) {
Application appInstanceTillNow =
apps.getRegisteredApplications(application.getName());
if (appInstanceTillNow == null) {
appInstanceTillNow = new Application(application.getName());
apps.addApplication(appInstanceTillNow);
}
for (InstanceInfo instanceInfo : application.getInstances()) {
appInstanceTillNow.addInstance(new InstanceInfo(instanceInfo));
}
}
}
}
}
}
}
Applications allApps = getApplicationsFromMultipleRegions(remoteRegions);
apps.setAppsHashCode(allApps.getReconcileHashCode());
return apps;
} finally {
write.unlock();
}
}
获取增量式注册表信息将会从recentlyChangedQueue中获取最近发生变化的服务实例信息。recentlyChangedQueue中统计了最近时间段内进行注册、修改和剔除的服务实例信息，在服务注册 AbstractlnstanceRegistry 的registry、 接受心跳请求 AbstractlnstanceRegistry 的renew、和服务下线 AbstractlnstanceRegistry 的internalCancel 方法中均可见到recentlyChangedQueue对这些服务实例进行登记，用于记录增量式注册表信息。
4274
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
