一.cxf-dosgi暴露服务和在zookeeper上注册的过程说明:
1.有2个地方可以调用TopologyManager protected void exportService(ServiceReference sref) :
a.TopologyManager 初始化
b.org.apache.cxf.dosgi.topologymanager.ServiceListenerImpl 有服务事件时。
public void serviceChanged(ServiceEvent event)
2.调用TopologyManager triggerExport(sref);
for (final RemoteServiceAdmin remoteServiceAdmin : remoteServiceAdminList) {
LOG
.info("TopologyManager: handling remoteServiceAdmin "
+ remoteServiceAdmin);
if (exports.containsKey(remoteServiceAdmin)) {
// already handled by this remoteServiceAdmin
LOG.info("TopologyManager: already handled by this remoteServiceAdmin -> skipping");
} else {
// TODO: additional parameter Map ?
LOG.info("TopologyManager: exporting ...");
Collection<ExportRegistration> endpoints = remoteServiceAdmin
.exportService(sref, null);
if (endpoints == null) {
// TODO export failed -> What should be done here?
LOG.info("TopologyManager: export failed");
exports.put(remoteServiceAdmin, null);
} else {
LOG.info("TopologyManager: export sucessful Endpoints:" + endpoints);
// enqueue in local list of endpoints
exports.put(remoteServiceAdmin, endpoints);
// publish to endpoint listeners
nofifyListeners(endpoints);
}
}
}
调用所有的RemoteServiceAdmin来暴露服务remoteServiceAdmin.exportService(sref, null);
并通知linstenser有节点变化nofifyListeners(endpoints),要连zookeeper做节点操作。
3. 调用RemoteServiceAdminInstance 暴露服务
public List /* ExportRegistration */exportService(ServiceReference ref, Map properties)
throws IllegalArgumentException, UnsupportedOperationException {
....
synchronized (exportedServices) {
List er = rsaCore.exportService(refFinal, propertiesFinal);
if (er != null)
exportedServices.addAll(er);
return er;
}
......
}
4.调用RemoteServiceAdminCore暴露服务
根据暴露服务的类型,来选择typehandler来创建web service服务
public List exportService(ServiceReference serviceReference, Map additionalProperties)
throws IllegalArgumentException, UnsupportedOperationException {
...
ExportRegistrationImpl exportRegistration = exportRegs.get(iface);
ConfigurationTypeHandler handler = getHandler(configurationTypes, serviceProperties,
getHandlerProperties());
Object serviceObject = bctx.getService(serviceReference);
BundleContext callingContext = serviceReference.getBundle().getBundleContext();
if (handler == null) {
// TODO: publish error event ? not sure
return Collections.EMPTY_LIST;
}
LOG.info("found handler for " + iface + " -> " + handler);
String interfaceName = iface;
// this is an extra sanity check, but do we really need it now ?
Class<?> interfaceClass = ClassUtils.getInterfaceClass(serviceObject, interfaceName);
if (interfaceClass != null) {
handler.createServer(exportRegistration, bctx, callingContext, serviceProperties,
interfaceClass, serviceObject);
if(exportRegistration.getException()==null){
LOG.info("created server for interface " + iface);
exportRegistration.startServiceTracker(bctx);
}else{
LOG.warning("server creation for interface " + iface + " failed!");
// Fire event happens at the end
}
}
5.后面的步骤和 Apache CXF暴露服务-学习笔记 描述的类似
6.步骤2提到nofifyListeners(endpoints);会连zookeeper,继续看看 TopologyManager
protected void nofifyListeners(Collection<ExportRegistration> exportRegistrations) {
try {
// Find all EndpointListeners; They must have the Scope property otherwise they have to be ignored
ServiceReference[] refs = Utils.getEndpointListeners(bctx);
if (refs != null) {
for (ServiceReference sref : refs) {
notifyListenerOfAddingIfAppropriate(sref, exportRegistrations);
}
}
} catch (InvalidSyntaxException e) {
e.printStackTrace();
}
7.TopologyManager 获取ServiceListener来处理
protected void notifyListenerOfAddingIfAppropriate(ServiceReference sref,
Collection<ExportRegistration> exportRegistrations) {
// if (sref.getBundle().equals(bctx.getBundle())) {
// LOG
// .info("TopologyManager: notifyListenerOfAddingIfAppropriate() called for own listener -> skipping ");
// return;
// }
EndpointListener epl = (EndpointListener)bctx.getService(sref);
LOG.info("TopologyManager: notifyListenerOfAddingIfAppropriate() ");
try {
List<Filter> filters = Utils.normalizeScope(sref, bctx);
for (ExportRegistration exReg : exportRegistrations) {
// FIXME!!!!!!!!!!!!! There needs to be a better way ?!?!?!
Map props = exReg.getExportReference().getExportedEndpoint().getProperties();
Dictionary d = new Hashtable(props);
for (Filter filter : filters) {
LOG.info("Matching: " + filter + " against " + d);
}
for (Filter filter : filters) {
if (filter.match(d)) {
LOG.info("Listener mached one of the Endpoints !!!!: " + epl);
epl
.endpointAdded(exReg.getExportReference().getExportedEndpoint(), filter
.toString());
}
}
}
} catch (InvalidSyntaxException e) {
e.printStackTrace();
}
}
8.org.apache.cxf.dosgi.discovery.zookeeper.EndpointListenerImpl
连zookeeper创建节点,到此服务暴露完毕。
public void endpointAdded(EndpointDescription endpoint, String matchedFilter) {
LOG.info("endpointDescription added: " + endpoint);
if (closed)
return;
synchronized (endpoints) {
if (endpoints.contains(endpoint)) {
// TODO -> Should the published endpoint be updated here ?
return;
}
try {
Collection<String> interfaces = endpoint.getInterfaces();
String endpointKey = getKey(endpoint.getId());
ZooKeeper zk = getZooKeeper();
for (String name : interfaces) {
String path = Util.getZooKeeperPath(name);
String fullPath = path + '/' + endpointKey;
LOG.info("Creating ZooKeeper node: " + fullPath);
ensurePath(path, zk);
zk.create(fullPath, getData(endpoint), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
}
endpoints.add(endpoint);
} catch (Exception ex) {
LOG.log(Level.SEVERE, "Exception while processing the addition of a ServicePublication.", ex);
}
}
}
二、暴露服务删除
1.在Bundle stop后,org.apache.cxf.dosgi.topologymanager.ServiceListenerImpl收到ServiceEvent
public void serviceChanged(ServiceEvent event) {
LOG.fine("Received ServiceEvent: " + event);
ServiceReference sref = event.getServiceReference();
if (event.getType() == ServiceEvent.REGISTERED) {
LOG.fine("Registered");
if (analyzeService(sref)) {
LOG.info("calling TopologyManager -> registered service");
topManager.exportService(sref);
}
} else if (event.getType() == ServiceEvent.UNREGISTERING) {
topManager.removeService(sref);
}
}
2.TopologyManager删除服务
void removeService(ServiceReference sref) {
synchronized (exportedServices) {
if (exportedServices.containsKey(sref)) {
Map<RemoteServiceAdmin, Collection<ExportRegistration>> rsas = exportedServices.get(sref);
for (Map.Entry<RemoteServiceAdmin, Collection<ExportRegistration>> entry : rsas.entrySet()) {
if (entry.getValue() != null) {
for (ExportRegistration exReg : entry.getValue()) {
if (exReg != null)
exReg.close();
}
}
}
exportedServices.remove(sref);
}
}
}
3.org.apache.cxf.dosgi.dsw.service.ExportRegistrationImpl调用RemoteServiceAdminCore删除服务注册
public synchronized void close() {
if (closed)
return;
closed = true;
rsaCore.removeExportRegistration(this);
parent.instanceClosed();
}
三、Zookeeper节点删除
- Bundle 状态变化引起
public void bundleChanged(BundleEvent be) {
switch (be.getType()) {
case BundleEvent.STARTED:
findDeclaredRemoteServices(be.getBundle());
break;
case BundleEvent.STOPPED:
removeServicesDeclaredInBundle(be.getBundle());
break;
}
}
private void removeServicesDeclaredInBundle(Bundle bundle) {
for (Iterator<Entry<EndpointDescription, Bundle>> i = endpointDescriptions.entrySet().iterator(); i.hasNext(); ) {
Entry<EndpointDescription, Bundle> entry = i.next();
if (bundle.equals(entry.getValue())) {
removedEndpointDescription(entry.getKey());
i.remove();
}
}
}
private void removedEndpointDescription(EndpointDescription ed) {
triggerCallbacks(ed, false);
}
private void triggerCallbacks(EndpointDescription ed, boolean added) {
for (Map.Entry<EndpointListener, Collection<String>> entry : listenerToFilters.entrySet()) {
for (String match : entry.getValue()) {
triggerCallbacks(entry.getKey(), match, ed, added);
}
}
}
private void triggerCallbacks(EndpointListener listener, String toMatch,
EndpointDescription ed, boolean added) {
if (!filterMatches(toMatch, ed)) {
return;
}
if (added) {
listener.endpointAdded(ed, toMatch);
} else {
listener.endpointRemoved(ed, toMatch);
}
}
2.org.apache.cxf.dosgi.discovery.zookeeper.EndpointListenerImpl收到通知删除endpoint节点
public void endpointRemoved(EndpointDescription endpoint, String matchedFilter) {
LOG.info("endpointDescription removed: " + endpoint);
if (closed)
return;
synchronized (endpoints) {
if (!endpoints.contains(endpoint)) {
return;
}
try {
removeEndpoint(endpoint);
endpoints.remove(endpoint);
} catch (Exception ex) {
LOG.log(Level.SEVERE, "Exception while processing the removal of a ServicePublication.", ex);
}
}
}
private void removeEndpoint(EndpointDescription endpoint) throws UnknownHostException,
URISyntaxException, InterruptedException, KeeperException {
Collection<String> interfaces = endpoint.getInterfaces();
String endpointKey = getKey(endpoint.getId());
ZooKeeper zk = getZooKeeper();
for (String name : interfaces) {
String path = Util.getZooKeeperPath(name);
String fullPath = path + '/' + endpointKey;
LOG.fine("Removing ZooKeeper node: " + fullPath);
zk.delete(fullPath, -1);
}
}
- Service 变化引起
stRemoteServiceAdmin = new ServiceTracker(bctx, RemoteServiceAdmin.class.getName(), null) {
@Override
public Object addingService(ServiceReference reference) {
LOG.info("Adding RemoteServiceAdmin to list of admins ");
RemoteServiceAdmin rsa = (RemoteServiceAdmin)bctx.getService(reference);
synchronized (rsal) {
rsal.add(rsa);
}
LOG.info("enlisted RemoteEventAdmins: " + this.size());
triggerExportImportForRemoteSericeAdmin(rsa);
return super.addingService(reference);
}
@Override
public void removedService(ServiceReference reference, Object service) {
LOG.info("TopologyManager: Removing RemoteServiceAdmin from list of admins ");
synchronized (rsal) {
rsal.remove(service);
}
// TODO: remove service exports from management structure and notify discovery stuff...
removeRemoteServiceAdmin((RemoteServiceAdmin)service);
LOG.info("TopologyManager: enlisted RemoteEventAdmins: " + rsal.size());
super.removedService(reference, service);
}
};
protected void removeRemoteServiceAdmin(RemoteServiceAdmin service) {
topManager.removeRemoteServiceAdmin(service);
}
2.调用TopologyManager删除RemoteServiceAdmin
protected void removeRemoteServiceAdmin(RemoteServiceAdmin rsa) {
synchronized (exportedServices) {
for (Map.Entry<ServiceReference, HashMap<RemoteServiceAdmin, Collection<ExportRegistration>>> exports : exportedServices
.entrySet()) {
if (exports.getValue().containsKey(rsa)) {
// service was handled by this RemoteServiceAdmin
Collection<ExportRegistration> endpoints = exports.getValue().get(rsa);
// TODO for each notify discovery......
try {
ServiceReference[] refs = Utils.getEndpointListeners(bctx);
for (ServiceReference sref : refs) {
notifyListenersOfRemovalIfAppropriate(sref, endpoints);
}
} catch (InvalidSyntaxException e) {
e.printStackTrace();
}
// remove all management information for the RemoteServiceAdmin
exports.getValue().remove(rsa);
}
}
}
}
protected void notifyListenersOfRemovalIfAppropriate(ServiceReference sref,
Collection<ExportRegistration> exportRegistrations) {
EndpointListener epl = (EndpointListener)bctx.getService(sref);
LOG.info("TopologyManager: notifyListenerOfREMOVALIfAppropriate() ");
List<Filter> filters;
try {
filters = Utils.normalizeScope(sref, bctx);
for (ExportRegistration exReg : exportRegistrations) {
// FIXME!!!!!!!!!!!!! There needs to be a better way ?!?!?!
Map props = exReg.getExportReference().getExportedEndpoint().getProperties();
Dictionary d = new Hashtable(props);
for (Filter filter : filters) {
LOG.info("Matching: " + filter + " against " + d);
}
for (Filter filter : filters) {
if (filter.match(d)) {
LOG.info("Listener matched one of the Endpoints !!!! --> calling removed() ...");
epl.endpointRemoved(exReg.getExportReference().getExportedEndpoint(), filter
.toString());
}
}
}
} catch (InvalidSyntaxException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
3.org.apache.cxf.dosgi.discovery.zookeeper.EndpointListenerImpl
public void endpointRemoved(EndpointDescription endpoint, String matchedFilter) {
LOG.info("endpointDescription removed: " + endpoint);
if (closed)
return;
synchronized (endpoints) {
if (!endpoints.contains(endpoint)) {
return;
}
try {
removeEndpoint(endpoint);
endpoints.remove(endpoint);
} catch (Exception ex) {
LOG.log(Level.SEVERE, "Exception while processing the removal of a ServicePublication.", ex);
}
}
}
- 直接调用Stop
调用ZooKeeperDiscovery的stop方法,也会EndpointListenerImpl的close方法,进而调用endpointRemoved方法。这里不详细描述了。
服务的暴露:
在TopologyManager初始化时,会做服务暴露初始化,
另外在Service变化时,注册ServiceLisentener来获取服务的变化通知,通过调用RemoteServiceAdmin来启动http server和注册Web Service;通过EndpointListenerImpl来获取EndPoint的变化通知,注册到zookeeper上。
删除在bundle ,service changed时触发。
附:zookeeper上的日志信息
连接:
2011-10-17 17:34:41,539 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:62107
2011-10-17 17:34:41,555 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@777] - Client attempting to establish new session at /127.0.0.1:62107
2011-10-17 17:34:41,602 - INFO [SyncThread:0:NIOServerCnxn@1580] - Established
session 0x133113b5f2e0000 with negotiated timeout 4000 for client /127.0.0.1:62107
中断:
2011-10-17 17:35:45,704 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x133113b5f2e0000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2011-10-17 17:35:45,704 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:62107 which had
sessionid 0x133113b5f2e0000
2011-10-17 17:35:50,011 - INFO [SessionTracker:ZooKeeperServer@314] - Expiring
session 0x133113b5f2e0000, timeout of 4000ms exceeded
2011-10-17 17:35:50,011 - INFO [ProcessThread:-1:PrepRequestProcessor@387] - Pr
ocessed session termination for sessionid: 0x133113b5f2e0000