title: Spring加载资源文件冲突问题 tags:
- tomcat8
- spring
- mac
- webjars
- j2cache categories: spring date: 2017-11-14 16:42:08
背景
开发者在使用Spring做Properties文件载入时，这也是spring做placeholder的便捷之处。
classpath:和classpath*:想必也是大部分开发都会写到的文件schema。
百度一下classpath能检索到一大堆区别比如
classpath 和 classpath* 区别：
classpath：只会到你指定的class路径中查找文件;
classpath*：不仅包含class路径，还包括jar文件中(class路径)进行查找.
OK 本次说一些不同的
源码
spring在使用时如下会这样配置比如
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath*:httpclient.properties</value>
<value>classpath*:config.properties</value>
<value>classpath*:uploadPath.properties</value>
<value>classpath*:branches.properties</value>
<value>classpath*:wechat.properties</value>
<value>classpath*:email.properties</value>
<value>classpath:j2cache.properties</value>
<value>classpath:sso.properties</value>
<value>classpath*:override.properties</value>
</list>
</property>
</bean>
复制代码我们来看一下事实上该方法接受的参数是Resource[] 但是我们传递的是String Spring做了什么魔法呢？
我们简明概要的看一下关键的源码
spring在默认情况下帮我们做了一系列的Convert。我们
此处我们需要关心的是ResourceArray
/**
* Treat the given text as a location pattern and convert it to a Resource array.
*/
@Override
public void setAsText(String text) {
String pattern = resolvePath(text).trim();
try {
setValue(this.resourcePatternResolver.getResources(pattern));
}
catch (IOException ex) {
throw new IllegalArgumentException(
"Could not resolve resource location pattern [" + pattern + "]: " + ex.getMessage());
}
}
复制代码看到了关键代码resourcePatternResolver.getResources
public Resource[] getResources(String locationPattern) throws IOException {
Assert.notNull(locationPattern, "Location pattern must not be null");
if (locationPattern.startsWith(CLASSPATH_ALL_URL_PREFIX)) {
// a class path resource (multiple resources for same name possible)
if (getPathMatcher().isPattern(locationPattern.substring(CLASSPATH_ALL_URL_PREFIX.length()))) {
// a class path resource pattern
return findPathMatchingResources(locationPattern);
}
else {
// all class path resources with the given name
return findAllClassPathResources(locationPattern.substring(CLASSPATH_ALL_URL_PREFIX.length()));
}
}
else {
// Only look for a pattern after a prefix here
// (to not get fooled by a pattern symbol in a strange prefix).
int prefixEnd = locationPattern.indexOf(":") + 1;
if (getPathMatcher().isPattern(locationPattern.substring(prefixEnd))) {
// a file pattern
return findPathMatchingResources(locationPattern);
}
else {
// a single resource with the given name
return new Resource[] {getResourceLoader().getResource(locationPattern)};
}
}
}
复制代码很明显可以看到对于classpath*做了特殊处理
那么对于classpath:打头的呢 事实上到最后都将走到DefaultResourceLoader中getResource方法
public Resource getResource(String location) {
Assert.notNull(location, "Location must not be null");
if (location.startsWith(CLASSPATH_URL_PREFIX)) {
return new ClassPathResource(location.substring(CLASSPATH_URL_PREFIX.length()), getClassLoader());
}
else {
try {
// Try to parse the location as a URL...
URL url = new URL(location);
return new UrlResource(url);
}
catch (MalformedURLException ex) {
// No URL -> resolve as resource path.
return getResourceByPath(location);
}
}
}
复制代码对于classpath:打头的资源文件将返回classpathResource 否则返回UrlResource
对于classpath*的文件会优先做一次匹配如下
/**
* Find all class location resources with the given location via the ClassLoader.
* @param location the absolute path within the classpath
* @return the result as Resource array
* @throws IOException in case of I/O errors
* @see java.lang.ClassLoader#getResources
* @see #convertClassLoaderURL
*/
protected Resource[] findAllClassPathResources(String location) throws IOException {
String path = location;
if (path.startsWith("/")) {
path = path.substring(1);
}
Enumeration<URL> resourceUrls = getClassLoader().getResources(path);
Set<Resource> result = new LinkedHashSet<Resource>(16);
while (resourceUrls.hasMoreElements()) {
URL url = resourceUrls.nextElement();
result.add(convertClassLoaderURL(url));
}
return result.toArray(new Resource[result.size()]);
}
/**
* Convert the given URL as returned from the ClassLoader into a Resource object.
* <p>The default implementation simply creates a UrlResource instance.
* @param url a URL as returned from the ClassLoader
* @return the corresponding Resource object
* @see java.lang.ClassLoader#getResources
* @see org.springframework.core.io.Resource
*/
protected Resource convertClassLoaderURL(URL url) {
return new UrlResource(url);
}
复制代码也就是返回UrlResource【从上述代码可以见到集合为LinkedHashSet】
回到classpath上来 一个classpathResource文件如何找到对应的资源呢？
一般来说都是调用classLoader的getReource方法
/**
* Finds the resource with the given name. A resource is some data
* (images, audio, text, etc) that can be accessed by class code in a way
* that is independent of the location of the code.
*
* <p> The name of a resource is a '<tt>/</tt>'-separated path name that
* identifies the resource.
*
* <p> This method will first search the parent class loader for the
* resource; if the parent is <tt>null</tt> the path of the class loader
* built-in to the virtual machine is searched. That failing, this method
* will invoke {@link #findResource(String)} to find the resource. </p>
*
* @param name
* The resource name
*
* @return A <tt>URL</tt> object for reading the resource, or
* <tt>null</tt> if the resource could not be found or the invoker
* doesn't have adequate privileges to get the resource.
*
* @since 1.1
*/
public URL getResource(String name) {
URL url;
if (parent != null) {
url = parent.getResource(name);
} else {
url = getBootstrapResource(name);
}
if (url == null) {
url = findResource(name);
}
return url;
}
复制代码这是一个典型的委托模型的结构 从最下面的classLoader一直回溯到BootstrapLoader 逐渐寻找
我们可以认为在tomcat容器中一个配置文件通常需要由WebappClassLoader进行加载
而在tomcat7.0.70中代码如下【我们忽略需要向parent获取资源文件的过程】
当不同jar中包含同一个名称的资源文件时那么会做何种操作呢？
我们先看一下如何加载jar的顺序的。
在tomcat7.0.70中tomcat的classloader会作如下操作
NamingEnumeration<NameClassPair> enumeration = null;
try {
enumeration = libDir.list("");
} catch (NamingException e) {
IOException ioe = new IOException(sm.getString(
"webappLoader.namingFailure", libPath));
ioe.initCause(e);
throw ioe;
}
复制代码也就是以libDir的list的结果加入对应jar
/**
* Enumerates the names bound in the named context, along with the class
* names of objects bound to them.
*
* @param name the name of the context to list
* @return an enumeration of the names and class names of the bindings in
* this context. Each element of the enumeration is of type NameClassPair.
* @exception NamingException if a naming exception is encountered
*/
@Override
public NamingEnumeration<NameClassPair> list(String name)
throws NamingException {
if (!aliases.isEmpty()) {
AliasResult result = findAlias(name);
if (result.dirContext != null) {
return result.dirContext.list(result.aliasName);
}
}
// Next do a standard lookup
List<NamingEntry> bindings = doListBindings(name);
// Check the alternate locations
List<NamingEntry> altBindings = null;
String resourceName = "/META-INF/resources" + name;
for (DirContext altDirContext : altDirContexts) {
if (altDirContext instanceof BaseDirContext) {
altBindings = ((BaseDirContext) altDirContext).doListBindings(resourceName);
}
if (altBindings != null) {
if (bindings == null) {
bindings = altBindings;
} else {
bindings.addAll(altBindings);
}
}
}
if (bindings != null) {
return new NamingContextEnumeration(bindings.iterator());
}
// Really not found
throw new NameNotFoundException(
sm.getString("resources.notFound", name));
}
复制代码META-INF/resource多么熟悉的文件夹，这也是为啥出现了WebJar这类的jar的原因 通过META-INF/resource的映射可以在对应地方找到资源文件
这样我们也可以找到WebJar的代码
继续查看如下
/**
* Enumerates the names bound in the named context, along with the
* objects bound to them. The contents of any subcontexts are not
* included.
* <p>
* If a binding is added to or removed from this context, its effect on
* an enumeration previously returned is undefined.
*
* @param name the name of the context to list
* @return an enumeration of the bindings in this context.
* Each element of the enumeration is of type Binding.
* @exception NamingException if a naming exception is encountered
*/
@Override
protected List<NamingEntry> doListBindings(String name)
throws NamingException {
File file = file(name);
if (file == null)
return null;
return list(file);
}
/**
* Return a File object representing the specified normalized
* context-relative path if it exists and is readable. Otherwise,
* return <code>null</code>.
*
* @param name Normalized context-relative path (with leading '/')
*/
protected File file(String name) {
File file = new File(base, name);
if (file.exists() && file.canRead()) {
if (allowLinking)
return file;
// Check that this file belongs to our root path
String canPath = null;
try {
canPath = file.getCanonicalPath();
} catch (IOException e) {
// Ignore
}
if (canPath == null)
return null;
// Check to see if going outside of the web application root
if (!canPath.startsWith(absoluteBase)) {
return null;
}
// Case sensitivity check - this is now always done
String fileAbsPath = file.getAbsolutePath();
if (fileAbsPath.endsWith("."))
fileAbsPath = fileAbsPath + "/";
String absPath = normalize(fileAbsPath);
canPath = normalize(canPath);
if ((absoluteBase.length() < absPath.length())
&& (absoluteBase.length() < canPath.length())) {
absPath = absPath.substring(absoluteBase.length() + 1);
if (absPath.equals(""))
absPath = "/";
canPath = canPath.substring(absoluteBase.length() + 1);
if (canPath.equals(""))
canPath = "/";
if (!canPath.equals(absPath))
return null;
}
} else {
return null;
}
return file;
}
/**
* List the resources which are members of a collection.
*
* @param file Collection
* @return Vector containing NamingEntry objects
*/
protected List<NamingEntry> list(File file) {
List<NamingEntry> entries = new ArrayList<NamingEntry>();
if (!file.isDirectory())
return entries;
String[] names = file.list();
if (names==null) {
/* Some IO error occurred such as bad file permissions.
Prevent a NPE with Arrays.sort(names) */
log.warn(sm.getString("fileResources.listingNull",
file.getAbsolutePath()));
return entries;
}
Arrays.sort(names); // Sort alphabetically
NamingEntry entry = null;
for (int i = 0; i < names.length; i++) {
File currentFile = new File(file, names[i]);
Object object = null;
if (currentFile.isDirectory()) {
FileDirContext tempContext = new FileDirContext(env);
tempContext.setDocBase(currentFile.getPath());
tempContext.setAllowLinking(getAllowLinking());
object = tempContext;
} else {
object = new FileResource(currentFile);
}
entry = new NamingEntry(names[i], object, NamingEntry.ENTRY);
entries.add(entry);
}
return entries;
}
复制代码一个Arrays.sort(names); 可以看出下面返回的结果将是按照字典序进行排列的。
那么对应的在tomcat中调用classLoader如下
/**
* Find the specified resource in our local repository, and return a
* <code>URL</code> referring to it, or <code>null</code> if this resource
* cannot be found.
*
* @param name Name of the resource to be found
*/
@Override
public URL findResource(final String name) {
if (log.isDebugEnabled())
log.debug(" findResource(" + name + ")");
URL url = null;
String path = nameToPath(name);
if (hasExternalRepositories && searchExternalFirst)
url = super.findResource(name);
if (url == null) {
ResourceEntry entry = resourceEntries.get(path);
if (entry == null) {
if (securityManager != null) {
PrivilegedAction<ResourceEntry> dp =
new PrivilegedFindResourceByName(name, path, false);
entry = AccessController.doPrivileged(dp);
} else {
entry = findResourceInternal(name, path, false);
}
}
if (entry != null) {
url = entry.source;
}
}
if ((url == null) && hasExternalRepositories && !searchExternalFirst)
url = super.findResource(name);
if (log.isDebugEnabled()) {
if (url != null)
log.debug(" --> Returning '" + url.toString() + "'");
else
log.debug(" --> Resource not found, returning null");
}
return (url);
}
复制代码而在对应的findResourceInternal方法
/**
* Find specified resource in local repositories.
*
* @return the loaded resource, or null if the resource isn't found
*/
protected ResourceEntry findResourceInternal(final String name, final String path,
final boolean manifestRequired) {
if (!started) {
log.info(sm.getString("webappClassLoader.stopped", name));
return null;
}
if ((name == null) || (path == null))
return null;
JarEntry jarEntry = null;
// Need to skip the leading / to find resoucres in JARs
String jarEntryPath = path.substring(1);
ResourceEntry entry = resourceEntries.get(path);
if (entry != null) {
if (manifestRequired && entry.manifest == MANIFEST_UNKNOWN) {
// This resource was added to the cache when a request was made
// for the resource that did not need the manifest. Now the
// manifest is required, the cache entry needs to be updated.
synchronized (jarFiles) {
if (openJARs()) {
for (int i = 0; i < jarFiles.length; i++) {
jarEntry = jarFiles[i].getJarEntry(jarEntryPath);
if (jarEntry != null) {
try {
entry.manifest = jarFiles[i].getManifest();
} catch (IOException ioe) {
// Ignore
}
break;
}
}
}
}
}
return entry;
}
int contentLength = -1;
InputStream binaryStream = null;
boolean isClassResource = path.endsWith(CLASS_FILE_SUFFIX);
boolean isCacheable = isClassResource;
if (!isCacheable) {
isCacheable = path.startsWith(SERVICES_PREFIX);
}
int jarFilesLength = jarFiles.length;
int repositoriesLength = repositories.length;
int i;
Resource resource = null;
boolean fileNeedConvert = false;
for (i = 0; (entry == null) && (i < repositoriesLength); i++) {
try {
String fullPath = repositories[i] + path;
Object lookupResult = resources.lookup(fullPath);
if (lookupResult instanceof Resource) {
resource = (Resource) lookupResult;
}
// Note : Not getting an exception here means the resource was
// found
ResourceAttributes attributes =
(ResourceAttributes) resources.getAttributes(fullPath);
contentLength = (int) attributes.getContentLength();
String canonicalPath = attributes.getCanonicalPath();
if (canonicalPath != null) {
// we create the ResourceEntry based on the information returned
// by the DirContext rather than just using the path to the
// repository. This allows to have smart DirContext implementations
// that "virtualize" the docbase (e.g. Eclipse WTP)
entry = findResourceInternal(new File(canonicalPath), "");
} else {
// probably a resource not in the filesystem (e.g. in a
// packaged war)
entry = findResourceInternal(files[i], path);
}
entry.lastModified = attributes.getLastModified();
if (resource != null) {
try {
binaryStream = resource.streamContent();
} catch (IOException e) {
return null;
}
if (needConvert) {
if (path.endsWith(".properties")) {
fileNeedConvert = true;
}
}
// Register the full path for modification checking
// Note: Only syncing on a 'constant' object is needed
synchronized (allPermission) {
int j;
long[] result2 =
new long[lastModifiedDates.length + 1];
for (j = 0; j < lastModifiedDates.length; j++) {
result2[j] = lastModifiedDates[j];
}
result2[lastModifiedDates.length] = entry.lastModified;
lastModifiedDates = result2;
String[] result = new String[paths.length + 1];
for (j = 0; j < paths.length; j++) {
result[j] = paths[j];
}
result[paths.length] = fullPath;
paths = result;
}
}
} catch (NamingException e) {
// Ignore
}
}
if ((entry == null) && (notFoundResources.containsKey(name)))
return null;
synchronized (jarFiles) {
try {
if (!openJARs()) {
return null;
}
for (i = 0; (entry == null) && (i < jarFilesLength); i++) {
jarEntry = jarFiles[i].getJarEntry(jarEntryPath);
if (jarEntry != null) {
entry = new ResourceEntry();
try {
entry.codeBase = getURI(jarRealFiles[i]);
entry.source =
UriUtil.buildJarUrl(entry.codeBase.toString(), jarEntryPath);
entry.lastModified = jarRealFiles[i].lastModified();
} catch (MalformedURLException e) {
return null;
}
contentLength = (int) jarEntry.getSize();
try {
if (manifestRequired) {
entry.manifest = jarFiles[i].getManifest();
} else {
entry.manifest = MANIFEST_UNKNOWN;
}
binaryStream = jarFiles[i].getInputStream(jarEntry);
} catch (IOException e) {
return null;
}
// Extract resources contained in JAR to the workdir
if (antiJARLocking && !(path.endsWith(CLASS_FILE_SUFFIX))) {
byte[] buf = new byte[1024];
File resourceFile = new File(loaderDir, jarEntry.getName());
if (!resourceFile.exists()) {
Enumeration<JarEntry> entries = jarFiles[i].entries();
while (entries.hasMoreElements()) {
JarEntry jarEntry2 = entries.nextElement();
if (!(jarEntry2.isDirectory()) &&
(!jarEntry2.getName().endsWith(CLASS_FILE_SUFFIX))) {
resourceFile = new File(loaderDir, jarEntry2.getName());
try {
if (!resourceFile.getCanonicalPath().startsWith(
canonicalLoaderDir)) {
throw new IllegalArgumentException(
sm.getString("webappClassLoader.illegalJarPath",
jarEntry2.getName()));
}
} catch (IOException ioe) {
throw new IllegalArgumentException(
sm.getString("webappClassLoader.validationErrorJarPath",
jarEntry2.getName()), ioe);
}
File parentFile = resourceFile.getParentFile();
if (!parentFile.mkdirs() && !parentFile.exists()) {
// Ignore the error (like the IOExceptions below)
}
FileOutputStream os = null;
InputStream is = null;
try {
is = jarFiles[i].getInputStream(jarEntry2);
os = new FileOutputStream(resourceFile);
while (true) {
int n = is.read(buf);
if (n <= 0) {
break;
}
os.write(buf, 0, n);
}
resourceFile.setLastModified(jarEntry2.getTime());
} catch (IOException e) {
// Ignore
} finally {
try {
if (is != null) {
is.close();
}
} catch (IOException e) {
// Ignore
}
try {
if (os != null) {
os.close();
}
} catch (IOException e) {
// Ignore
}
}
}
}
}
}
}
}
if (entry == null) {
synchronized (notFoundResources) {
notFoundResources.put(name, name);
}
return null;
}
/* Only cache the binary content if there is some content
* available one of the following is true:
* a) It is a class file since the binary content is only cached
* until the class has been loaded
* or
* b) The file needs conversion to address encoding issues (see
* below)
* or
* c) The resource is a service provider configuration file located
* under META=INF/services
*
* In all other cases do not cache the content to prevent
* excessive memory usage if large resources are present (see
* https://bz.apache.org/bugzilla/show_bug.cgi?id=53081).
*/
if (binaryStream != null && (isCacheable || fileNeedConvert)) {
byte[] binaryContent = new byte[contentLength];
int pos = 0;
try {
while (true) {
int n = binaryStream.read(binaryContent, pos,
binaryContent.length - pos);
if (n <= 0)
break;
pos += n;
}
} catch (IOException e) {
log.error(sm.getString("webappClassLoader.readError", name), e);
return null;
}
if (fileNeedConvert) {
// Workaround for certain files on platforms that use
// EBCDIC encoding, when they are read through FileInputStream.
// See commit message of rev.303915 for details
// http://svn.apache.org/viewvc?view=revision&revision=303915
String str = new String(binaryContent,0,pos);
try {
binaryContent = str.getBytes(CHARSET_UTF8);
} catch (Exception e) {
return null;
}
}
entry.binaryContent = binaryContent;
// The certificates are only available after the JarEntry
// associated input stream has been fully read
if (jarEntry != null) {
entry.certificates = jarEntry.getCertificates();
}
}
} finally {
if (binaryStream != null) {
try {
binaryStream.close();
} catch (IOException e) { /* Ignore */}
}
}
}
if (isClassResource && entry.binaryContent != null &&
this.transformers.size() > 0) {
// If the resource is a class just being loaded, decorate it
// with any attached transformers
String className = name.endsWith(CLASS_FILE_SUFFIX) ?
name.substring(0, name.length() - CLASS_FILE_SUFFIX.length()) : name;
String internalName = className.replace(".", "/");
for (ClassFileTransformer transformer : this.transformers) {
try {
byte[] transformed = transformer.transform(
this, internalName, null, null, entry.binaryContent
);
if (transformed != null) {
entry.binaryContent = transformed;
}
} catch (IllegalClassFormatException e) {
log.error(sm.getString("webappClassLoader.transformError", name), e);
return null;
}
}
}
// Add the entry in the local resource repository
synchronized (resourceEntries) {
// Ensures that all the threads which may be in a race to load
// a particular class all end up with the same ResourceEntry
// instance
ResourceEntry entry2 = resourceEntries.get(path);
if (entry2 == null) {
resourceEntries.put(path, entry);
} else {
entry = entry2;
}
}
return entry;
}
复制代码很明显其实按照之前的file的顺序来做的遍历 返回第一个即可。
在mac新版本系统【macos high-sierra】和tomcat8的组合中发生了资源文件加载覆盖的问题【目前tomcat7无问题】 初步怀疑tomcat8中classloader更换了实现
同时mac新版本在升级时也更换了实现导致出现聪明的资源文件时加载顺序不一致 出现部分资源定义无法加载的错误
tomcat8中未作排序
@Override
public String[] list(String path) {
checkPath(path);
String webAppMount = getWebAppMount();
if (path.startsWith(webAppMount)) {
File f = file(path.substring(webAppMount.length()), true);
if (f == null) {
return EMPTY_STRING_ARRAY;
}
String[] result = f.list();
if (result == null) {
return EMPTY_STRING_ARRAY;
} else {
return result;
}
} else {
if (!path.endsWith("/")) {
path = path + "/";
}
if (webAppMount.startsWith(path)) {
int i = webAppMount.indexOf('/', path.length());
if (i == -1) {
return new String[] {webAppMount.substring(path.length())};
} else {
return new String[] {
webAppMount.substring(path.length(), i)};
}
}
return EMPTY_STRING_ARRAY;
}
}
复制代码tomcat8直接调用f.list 并未作排序
/**
* Returns an array of strings naming the files and directories in the
* directory denoted by this abstract pathname.
*
* <p> If this abstract pathname does not denote a directory, then this
* method returns {@code null}. Otherwise an array of strings is
* returned, one for each file or directory in the directory. Names
* denoting the directory itself and the directory's parent directory are
* not included in the result. Each string is a file name rather than a
* complete path.
*
* <p> There is no guarantee that the name strings in the resulting array
* will appear in any specific order; they are not, in particular,
* guaranteed to appear in alphabetical order.
*
* <p> Note that the {@link java.nio.file.Files} class defines the {@link
* java.nio.file.Files#newDirectoryStream(Path) newDirectoryStream} method to
* open a directory and iterate over the names of the files in the directory.
* This may use less resources when working with very large directories, and
* may be more responsive when working with remote directories.
*
* @return An array of strings naming the files and directories in the
* directory denoted by this abstract pathname. The array will be
* empty if the directory is empty. Returns {@code null} if
* this abstract pathname does not denote a directory, or if an
* I/O error occurs.
*
* @throws SecurityException
* If a security manager exists and its {@link
* SecurityManager#checkRead(String)} method denies read access to
* the directory
*/
public String[] list() {
SecurityManager security = System.getSecurityManager();
if (security != null) {
security.checkRead(path);
}
return fs.list(this);
}
复制代码而list方法写的比较清楚这个顺序不受保证【通常意义是字典序但是不保证】
There is no guarantee that the name strings in the resulting array
will appear in any specific order; they are not, in particular,
guaranteed to appear in alphabetical order.
复制代码因此部分小伙伴升级mac新系统使用tomcat8时出现了加载顺序不一致
扫一扫
584
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
