B/S系统, 应用服务器为linux集群服务器,
中间件为weblogic,版本为:Version: WebLogic Server 9.2 MP3 Mon Mar 10 08:28:41 EDT 2008 1096261
共计10台,IP地址分别为：
192.168.0.1
....
192.168.0.8
192.168.0.9
192.168.0.10
应用服务器配置完全相同,
为两颗物理CPU(physical id为0 , 1 ) ，不知道对不对？？
详细信息如下
---------------------------------应用服务器CPU信息-------------------------------------------------
[weblogic@server01]$ cat proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5656.37
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 1
siblings : 4
core id : 0
cpu cores : 4
apicid : 4
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5652.54
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 4
apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5652.50
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 1
siblings : 4
core id : 2
cpu cores : 4
apicid : 6
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5652.54
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 4
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5652.51
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 5
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 1
siblings : 4
core id : 1
cpu cores : 4
apicid : 5
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5652.55
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 6
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 3
cpu cores : 4
apicid : 3
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5652.51
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU E5440 @ 2.83GHz
stepping : 10
cpu MHz : 2826.256
cache size : 6144 KB
physical id : 1
siblings : 4
core id : 3
cpu cores : 4
apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5652.53
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
--------------------------------------------------------------------------------------------------------------------------------
192.168.0.1 : 参数配置情况
[weblogic@server01]$ free -m
total used free shared buffers cached
Mem: 16047 4382 11664 0 649 1124
-/+ buffers/cache: 2609 13437
Swap: 18433 0 18433
JVM配置:
MEM_ARGS="-Xms2560m -Xmx2560m "
setDomainEnv.sh代码
----------------------------------------------------------------------------------------------------
#!/bin/sh
# WARNING: This file is created by the Configuration Wizard.
# Any changes to this script may be lost when adding extensions to this configuration.
# --- Start Functions ---
BP=100
SP=$BP
pushd()
{
if [ -z "$1" ]
then
return
fi
SP=`expr $SP - 1`
eval _stack$SP=`pwd`
cd $1
return
}
popd()
{
if [ $SP -eq $BP ]
then
return
fi
eval cd \${_stack$SP}
SP=`expr $SP + 1`
return
}
# --- End Functions ---
# *************************************************************************
# This script is used to setup the needed environment to be able to start Weblogic Server in this domain.
#
# This script initializes the following variables before calling commEnv to set other variables:
#
# WL_HOME - The BEA home directory of your WebLogic installation.
# JAVA_VM - The desired Java VM to use. You can set this environment variable before calling
# this script to switch between Sun or BEA or just have the default be set.
# JAVA_HOME - Location of the version of Java used to start WebLogic
# Server. Depends directly on which JAVA_VM value is set by default or by the environment.
# USER_MEM_ARGS - The variable to override the standard memory arguments
# passed to java.
# PRODUCTION_MODE - The variable that determines whether Weblogic Server is started in production mode.
# DOMAIN_PRODUCTION_MODE
# - The variable that determines whether the workshop related settings like the debugger,
# testconsole or iterativedev should be enabled. ONLY settable using the
# command-line parameter named production
# NOTE: Specifying the production command-line param will force
# the server to start in production mode.
#
# Other variables used in this script include:
# SERVER_NAME - Name of the weblogic server.
# JAVA_OPTIONS - Java command-line options for running the server. (These
# will be tagged on to the end of the JAVA_VM and
# MEM_ARGS)
#
# For additional information, refer to the WebLogic Server Administration
# Console Online Help(http://e-docs.bea.com/wls/docs92/ConsoleHelp/startstop.html).
# *************************************************************************
WL_HOME="/home/weblogic/bea/weblogic92"
export WL_HOME
BEA_JAVA_HOME=""
export BEA_JAVA_HOME
SUN_JAVA_HOME="/usr/java/jdk1.5.0_15"
export SUN_JAVA_HOME
if [ "${JAVA_VENDOR}" = "BEA" ] ; then
JAVA_HOME="${BEA_JAVA_HOME}"
export JAVA_HOME
else
if [ "${JAVA_VENDOR}" = "Sun" ] ; then
JAVA_HOME="${SUN_JAVA_HOME}"
export JAVA_HOME
else
JAVA_VENDOR="Sun"
export JAVA_VENDOR
JAVA_HOME="/usr/java/jdk1.5.0_15"
export JAVA_HOME
fi
fi
# We need to reset the value of JAVA_HOME to get it shortened AND
# we can not shorten it above because immediate variable expansion will blank it
JAVA_HOME="${JAVA_HOME}"
export JAVA_HOME
SAMPLES_HOME="${WL_HOME}/samples"
export SAMPLES_HOME
DOMAIN_HOME="/home/weblogic/bea/user_projects/domains/pms"
export DOMAIN_HOME
LONG_DOMAIN_HOME="/home/weblogic/bea/user_projects/domains/pms"
export LONG_DOMAIN_HOME
if [ "${DEBUG_PORT}" = "" ] ; then
DEBUG_PORT="8453"
export DEBUG_PORT
fi
if [ "${SERVER_NAME}" = "" ] ; then
SERVER_NAME="AdminServer"
export SERVER_NAME
fi
POINTBASE_FLAG="false"
export POINTBASE_FLAG
enableHotswapFlag=""
export enableHotswapFlag
PRODUCTION_MODE="true"
export PRODUCTION_MODE
doExitFlag="false"
export doExitFlag
verboseLoggingFlag="false"
export verboseLoggingFlag
while [ $# -gt 0 ]
do
case $1 in
nodebug)
debugFlag="false"
export debugFlag
;;
production)
DOMAIN_PRODUCTION_MODE="true"
export DOMAIN_PRODUCTION_MODE
;;
notestconsole)
testConsoleFlag="false"
export testConsoleFlag
;;
noiterativedev)
iterativeDevFlag="false"
export iterativeDevFlag
;;
noLogErrorsToConsole)
logErrorsToConsoleFlag="false"
export logErrorsToConsoleFlag
;;
nopointbase)
POINTBASE_FLAG="false"
export POINTBASE_FLAG
;;
doExit)
doExitFlag="true"
export doExitFlag
;;
noExit)
doExitFlag="false"
export doExitFlag
;;
verbose)
verboseLoggingFlag="true"
export verboseLoggingFlag
;;
enableHotswap)
enableHotswapFlag="-javaagent:${WL_HOME}/server/lib/diagnostics-agent.jar"
export enableHotswapFlag
;;
*)
PROXY_SETTINGS="${PROXY_SETTINGS} $1"
export PROXY_SETTINGS
;;
esac
shift
done
MEM_DEV_ARGS=""
export MEM_DEV_ARGS
if [ "${DOMAIN_PRODUCTION_MODE}" = "true" ] ; then
PRODUCTION_MODE="${DOMAIN_PRODUCTION_MODE}"
export PRODUCTION_MODE
fi
if [ "${PRODUCTION_MODE}" = "true" ] ; then
debugFlag="false"
export debugFlag
testConsoleFlag="false"
export testConsoleFlag
iterativeDevFlag="false"
export iterativeDevFlag
fi
# If you want to override the default Patch Classpath, Library Path and Path for this domain,
# Please uncomment the following lines and add a valid value for the environment variables
# set PATCH_CLASSPATH=[myPatchClasspath] (windows)
# set PATCH_LIBPATH=[myPatchLibpath] (windows)
# set PATCH_PATH=[myPatchPath] (windows)
# PATCH_CLASSPATH=[myPatchClasspath] (unix)
# PATCH_LIBPATH=[myPatchLibpath] (unix)
# PATCH_PATH=[myPatchPath] (unix)
. ${WL_HOME}/common/bin/commEnv.sh
WLS_HOME="${WL_HOME}/server"
export WLS_HOME
WLI_HOME="${WL_HOME}/integration"
export WLI_HOME
MEM_ARGS="-Xms2560m -Xmx2560m "
export MEM_ARGS
if [ "${JAVA_VENDOR}" = "Sun" ] ; then
if [ "${PRODUCTION_MODE}" = "true" ] ; then
MEM_DEV_ARGS="-XX:CompileThreshold=8000 -XX:PermSize=128m "
export MEM_DEV_ARGS
fi
fi
# Had to have a separate test here BECAUSE of immediate variable expansion on windows
if [ "${JAVA_VENDOR}" = "Sun" ] ; then
MEM_ARGS="${MEM_ARGS} ${MEM_DEV_ARGS} -XX:MaxPermSize=512m"
export MEM_ARGS
fi
if [ "${JAVA_VENDOR}" = "HP" ] ; then
MEM_ARGS="${MEM_ARGS} -XX:MaxPermSize=128m"
export MEM_ARGS
fi
# IF USER_MEM_ARGS the environment variable is set, use it to override ALL MEM_ARGS values
if [ "${USER_MEM_ARGS}" != "" ] ; then
MEM_ARGS="${USER_MEM_ARGS}"
export MEM_ARGS
fi
JAVA_PROPERTIES="-Dplatform.home=${WL_HOME} -Dwls.home=${WLS_HOME} -Dwli.home=${WLI_HOME}"
export JAVA_PROPERTIES
# To use Java Authorization Contract for Containers (JACC) in this domain,
# please uncomment the following section. If there are multiple machines in
# your domain, be sure to edit the setDomainEnv in the associated domain on
# each machine.
#
# -Djava.security.manager
# -Djava.security.policy=location of weblogic.policy
# -Djavax.security.jacc.policy.provider=weblogic.security.jacc.simpleprovider.SimpleJACCPolicy
# -Djavax.security.jacc.PolicyConfigurationFactory.provider=weblogic.security.jacc.simpleprovider.PolicyConfigurationFactoryImpl
# -Dweblogic.security.jacc.RoleMapperFactory.provider=weblogic.security.jacc.simpleprovider.RoleMapperFactoryImpl
JAVA_PROPERTIES="${JAVA_PROPERTIES} ${EXTRA_JAVA_PROPERTIES}"
export JAVA_PROPERTIES
ARDIR="${WL_HOME}/server/lib"
export ARDIR
pushd ${LONG_DOMAIN_HOME}
# Clustering support (edit for your cluster!)
if [ "${ADMIN_URL}" = "" ] ; then
# The then part of this block is telling us we are either starting an admin server OR we are non-clustered
CLUSTER_PROPERTIES="-Dweblogic.management.discover=true"
export CLUSTER_PROPERTIES
else
CLUSTER_PROPERTIES="-Dweblogic.management.discover=false -Dweblogic.management.server=${ADMIN_URL}"
export CLUSTER_PROPERTIES
fi
if [ "${LOG4J_CONFIG_FILE}" != "" ] ; then
JAVA_PROPERTIES="${JAVA_PROPERTIES} -Dlog4j.configuration=file:${LOG4J_CONFIG_FILE}"
export JAVA_PROPERTIES
fi
JAVA_PROPERTIES="${JAVA_PROPERTIES} ${CLUSTER_PROPERTIES}"
export JAVA_PROPERTIES
# Clear the pre_classpath here in case an application template wants to set it before the larger pre_classpath is invoked below
PRE_CLASSPATH=""
export PRE_CLASSPATH
JAVA_DEBUG=""
export JAVA_DEBUG
if [ "${debugFlag}" = "true" ] ; then
JAVA_DEBUG="-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=${DEBUG_PORT},server=y,suspend=n -Djava.compiler=NONE"
export JAVA_DEBUG
JAVA_OPTIONS="${JAVA_OPTIONS} ${enableHotswapFlag} -ea -da:com.bea... -da:javelin... -da:weblogic... -ea:com.bea.wli... -ea:com.bea.broker... -ea:com.bea.sbconsole..."
export JAVA_OPTIONS
else
JAVA_OPTIONS="${JAVA_OPTIONS} ${enableHotswapFlag} -da"
export JAVA_OPTIONS
fi
if [ ! -d ${JAVA_HOME}/lib ] ; then
echo "The JRE was not found in directory ${JAVA_HOME}. (JAVA_HOME)"
echo "Please edit your environment and set the JAVA_HOME"
echo "variable to point to the root directory of your Java installation."
popd
read _val
exit
fi
if [ "${POINTBASE_FLAG}" = "true" ] ; then
DATABASE_CLASSPATH="${POINTBASE_CLASSPATH}"
export DATABASE_CLASSPATH
else
DATABASE_CLASSPATH="${POINTBASE_CLIENT_CLASSPATH}"
export DATABASE_CLASSPATH
fi
POST_CLASSPATH=""
export POST_CLASSPATH
POST_CLASSPATH="${POST_CLASSPATH}${CLASSPATHSEP}${DATABASE_CLASSPATH}${CLASSPATHSEP}${ARDIR}/xqrl.jar"
export POST_CLASSPATH
# PROFILING SUPPORT
JAVA_PROFILE=""
export JAVA_PROFILE
SERVER_CLASS="weblogic.Server"
export SERVER_CLASS
JAVA_PROPERTIES="${JAVA_PROPERTIES} ${WLP_JAVA_PROPERTIES}"
export JAVA_PROPERTIES
JAVA_OPTIONS="${JAVA_OPTIONS} ${JAVA_PROPERTIES} -Dwlw.iterativeDev=${iterativeDevFlag} -Dwlw.testConsole=${testConsoleFlag} -Dwlw.logErrorsToConsole=${logErrorsToConsoleFlag}"
export JAVA_OPTIONS
# -- Setup properties so that we can save stdout and stderr to files
if [ "${WLS_STDOUT_LOG}" != "" ] ; then
echo "Logging WLS stdout to ${WLS_STDOUT_LOG}"
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.Stdout=${WLS_STDOUT_LOG}"
export JAVA_OPTIONS
fi
if [ "${WLS_STDERR_LOG}" != "" ] ; then
echo "Logging WLS stderr to ${WLS_STDERR_LOG}"
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.Stderr=${WLS_STDERR_LOG}"
export JAVA_OPTIONS
fi
# ADD EXTENSIONS TO CLASSPATHS
if [ "${EXT_PRE_CLASSPATH}" != "" ] ; then
PRE_CLASSPATH="${EXT_PRE_CLASSPATH}${CLASSPATHSEP}${PRE_CLASSPATH}"
export PRE_CLASSPATH
fi
if [ "${EXT_POST_CLASSPATH}" != "" ] ; then
POST_CLASSPATH="${POST_CLASSPATH}${CLASSPATHSEP}${EXT_POST_CLASSPATH}"
export POST_CLASSPATH
fi
if [ "${WEBLOGIC_EXTENSION_DIRS}" != "" ] ; then
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.ext.dirs=${WEBLOGIC_EXTENSION_DIRS}"
export JAVA_OPTIONS
fi
JAVA_OPTIONS="${JAVA_OPTIONS}"
export JAVA_OPTIONS
# SET THE CLASSPATH
CLASSPATH="${PRE_CLASSPATH}${CLASSPATHSEP}${WEBLOGIC_CLASSPATH}${CLASSPATHSEP}${POST_CLASSPATH}${CLASSPATHSEP}${WLP_POST_CLASSPATH}"
export CLASSPATH
JAVA_VM="${JAVA_VM} ${JAVA_DEBUG} ${JAVA_PROFILE}"
export JAVA_VM
----------------------------------------------------------------------------------------------------
192.168.0.2 : 参数配置情况
[weblogic@server02]$ free -m
total used free shared buffers cached
Mem: 16047 2086 13960 0 580 1213
-/+ buffers/cache: 292 15754
Swap: 18433 0 18433
JVM配置：
MEM_ARGS="-Xms2048m -Xmx2048m"
存在的问题:
192.168.0.1运行较稳定,但是其它服务器出现响应缓慢,表现在 http://192.168.0.x:7001/xmlrpc打不开.
上述服务器唯一的差别为: 192.168.0.1参数MEM_ARGS="-Xms2560m -Xmx2560m "
192.168.0.x参数MEM_ARGS="-Xms2048m -Xmx2048m "
---------------------------------------192.168.0.x服务器weblogic中间件出现的错误信息包括: ----------------------------
1. <2011-1-6 下午03时42分41秒 CST>
ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'
has been busy for "1,284" seconds working on the request "Http Request:
/Default.jsp", which is more than the configured time (StuckThreadMaxTime)
of "1,200" seconds. Stack trace:
网上查找了原因，包括以下三方面：
一.某个request占用的时间太长了
二.看一下这个请求是干什么？比如说有的工具可以看出是那条sql语句，你可以优化一下。
另外一种方法调整StuckThreadMaxTime更大
三.重启Weblogic
针对第二条，请问有相关经历的人介绍下相应的工具及使用介绍，以便定位SQL语句
StuckThreadMaxTime目前为1200S，关于这个参数，应该是合适即可，个人理解为太大了，
当某个SQL执行效率很差时，而设置的StuckThreadMaxTime的值又很大，进程一直处于执行任务状态，
可续的任务或许需要等待空闲进程。2分钟时间应该足够了。
第三条是治标不治本，不能从根本上解决此问题。
2.
因为内存溢出或没有可用的内存提供给垃圾回收器时，Java 虚拟机无法分配一个对象，这时抛出该异常
内存溢出是程序编写的问题，没有办法解决，但是可以用工具进行监控，具体什么工具及操作请介绍下。
Java heap space可否通过Weblogic控制台进行优化，某具体一台应用服务器JVM统计信息
Java Virtual Machine Memory Utilization Statistics
--------------------------------------------------------------------------------
Heap Size Current: 1833172992 The current size (in bytes) of the JVM heap.More Info...
Heap Free Current: 124638424 The current amount of memory (in bytes) that is available in the JVM heap.More Info...
Heap Free Percent: 6 Percentage of the maximum memory that is free.More Info...
Heap Size Max: 1908932608 The maximum free memory configured for this JVM.More Info...
存在的疑问:
1. Heap Size Max与 MEM_ARGS最大值不相等,请高人解释下
2. Heap Free Percent仅为6%, 即使运行GC后,也就10%多,最高20%多点,是不是有问题?
3. MEM_ARGS设为2G会更多，还不够。查阅相关资料，应该是内存泄露
把问题再汇总下,有点多:
1. 上述问题请高手尽可能详尽的分析解释下.
2. 当应用服务器出现问题后,收集哪些信息对于解决应用服务器缓慢,
无响应最有为用,下次我好把更详细的错误信息发上来?
3. 对于Weblogic还可以通过哪些参数,配置来 进行调优,解决应用服务器缓慢,无响应的问题.
[本帖最后由 jj2001hh 于 2011-1-8 19:13 编辑]
8074
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
