Environment
- Red Hat JBoss Enterprise Application Platform (EAP)
- 6.x
- 7.x
Issue
- Server throws following error while start up :
12:21:13,956 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS013412: Timeout after [300] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'add' at address '[("interface" => "management")]'
-
There are pretty big applications running on server and it may take 5-6 min to start the servers. So how to set the timeout to 600 or 900 seconds ? Where and how to set it ?
-
Slave
JBoss EAP
instance won't start when application code takes too long to load. -
We are coming across this bug mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1117945 and were wondering if it has been fixed in 6.3.3 or the up-coming 6.4.
-
When trying to deploy a new version of our application I get the following error. ” Operation timeout awaiting service container stability" typically the application will deploy pretty quick. Now it waits until it times out, What could be causing this problem?
-
Jboss application server is not able to restart after throwing the below error. We had this issue in of our servers in the past, so had set this property, looks like the server is waiting 600 seconds for the deployment to happen and failing after that. Could you please advice us on what the issue could be, as our QA environment is down at this moment?
02:30:20,757 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS013412: Timeout after [600] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'add' at address '[
("core-service" => "management"),
("management-interface" => "native-interface")
]'
- We run
jboss
on a rather underpowered VM for test purposes,sometimes it can take a long time to start, and then times out.Is it possible to extend the timeout? - Unable to deploy WAR file on AWS VM
- When we start the JBOSSEAP6 windows service, the deployment produces a ".failed" file in the deployments folder, rather than a ".deployment" file. This is a newly-created AWS EC2 Windows Server 2012 R@ instance - a test environment for now and issue comes every time we try to start the JBoss service.
Resolution
In EAP 7.x and EAP 6.3.0 or later, you can configure jboss.as.management.blocking.timeout
system property to tune timeout (seconds) waiting for service container stability.
For standalone mode :
-
Use following CLI command to set
jboss.as.management.blocking.timeout
. For example:/system-property=jboss.as.management.blocking.timeout:add(value=600)
the above will set
jboss.as.management.blocking.timeout
system property in configuration file (i.estandalone-*.xml
) like :... </extensions> <system-properties> <property name="jboss.as.management.blocking.timeout" value="600"/> </system-properties> <management> ...
or
-
Add
-Djboss.as.management.blocking.timeout=600
to jvm argument ofstandalone.sh
or add the following JAVA_OPTS instandalone.conf
:JAVA_OPTS="$JAVA_OPTS -Djboss.as.management.blocking.timeout=600"
For domain mode :
- Edit the $JBOSS_HOME/bin/domain.conf and set it via this line
-Djboss.as.management.blocking.timeout=600
after the if block:
...
#
# Specify options to pass to the Java VM.
#
if [ "x$JAVA_OPTS" = "x" ]; then
JAVA_OPTS="-Xms64m -Xmx512m -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true"
JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"
else
echo "JAVA_OPTS already set in environment; overriding default settings with values: $JAVA_OPTS"
fi
JAVA_OPTS="$JAVA_OPTS -Djboss.as.management.blocking.timeout=600"
...
Note: The only use case for setting a property at the server-group or individual domain server level would be if you wanted a lower timeout than what you configure on the host controllers.
Root Cause
- In EAP 6.3.0.ER8 there is a new class in
jboss.as.controller: BlockingTimeout
. This class loads the value of system propertyjboss.as.management.blocking.timeout
or defaults to 300 (seconds). Note: This property is not a timeout per deployment but a timeout on container stability and ifjboss.as.management.blocking.timeout
is reached during startup then all applications will be undeployed and the container shutdown. The reasoning behind this is that having a half-working server is potentially dangerous as you may not notice major failures.
Diagnostic Steps
- Confirm whether a virus scanner is installed to the server
- If the issue was caused after making changes in application, check what exactly has changed since it was working previously? Was something upgraded? Added? Pointing to a new database or other external system?
- Collect a series of thread dumps during your startup period so we can see what it might be getting stuck on.
- Make sure to add the setting to the domain.conf on the Domain controller and Slave Host controller
Once you collect we can analyse them and to see whether there is some sort of deadlock or resource that your threads are waiting on that's preventing them from completing the deployment etc.
Symptoms
When starting JBoss, the following error is printed in application server log:
ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS013412: Timeout after [300] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'add' at address '[
("core-service" => "management"),
("management-interface" => "native-interface")
]'
JBoss then fails to deploy.
Cause
This error message is printed out when the management-native
socket defined in standalone.xml located in <JBOSS_HOME>/standalone/configuration
takes longer than the configured timeout value to load. By default, this value is set to 300 seconds.
Action
-
Open standalone.conf (Unix) or standalone.conf.bat (Windows) located in
<JBOSS_HOME>/bin
and verify the following timeout value is large enough:JAVA_OPTS="$JAVA_OPTS -Djboss.as.management.blocking.timeout=3600"
-
Open standalone.xml and modify the
default-timeout
property in thecoordinator-environment
element to a larger value (e.g., 1000):<coordinator-environment default-timeout="..."/>
Note: In case the changes are not applied, clear the JBoss cache by deleting the following folders:
<JBOSS_HOME>/standalone/data
<JBOSS_HOME>/standalone/tmp
Affected Versions
This article applies to all versions of Appian using JBoss as an application server.
Last Reviewed: February 2017