小记EM12C无法访问问题

今天重启了下DB后在使用EM12C的时候尽然报错无法访问，登陆页面如下提示：

Backend WLS or EM application seems to be down

检查了下数据库，都是ok的，部分日志信息：
2015-Feb-04 11:27:13 : Failed to get repository details from credstore
: Repository credentials are not available in Credential Store
Loading integration file at /u01/app/gc_inst/em/EMGC_OMS1/sysman/config/, emomsintg.xml
intg.xml file does not exist at emomsintg.xml
securePort = -1
isOMSSecure = 0 isOMSSecureLocked = 0

policystore.PolicyStoreConnectivityException: JPS-10000: There was an internal error in the policy store..> 
Exception caught while getting credentials JPS-01030: Cannot get credential. Reason oracle.security.jps.service.policystore.PolicyStor
eConnectivityException: JPS-10000: There was an internal error in the policy store..
oracle.security.jps.service.credstore.CredStoreException: JPS-01030: Cannot get credential. Reason oracle.security.jps.service.policys
tore.PolicyStoreConnectivityException: JPS-10000: There was an internal error in the policy store..
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.getCredential(LdapCredentialStore.java:364)
at oracle.sysman.eml.sec.credstore.EMCredentialStore$1.run(EMCredentialStore.java:105)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.sysman.eml.sec.credstore.EMCredentialStore.getRepositoryCredInfo(EMCredentialStore.java:98)
at oracle.sysman.emSDK.conf.ConfigManager.getReposPropFromCredStore(ConfigManager.java:2178)
at oracle.sysman.emSDK.svc.conn.ConnectionService.verifyRepositoryEx(ConnectionService.java:890)
at oracle.sysman.emSDK.svc.conn.ConnectionService.verifyRepository(ConnectionService.java:1018)
at oracle.sysman.core.common.server.Server.checkRepositoryConnection(Server.java:266)
at oracle.sysman.eml.app.EMConsoleServer.checkPointOnRepositoryConnectivity(EMConsoleServer.java:444)
at oracle.sysman.core.common.server.Server.handleCheckPoints(Server.java:326)
at oracle.sysman.core.common.server.StartUpShutdownHandler.initiateOrResumeStartupProcess(StartUpShutdownHandler.java:182)
at oracle.sysman.eml.app.EMConsoleServer.onStartUp(EMConsoleServer.java:163)
at oracle.sysman.eml.app.ContextInitializer.contextInitialized(ContextInitializer.java:203)
at weblogic.servlet.internal.EventsManager$FireContextListenerAction.run(EventsManager.java:481)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:181)
at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1868)
at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:3154)
at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1518)
at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:484)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:671)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:212)
at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:59)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
at weblogic.deploy.internal.targetserver.BasicDeployment.activate(BasicDeployment.java:184)
at weblogic.deploy.internal.targetserver.BasicDeployment.activateFromServerLifecycle(BasicDeployment.java:361)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doActivate(DeploymentAdapter.java:51)
at weblogic.management.deploy.internal.DeploymentAdapter.activate(DeploymentAdapter.java:200)
at weblogic.management.deploy.internal.AppTransition$2.transitionApp(AppTransition.java:30)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.activate(ConfiguredDeployments.java:169)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:123)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:180)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:96)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
2015-Feb-04 11:27:29 : Failed to get repository details from credstore
: Repository credentials are not available in Credential Store
Unable to retrive repository credentials from credential store. Verify that repository credentials are set in credential store.
:Enter.
In ConfigManager.loadProperties. Loaded properties from property file
ties>

policystore.PolicyStoreConnectivityException: JPS-10000: There was an internal error in the policy store..> 
Exception caught while getting credentials JPS-01030: Cannot get credential. Reason oracle.security.jps.service.policystore.PolicyStor
eConnectivityException: JPS-10000: There was an internal error in the policy store..
oracle.security.jps.service.credstore.CredStoreException: JPS-01030: Cannot get credential. Reason oracle.security.jps.service.policys
tore.PolicyStoreConnectivityException: JPS-10000: There was an internal error in the policy store..
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.getCredential(LdapCredentialStore.java:364)
at oracle.sysman.eml.sec.credstore.EMCredentialStore$1.run(EMCredentialStore.java:105)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.sysman.eml.sec.credstore.EMCredentialStore.getRepositoryCredInfo(EMCredentialStore.java:98)
at oracle.sysman.emSDK.conf.ConfigManager.getReposPropFromCredStore(ConfigManager.java:2178)
at oracle.sysman.emSDK.svc.conn.ConnectionService.verifyRepositoryEx(ConnectionService.java:890)
at oracle.sysman.emSDK.svc.conn.ConnectionService.verifyRepository(ConnectionService.java:1018)
at oracle.sysman.emSDK.svc.conn.ConnectionService.blockUntilValid(ConnectionService.java:1049)
at oracle.sysman.core.common.server.Server$WaitForConnection.run(Server.java:574)
2015-Feb-04 11:27:44 : Failed to get repository details from credstore
: Repository credentials are not available in Credential Store
Unable to retrive repository credentials from credential store. Verify that repository credentials are set in credential store.

policystore.PolicyStoreConnectivityException: JPS-10000: There was an internal error in the policy store..> 
Exception caught while getting credentials JPS-01030: Cannot get credential. Reason oracle.security.jps.service.policystore.PolicyStor
eConnectivityException: JPS-10000: There was an internal error in the policy store..
oracle.security.jps.service.credstore.CredStoreException: JPS-01030: Cannot get credential. Reason oracle.security.jps.service.policys
tore.PolicyStoreConnectivityException: JPS-10000: There was an internal error in the policy store..
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.getCredential(LdapCredentialStore.java:364)
at oracle.sysman.eml.sec.credstore.EMCredentialStore$1.run(EMCredentialStore.java:105)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.sysman.eml.sec.credstore.EMCredentialStore.getRepositoryCredInfo(EMCredentialStore.java:98)
at oracle.sysman.emSDK.conf.ConfigManager.getReposPropFromCredStore(ConfigManager.java:2178)
at oracle.sysman.emSDK.svc.conn.ConnectionService.verifyRepositoryEx(ConnectionService.java:890)
at oracle.sysman.emSDK.svc.conn.ConnectionService.verifyRepository(ConnectionService.java:1018)
at oracle.sysman.eml.app.EMConsoleServer.setServerState(EMConsoleServer.java:281)
at oracle.sysman.eml.app.EMConsoleServer.onStartUp(EMConsoleServer.java:168)
at oracle.sysman.eml.app.ContextInitializer.contextInitialized(ContextInitializer.java:203)
at weblogic.servlet.internal.EventsManager$FireContextListenerAction.run(EventsManager.java:481)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:181)
at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1868)
at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:3154)
at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1518)
at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:484)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200)
at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247)
at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:671)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:212)
at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:59)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
at weblogic.deploy.internal.targetserver.BasicDeployment.activate(BasicDeployment.java:184)
at weblogic.deploy.internal.targetserver.BasicDeployment.activateFromServerLifecycle(BasicDeployment.java:361)
at weblogic.management.deploy.internal.DeploymentAdapter$1.doActivate(DeploymentAdapter.java:51)
at weblogic.management.deploy.internal.DeploymentAdapter.activate(DeploymentAdapter.java:200)
at weblogic.management.deploy.internal.AppTransition$2.transitionApp(AppTransition.java:30)
at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
at weblogic.management.deploy.internal.ConfiguredDeployments.activate(ConfiguredDeployments.java:169)
at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:123)
at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:180)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:96)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
2015-Feb-04 11:27:44 : Failed to get repository details from credstore
: Repository credentials are not available in Credential Store
Unable to retrive repository credentials from credential store. Verify that repository credentials are set in credential store.
Got connection service status as: 16

ested.> 
Fetched repository credentials from Credential Store
ConnectionService.init: Get repos properties from credential store.
Fetched repository credentials from Credential Store
Fetched repository credentials from Credential Store




检查数据库是否ok：
[oracle@oem u01]$ ps -ef | grep pmon
oracle    1546 20211  0 11:45 pts/3    00:00:00 grep pmon
oracle   29258     1  0 11:26 ?        00:00:00 ora_pmon_orcl
关闭oms服务：
[oracle@oem u01]$ /u01/app/em12c/oms/bin/emctl stop oms -all
Oracle Enterprise Manager Cloud Control 12c Release 4  
Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.
Stopping WebTier...
WebTier Successfully Stopped
Stopping Oracle Management Server...
Oracle Management Server Already Stopped
AdminServer Successfully Stopped
Oracle Management Server is Down
重启oms服务：
[oracle@oem u01]$ /u01/app/em12c/oms/bin/emctl start oms
Oracle Enterprise Manager Cloud Control 12c Release 4  
Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
Starting WebTier...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
[oracle@oem u01]$ 
登陆检测完全ok了

注意事项：在使用emctl工具时一定要注意此时使用的是oms路径下的，如果使用oracle路径下的会报错如下：
[oracle@oem u01]$ emctl start oms
EM Configuration issue. /u01/app/oracle/product/11.2.0/dbhome_1/oem.example.com_orcl not found. 

关于该问题在metalink上也有这样一段：
12C Cloud Control: Accessing the Console URL Shows Error Message 'Backend WLS or EM application seems to be down' [ID 1367997.1]
Cause
Repository Database has been shutdown / crashed due to which the OMS also crashed.
Solution
1. Re-start the repository database and verify that a connection can be made to the 'sysman' schema in the database via the 
listener.
2. Re-start the OMS:
cd /bin
emctl stop oms -all
emctl start oms

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/28612416/viewspace-1426045/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/28612416/viewspace-1426045/