Oracle under Linux - emctl start dbconsole command failed

After certain time of the testing on the Oracle 11.2.0.1.0 on Linux, encountered below errors.

[tom@ocp ~]$ emctl start dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.1.0
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
https://ocp:1158/em/console/aboutApplication
Starting Oracle Enterprise Manager 11g Database Control ............................................................................................. failed.
------------------------------------------------------------------
Logs are generated in directory /u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/log

Check the log, it shows as below,

[tom@ocp ~]$ emctl status dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.1.0
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
https://ocp:1158/em/console/aboutApplication
Oracle Enterprise Manager 11g is not running

[tom@ocp ~]$ netstat -ntulp | grep 1158
(Not all processes could be identified, non-owned process info

[tom@ocp ~]$ tail /u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/log/emagent.log
2019-07-25 11:21:51,431 Thread-2876659520 ORAXML-229, 文件=file:/u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/emd/lastupld.xml, 行=0, 消息=LPX-00229: input source is empty (01006)
2019-07-25 11:21:51,431 Thread-2876659520 启动代理 10.2.0.4.2 (从 /u01/app/tom/product/11.2.0/dbhome_1) (00701)
2019-07-25 11:21:51,480 Thread-2876659520 [Oracle Exadata Storage Server] InstanceProperty (MgmtIPAddr2) 已标记为 OPTIONAL, 但正在使用中 (00506)
2019-07-25 11:21:51,592 Thread-2876659520 [Load Balancer Switch] InstanceProperty (snmpTimeout) 已标记为 OPTIONAL, 但正在使用中 (00506)
2019-07-25 11:21:51,595 Thread-2876659520 <代理启动> : 初始化 TARGET MANAGER 失败 (00724)
2019-07-25 11:30:22,127 Thread-1127233344 ORAXML-229, 文件=file:/u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/emd/lastupld.xml, 行=0, 消息=LPX-00229: input source is empty (01006)
2019-07-25 11:30:22,127 Thread-1127233344 启动代理 10.2.0.4.2 (从 /u01/app/tom/product/11.2.0/dbhome_1) (00701)
2019-07-25 11:30:22,197 Thread-1127233344 [Oracle Exadata Storage Server] InstanceProperty (MgmtIPAddr2) 已标记为 OPTIONAL, 但正在使用中 (00506)
2019-07-25 11:30:22,317 Thread-1127233344 [Load Balancer Switch] InstanceProperty (snmpTimeout) 已标记为 OPTIONAL, 但正在使用中 (00506)
2019-07-25 11:30:22,321 Thread-1127233344 <代理启动> : 初始化 TARGET MANAGER 失败 (00724)


[tom@ocp ~]$ tail /u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/log/emdb.nohup
19/07/25 11:30:25 isOMSSecure = 1 isOMSSecureLocked = 1
19/07/25 11:30:29 ExtendClusterIntegration: init: start
19/07/25 11:30:29 ExtendClusterIntegration: init: end
19/07/25 11:30:29 ProvCommonIntegration.init, registering the Page Handlers and Page Segment Handlers
19/07/25 11:30:30 PAF Integration init BEGIN
19/07/25 11:30:30 PAF Integration init END
--- DBConsole internal stop. No OC4J admin passwd hence hard stop. ---
----- Thu Jul 25 11:30:30 2019::Commiting Process death. -----
----- Thu Jul 25 11:30:30 2019::Exiting watchdog loop
 -----



[tom@ocp ~]$ tail /u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/log/emdctl.trc
2019-07-25 11:33:13,888 Thread-2726082368 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:15,922 Thread-4011067200 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:17,954 Thread-2068617024 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:19,979 Thread-2846291776 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:22,030 Thread-3364038464 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:24,088 Thread-265647936 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:26,137 Thread-4226950976 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:28,182 Thread-2738038592 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:30,238 Thread-1064294208 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)
2019-07-25 11:33:32,285 Thread-109283136 WARN  http: nmehl_connect_internal: connect failed to (ocp:1158): Connection refused (error = 111)



[tom@ocp ~]$ tail /u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/log/emoms.log
2019-07-25 11:30:26,820 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.emCLI.CLIDownloadIntg
2019-07-25 11:30:27,366 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.eml.target.slb.common.SLBIntegration
2019-07-25 11:30:29,893 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.asprov.ui.intg.ASProvisioningIntegration
2019-07-25 11:30:29,894 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.paf.sample.ui.intg.PAFDemoIntegration
2019-07-25 11:30:29,942 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.sidb.ui.intg.SIDBProvisioningIntegration
2019-07-25 11:30:30,086 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.racprov.ui.intg.RACProvIntegration
2019-07-25 11:30:30,087 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.ec.ui.intg.ExtendClusterIntegration
2019-07-25 11:30:30,088 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.dn.ui.intg.DltNodeIntegration
2019-07-25 11:30:30,089 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.common.ui.intg.ProvCommonIntegration
2019-07-25 11:30:30,090 [OC4J Launcher] ERROR app.ContextInitializer contextInitialized.420 - Integration Class not found: oracle.sysman.pp.bpelprov.ui.intg.BPELProvisioningIntegration

Not sure where the problem is, found a solution, though might be the stupid solution, but it was working.

Follow the link How can I start Oracle Enterprise manager after restarting the Red Hat Server?
Try to re-configure em (Enterprise Manager) completely.

[tom@ocp dbhome_1]$ emca -config dbcontrol db

STARTED EMCA at Jul 25, 2019 5:32:45 PM
EM Configuration Assistant, Version 11.2.0.0.2 Production
Copyright (c) 2003, 2005, Oracle.  All rights reserved.

Enter the following information:
Database SID: orcl
Database Control is already configured for the database orcl
You have chosen to configure Database Control for managing the database orcl
This will remove the existing configuration and the default settings and perform a fresh configuration
Do you wish to continue? [yes(Y)/no(N)]: Y
Listener ORACLE_HOME [ /u01/app/tom/product/11.2.0/dbhome_1 ]:
Password for SYS user: [oracle]
Password for DBSNMP user: [oracle]
Password for SYSMAN user: [oracle]
Email address for notifications (optional):
Outgoing Mail (SMTP) server for notifications (optional):
-----------------------------------------------------------------

You have specified the following settings

Database ORACLE_HOME ................ /u01/app/tom/product/11.2.0/dbhome_1

Local hostname ................ ocp
Listener ORACLE_HOME ................ /u01/app/tom/product/11.2.0/dbhome_1
Listener port number ................ 1521
Database SID ................ orcl
Email address for notifications ...............
Outgoing Mail (SMTP) server for notifications ...............

-----------------------------------------------------------------
Do you wish to continue? [yes(Y)/no(N)]: Y
Jul 25, 2019 5:33:38 PM oracle.sysman.emcp.EMConfig perform
INFO: This operation is being logged at /u01/app/tom/cfgtoollogs/emca/orcl/emca_2019_07_25_17_32_45.log.
Jul 25, 2019 5:33:38 PM oracle.sysman.emcp.util.DBControlUtil stopOMS
INFO: Stopping Database Control (this may take a while) ...
log4j:ERROR No appenders could be found for category (emSDK.config).
log4j:ERROR Please initialize the log4j system properly.
Jul 25, 2019 5:33:40 PM oracle.sysman.emcp.EMReposConfig uploadConfigDataToRepository
INFO: Uploading configuration data to EM repository (this may take a while) ...
Jul 25, 2019 5:34:26 PM oracle.sysman.emcp.EMReposConfig invoke
INFO: Uploaded configuration data successfully
Jul 25, 2019 5:34:32 PM oracle.sysman.emcp.util.DBControlUtil configureSoftwareLib
INFO: Software library is already configured.
Jul 25, 2019 5:34:32 PM oracle.sysman.emcp.util.DBControlUtil configureSoftwareLib
INFO:  EM_SWLIB_STAGE_LOC (value) will be ignored.
Jul 25, 2019 5:34:32 PM oracle.sysman.emcp.EMDBPostConfig configureSoftwareLibrary
INFO: Deploying Provisioning archives ...
Jul 25, 2019 5:35:09 PM oracle.sysman.emcp.EMDBPostConfig configureSoftwareLibrary
INFO: Provisioning archives deployed successfully.
Jul 25, 2019 5:35:09 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Securing Database Control (this may take a while) ...
Jul 25, 2019 5:35:13 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsole
INFO: Database Control secured successfully.
Jul 25, 2019 5:35:13 PM oracle.sysman.emcp.util.DBControlUtil startOMS
INFO: Starting Database Control (this may take a while) ...
Jul 25, 2019 5:35:38 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: Database Control started successfully
Jul 25, 2019 5:35:38 PM oracle.sysman.emcp.EMDBPostConfig performConfiguration
INFO: >>>>>>>>>>> The Database Control URL is https://ocp:1158/em <<<<<<<<<<<
Jul 25, 2019 5:35:40 PM oracle.sysman.emcp.EMDBPostConfig invoke
WARNING:
************************  WARNING  ************************

Management Repository has been placed in secure mode wherein Enterprise Manager data will be encrypted.  The encryption key has been placed in the file: /u01/app/tom/product/11.2.0/dbhome_1/ocp_orcl/sysman/config/emkey.ora.   Please ensure this file is backed up as the encrypted data will become unusable if this file is lost.

***********************************************************
Enterprise Manager configuration completed successfully
FINISHED EMCA at Jul 25, 2019 5:35:40 PM

After re-configure, the problem was resolved.

[tom@ocp dbhome_1]$ emctl start dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.1.0
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
https://ocp:1158/em/console/aboutApplication
 - An instance of Oracle Enterprise Manager 11g Database Control is already running.

Check the link https://192.168.26.130:1158/em/, it is working properly now.

Reference

How can I start Oracle Enterprise manager after restarting the Red Hat Server?