openstack 重启mysql_突然断电导致mariadb数据库无法启动（openstack 命令无法使用）...

openstack是通过rdo openstack-allinone一键部署的单机模式。

因为突然断掉导致在物理机启动后无法挂载/srv/loopback-device/swiftloopback设备，该设备已经丢失，目前没找到解决办法

只能先注释：

[root@xc-jdsq ~(keystone_admin)]# tail -1 /etc/fstab

#/srv/loopback-device/swiftloopback     /srv/node/swiftloopback ext4    noatime,nodiratime,nobarrier,loop,user_xattr    0       0

启动服务器后产看实例报错：

[root@xc-jdsq ~]# . keystonerc_admin

[root@xc-jdsq ~(keystone_admin)]# nova list

0000   ERROR (InternalServerError): Internal Server Error (HTTP 500)

[root@xc-jdsq ~(keystone_admin)]# openstack server list

Internal Server Error (HTTP 500)

但是实例在正常运行：

[root@xc-jdsq ~(keystone_admin)]# virsh list

Id    Name                           State

----------------------------------------------------

1     instance-0000001e              running

3     instance-0000001a              running

4     instance-0000001b              running

5     instance-00000019              running

看一下nova-api的日志：

[root@xc-jdsq ~(keystone_admin)]# tailf /var/log/nova/nova-api.log

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler     return self.dbapi.connect(*cargs, **cparams)

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler   File "/usr/lib/python2.7/site-packages/pymysql/__init__.py", line 94, in Connect

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler     return Connection(*args, **kwargs)

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 327, in __init__

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler     self.connect()

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler   File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 629, in connect

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler     raise exc

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQLserver on '1                                                    92.168.1.99' ([Errno 111] ECONNREFUSED)") (Background on this error at: http://sqlalche.me/e/e3q8)

2020-06-16 09:07:13.851 6460 ERROR nova.api.metadata.handler

好像是数据库的问题：

[root@xc-jdsq ~(keystone_admin)]# journalctl -xe

Jun 16 09:28:56 xc-jdsq systemd[1]: Unit mariadb.service entered failed state.

Jun 16 09:28:56 xc-jdsq systemd[1]: mariadb.service failed.

ESCOC

l/mysql.sock exists.

lib/mysql/mysql.sock, which means it is a garbage, so it will be removed automatically.

bably initialized in /var/lib/mysql already, nothing is done.

, make sure the /var/lib/mysql is empty before running mysql-prepare-db-dir.

n 'open_files_limit': unsigned value 18446744073709551615 adjusted to 4294967295

exec/mysqld (mysqld 10.3.10-MariaDB) starting as process 7401 ...

not increase number of max_open_files to more than 1024 (request: 8127)

ed limits: max_open_files: 1024  max_connections: 594 (was 4096)  table_cache: 200 (was 2000)

rsync SST method requires wsrep_cluster_address to be configured on startup.

ode=killed, status=6/ABRT

erver.

te.

ESCOD

Jun 16 09:28:51 xc-jdsq container-server[2035]: 0 successes, 1 failures

Jun 16 09:28:51 xc-jdsq container-server[2035]: diff:0 diff_capped:0 empty:0 hashmatch:0 no_change:0 remote_merge:0 rsync:0 ts_repl:0

Jun 16 09:28:52 xc-jdsq swift[2192]: Account HEAD returning 503 for [] (txn: tx65f72d5953c24804b99a7-005ee82054)

Jun 16 09:28:52 xc-jdsq object-expirer[2192]: Unhandled exception: #012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/s

Jun 16 09:28:54 xc-jdsq object-server[1890]: Starting object reconstruction pass.

Jun 16 09:28:54 xc-jdsq object-server[1890]: Nothing reconstructed for 0.000771045684814 seconds.

Jun 16 09:28:54 xc-jdsq object-server[1890]: Object reconstruction complete. (0.00 minutes)

Jun 16 09:28:55 xc-jdsq systemd[1]: mariadb.service holdoff time over, scheduling restart.

Jun 16 09:28:55 xc-jdsq systemd[1]: Stopped MariaDB 10.3 database server.

-- Subject: Unit mariadb.service has finished shutting down

-- Defined-By: systemd

-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

--

-- Unit mariadb.service has finished shutting down.

Jun 16 09:28:55 xc-jdsq systemd[1]: Starting MariaDB 10.3 database server...

-- Subject: Unit mariadb.service has begun start-up

-- Defined-By: systemd

-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

--

-- Unit mariadb.service has begun starting up.

Jun 16 09:28:55 xc-jdsq mysql-check-socket[7327]: Socket file /var/lib/mysql/mysql.sock exists.

Jun 16 09:28:55 xc-jdsq mysql-check-socket[7327]: No process is using /var/lib/mysql/mysql.sock, which means it is a garbage, so it will be removed aut

Jun 16 09:28:55 xc-jdsq mysql-prepare-db-dir[7356]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done.

Jun 16 09:28:55 xc-jdsq mysql-prepare-db-dir[7356]: If this is not the case, make sure the /var/lib/mysql is empty before running mysql-prepare-db-dir.

Jun 16 09:28:55 xc-jdsq mysq