Nexus因磁盘空间占满异常停机orientdb数据损坏修复

小伙伴们说maven私服访问不了，登录服务器查看，发现Nexus服务的进程不存在，遂，启动服务，查看启动日志，发现磁盘无法写入，使用命令查看系统磁盘占用情况，发现磁盘Nexus的数据盘已占用100%

[root@pub_maven ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        40G  8.1G   30G  22% /
/dev/vdb1        99G   99G    0G 100% /data

此时，只能清理部分机器上不需要的文件（如历史日志，删除前需要讲Nexus进程停止）（千万不要手动删除/sonatype-work/目录下的任何文件），或者对磁盘扩容。

当解决磁盘空间占用100%问题后，重新启动Nexus服务，此时仍然报错：

2019-07-29 13:16:37,261+0800 ERROR [ForkJoinPool.commonPool-worker-1]  *SYSTEM org.sonatype.nexus.orient.DatabaseInstanceImpl - Lifecycle operation start failed
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/data/sonatype-work/nexus3/db/component' with mode=rw
        DB name="component"
        at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
        at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
        at org.sonatype.nexus.orient.DatabaseManagerSupport.connect(DatabaseManagerSupport.java:143)
        at org.sonatype.nexus.orient.DatabaseInstanceImpl.doStart(DatabaseInstanceImpl.java:56)
        at org.sonatype.goodies.lifecycle.LifecycleSupport.start(LifecycleSupport.java:104)
        at org.sonatype.goodies.lifecycle.Lifecycles.start(Lifecycles.java:44)
        at org.sonatype.nexus.orient.DatabaseManagerSupport.createInstance(DatabaseManagerSupport.java:252)
        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1688)
        at org.sonatype.nexus.orient.DatabaseManagerSupport.instance(DatabaseManagerSupport.java:231)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
        at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
        at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.NullPointerException: null
        at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
        at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3705)
        at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.recoverIfNeeded(OAbstractPaginatedStorage.java:3936)
        at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:288)
        ... 17 common frames omitted

此时问题是orientdb的数据库文件被损坏了，一般需要使用Nexus自带的console 工具执行修复命令即可：

java -jar ${NexusHomePath}/lib/support/nexus-orient-console.jar

 连接数据库：

CONNECT PLOCAL:/data/sonatype-work/nexus3/db/component admin admin

有时候在连接数据库这一步会报错，错误内容跟最初的错误很类似，但主要信息是这段：

2019-07-29 13:19:54:453 WARNI {db=component} Storage 'component' was not closed properly. Will try to recover from write ahead log...
2019-07-29 13:19:54:464 WARNI {db=component} Record com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OCheckpointEndRecord{lsn=LSN{segment=3176, position=52}} will be skipped during data restore$ANSI{green {db=component}} Exception during storage data restore
java.lang.NullPointerException

意思是数据库未被正确关闭，尝试从历史日志恢复，但是又恢复失败，此时需要进入到数据库的文件目录，删除*.wal文件，然后重新打开console工具，执行修复，文件即可解决

cd /data/sonatype-work/nexus3/db/component

rm -rf *.wal

 执行修复命名（下面是多个命令，需一个一个按顺序执行）： 

REBUILD INDEX *
REPAIR DATABASE --fix-graph
REPAIR DATABASE --fix-links
REPAIR DATABASE --fix-ridbags
REPAIR DATABASE --fix-bonsai
DISCONNECT

如果有多个数据库被损坏，重复上面的步骤，即可，注意修改数据库文件夹的目录。

此时基本解决了所有启动问题，重新启动Nexus服务即可

#####################################分割线#####################################

上面是解决了启动问题，让Nexus可以提供正常服务，但是我们要预防此类事情再次发生，需要在Nexus后台的管理配置页面添加定时器，定时对数据磁盘进行整理