总汇nexus 服务启动异常
故障描述1
nexus oss 3 一直运行得没什么问题,忽然发现运行特别慢然后到服务器去重启,莫名其妙一直启动失败,查看日志发现如下报错信息:
2022-07-05 18:09:50,048+0000 ERROR [FelixStartLevel] *SYSTEM org.sonatype.nexus.extender.NexusContextListener - Failed to start nexus
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/nexus-data/db/component' with mode=rw
DB name="component"
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
at org.sonatype.nexus.orient.DatabaseManagerSupport.connect(DatabaseManagerSupport.java:178)
at org.sonatype.nexus.orient.DatabaseManagerSupport.createInstance(DatabaseManagerSupport.java:312)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at org.sonatype.nexus.orient.DatabaseManagerSupport.instance(DatabaseManagerSupport.java:289)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
at org.sonatype.nexus.orient.restore.RestoreServiceImpl.doStart(RestoreServiceImpl.java:76)
at org.sonatype.nexus.common.stateguard.StateGuardLifecycleSupport.start(StateGuardLifecycleSupport.java:67)
at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:193)
at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
at org.sonatype.nexus.extender.NexusLifecycleManager.startComponent(NexusLifecycleManager.java:199)
at org.sonatype.nexus.extender.NexusLifecycleManager.to(NexusLifecycleManager.java:111)
at org.sonatype.nexus.extender.NexusContextListener.moveToPhase(NexusContextListener.java:304)
at org.sonatype.nexus.extender.NexusContextListener.frameworkEvent(NexusContextListener.java:201)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1429)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.ODiskWriteAheadLog.cutTill(ODiskWriteAheadLog.java:919)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeFullCheckpoint(OAbstractPaginatedStorage.java:3706)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.recoverIfNeeded(OAbstractPaginatedStorage.java:3937)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:288)
... 30 common frames omitted
一开始以为是/nexus-data/db/component 的权限问题,但一直以来启动都没有问题,重新设置 777 权限后仍然没有效果。
解决方法
最后在网上找到了解决方法:把 /nexus-data/db/component 下的出现 0字节 的wal文件全部删掉,重新启动即可!!
故障描述2
因为nexus所在服务器存储空间满了, 做了部分清理操作,结果nexus 启动异常,提示错误信息基本类似如下:
build 285537d2767275f460df32c6a3be01bfff6a517c, branch 2.2.x) com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage '/mnt/sonatype-work/nexus3/db/config' with mode=rw DB name="config" at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
解决方法
基本都是修复数据库,重建索引,但是有一个问题
-
一般处理方法
使用console 工具
java -jar ${nexus-path}/lib/support/nexus-orient-console.jar
连接数据库
connect plocal:${nexus-db-path}/db/component admin admin
修复数据库以及索引
rebuild index *
repair database --fix-links
repair database --fix-graph
repair database --fix-ridbags
repair database --fix-bonsai
-
特殊情况
如果以上方法还是有问题,一般的提示信息如下,参考如下操作:
基本还是包含了rw 的问题
with mode=rw DB name="config" at
删除wal 文件(对应数据库的)
比如处理component 数据库的,一般在
cd ${nexus-db-path}/db/component
rm -rf *.wal
连接数据库,以及进行数据库修复,可以参考上边的方法
connect plocal:${nexus-db-path}/db/component admin admin
rebuild index *
repair database --fix-links
repair database --fix-graph
repair database --fix-ridbags
repair database --fix-bonsai
故障描述3
因为nexus所在服务器文件丢失,结果nexus 启动异常,提示错误信息基本类似如下:
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage 'F:/nexus-3.19.1-01-win64/sonatype-work/nexus3/db/config' with mode=rw
DB name="config"
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:323)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:259)
at org.sonatype.nexus.orient.DatabaseManagerSupport.connect(DatabaseManagerSupport.java:178)
at org.sonatype.nexus.orient.DatabaseManagerSupport.createInstance(DatabaseManagerSupport.java:312)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at org.sonatype.nexus.orient.DatabaseManagerSupport.instance(DatabaseManagerSupport.java:289)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: com.orientechnologies.orient.core.exception.OStorageException: File with name 'quartz_job_detail_5.pcl' does not exist in storage 'config'
DB name="config"
at com.orientechnologies.orient.core.storage.cache.local.OWOWCache.loadFile(OWOWCache.java:475)
at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.openFile(ODurableComponent.java:180)
at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.open(OPaginatedCluster.java:217)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.openClusters(OAbstractPaginatedStorage.java:387)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:290)
解决方法
使用console 工具
java -jar ${nexus-path}/lib/support/nexus-orient-console.jar
连接数据库
connect plocal:${nexus-db-path}/db/component admin admin
执行命令
drop INDEX quartz_job_detail_name_group_idx
连接数据库,以及进行数据库修复,可以参考上边的方法
connect plocal:${nexus-db-path}/db/component admin admin
rebuild index *
repair database --fix-links
repair database --fix-graph
repair database --fix-ridbags
repair database --fix-bonsai
备份准备修复的数据库
1、停止nexus进程
./nexus stop
注意出错的库名,出错的库不见得与笔者一致,具体看出错日志(Nexus的OrientDB有多个库)。
cp -rv /usr/local/nexus/sonatype-work/nexus3/db/component /backup/
2、到Nexus安装目录下/lib/support下,运行nexus-orient-console.jar
cd /usr/local/nexus/nexus-3.21.2-03/lib/support
# 启动console
java -jar nexus-orient-console.jar
# 如果Nexus库已经很大,在执行后面的导出操作时可能会报heap OOM错误,可以在启动时调大内存
java -jar -Xms4g -Xmx4g nexus-orient-console.jar
3、连接出错的数据库,执行导出、备份、删除、重建、导入
**重要**:以下操作有删除动作,注意一定要先做好备份
# 连接出错的数据库(注意出错的库不见得与笔者一致,注意查看出错日志)
orientdb> connect PLOCAL:/usr/local/nexus/sonatype-work/nexus3/db/component admin admin
# 导出
orientdb {db=component}> EXPORT DATABASE /backup/orientdb-component.export
# 关闭连接
orientdb {db=component}> DISCONNECT
# 删除出问题的库
orientdb> DROP DATABASE PLOCAL:/usr/local/nexus/sonatype-work/nexus3/db/component admin admin
# 重新创建出问题的库
orientdb> CREATE DATABASE PLOCAL:/usr/local/nexus/sonatype-work/nexus3/db/component admin admin
# 导入
orientdb {db=component}> IMPORT DATABASE /backup/orientdb-component.export.gz
# 关闭连接
orientdb {db=component}> DISCONNECT
# 退出控制台
orientdb> EXIT
启动nexus进程
./nexus start
参考资料:
https://raeblog.com/posts/380.html
https://blog.csdn.net/u012927695/article/details/120351592
https://www.jianshu.com/p/62b2958ca471
https://blog.csdn.net/u012927695/article/details/120351592
https://blog.csdn.net/weixin_42550597/article/details/113086137
http://t.zoukankan.com/rongfengliang-p-10728632.html
https://blog.csdn.net/mutian6655/article/details/112758278?utm_medium=distribute.pc_relevant.none-task-blog-2defaultbaidujs_title~default-0-112758278-blog-97782720.pc_relevant_multi_platform_whitelistv1&spm=1001.2101.3001.4242.1&utm_relevant_index=3