kylin v10部署 ceph 14.2.22问题记录

yum install gcc-c++ ncurses-devel bison  python-sphinx

https://blog.csdn.net/qq_40085317/article/details/106932037

no module but module is there
https://stackoverflow.com/questions/45618286/python-prettytable-no-module-but-module-is-there

k-c6,M}cJ4

参考文档

ceph部署参考文档
https://blog.csdn.net/myqgmyqg/article/details/124561213

部署缺乏相应包报错参考文档
https://blog.csdn.net/bandaoyu/article/details/121414278

首先下载 

ceph 14.2.22
ceph-ansible stable-4.0

安装 python-dev-tool

pip install python-dev-tools

4.Could NOT find verbs (missing: VERBS_LIBRARIES VERBS_INCLUDE_DIR)

解决办法：

yum install rdma-core-devel

5.Could NOT find udev (missing: UDEV_LIBRARIES UDEV_INCLUDE_DIR)

解决办法：

yum install systemd-devel

7.Could NOT find keyutils (missing: KEYUTILS_LIBRARIES KEYUTILS_INCLUDE_DIR)

yum install keyutils-libs-devel

8.Could NOT find OpenLdap (missing: OPENLDAP_INCLUDE_DIR LDAP_LIBRARY

yum install openldap-devel

 Please install libaio, or enable WITH_SPDK or WITH_PMEM (experimental)

yum install libaio-devel

9.Could NOT find leveldb (missing: LEVELDB_LIBRARIES LEVELDB_INCLUDE_DIR)

yum install leveldb-devel

10.Could NOT find snappy (missing: SNAPPY_LIBRARIES SNAPPY_INCLUDE_DIR)

yum install snappy-devel

Could NOT find NSS (missing: NSS_INCLUDE_DIRS)
yum install nss-devel

11.Could NOT find LZ4 (missing: LZ4_LIBRARY LZ4_INCLUDE_DIR) (Required is at
  least version "1.7")

yum install lz4-devel

12.Could NOT find CURL (missing: CURL_LIBRARY CURL_INCLUDE_DIR)

yum install curl-devel

yum install nss-util-devel nspr-devel
yum install gcc-c++ ncurses-devel bison

Could not find cython3: /usr/bin/python3: No module named cython
这个包通过pip下载
python3 -m pip install cython

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
/bin/sh: line 1: /root/ceph-14.2.22/build/src/pybind/mgr/dashboard/node-env/bin/pip: No such file or directory
make[2]: *** [src/pybind/mgr/dashboard/CMakeFiles/mgr-dashboard-nodeenv.dir/build.make:74: src/pybind/mgr/dashboard/node-env/bin/npm] Error 127
make[1]: *** [CMakeFiles/Makefile2:5883: src/pybind/mgr/dashboard/CMakeFiles/mgr-dashboard-nodeenv.dir/all] Error 2

yum install npm

安装依赖

No matching package to install: 'lua-devel'
Package make-1:4.2.1-15.oe1.aarch64 is already installed.
No matching package to install: 'nasm'
Package perl-4:5.28.0-434.oe1.aarch64 is already installed.
Package pkgconf-1.6.3-6.oe1.aarch64 is already installed.
Package systemd-243-18.oe1.aarch64 is already installed.
Package procps-ng-3.3.16-11.oe1.aarch64 is already installed.
Package python3-3.7.4-8.oe1.aarch64 is already installed.
No matching package to install: 'python3-Cython'
No matching package to install: 'python3-bcrypt'
Package python3-dateutil-1:2.7.0-7.oe1.noarch is already installed.
Package python3-devel-3.7.4-8.oe1.aarch64 is already installed.
No matching package to install: 'python3-pecan'
Package python3-setuptools-40.4.3-4.oe1.noarch is already installed.
No matching package to install: 're2-devel'
No matching package to install: 'snappy-devel'
No matching package to install: 'socat'
Package sudo-1.8.27-4.oe1.aarch64 is already installed.
No matching package to install: 'thrift-devel >= 0.13.0'
Package which-2.21-14.oe1.aarch64 is already installed.
No matching package to install: 'xmlstarlet'
No matching package to install: 'yaml-cpp-devel >= 0.6'
Not all dependencies satisfied

yum install nasm python3-Cython python3-bcrypt python3-pecan re2-devel snappy-devel socat xmlstarlet yaml-cpp-devel

[root@host-10-10-17-63 oath-toolkit]# rpmbuild -bb oath-toolkit.spec
error: Failed build dependencies:
    libtool is needed by oath-toolkit-2.6.5-2.aarch64
    pam-devel is needed by oath-toolkit-2.6.5-2.aarch64
[root@host-10-10-17-63 oath-toolkit]# mkdir -p /home/oath
[root@host-10-10-17-63 oath-toolkit]# mv /home/rpmbuild/RPMS/* /home/oath/
mv: cannot stat '/home/rpmbuild/RPMS/*': No such file or directory
[root@host-10-10-17-63 oath-toolkit]# yum install libtool pam-devel

[root@host-10-10-17-63 oath]# createrepo .
-bash: createrepo: command not found
[root@host-10-10-17-63 oath]# yum install createrepo

解决make:sphinx-build: not found
https://blog.csdn.net/ziyoudebailu/article/details/113739311

do_make.sh 过程中可能遇到很多依赖包未安装的问题
可以参考博客
https://blog.csdn.net/helloanthea/article/details/103728684

yum install libaio-devel openssl-devel expat-devel liboath-devel lttng-ust-devel libbabeltrace-devel python36-Cython fuse-devel libnl3-devel librabbitmq-devel libcap-ng-devel gperf librabbitmq-devel librdkafka-devel

yum install libaio-devel openssl-devel expat-devel liboath-devel lttng-ust-devel libbabeltrace-devel python3-Cython fuse-devel libnl3-devel librabbitmq-devel libcap-ng-devel gperf librabbitmq-devel

linux 解决找不到python库无法找到的问题Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS)

yum install python3-devel

pip3 up
 "detail": [
            {
                "message": "Module 'dashboard' has failed dependency: No module named 'cherrypy'"
            },
            {
                "message": "Module 'restful' has failed dependency: No module named 'werkzeug'"
            }

 "detail": [
            {
                "message": "Module 'crash' has failed dependency: No module named 'prettytable'"
            },
            {
                "message": "Module 'dashboard' has failed dependency: No module named 'OpenSSL'"
            },
            {
                "message": "Module 'restful' has failed dependency: No module named 'pecan'"
            },
            {
                "message": "Module 'status' has failed dependency: No module named 'prettytable'"
            }

pip3 install prettytable
pip3 install pecan

 =============================DEBUG ASSISTANCE=============================
        If you are seeing a compilation error please try the following steps to
        successfully install cryptography:
        1) Upgrade to the latest pip and try again. This will fix errors for most
           users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
        2) Read https://cryptography.io/en/latest/installation/ for specific
           instructions for your platform.
        3) Check our frequently asked questions for more information:
           https://cryptography.io/en/latest/faq/
        4) Ensure you have a recent Rust toolchain installed:
           https://cryptography.io/en/latest/installation/#rust
    
        Python: 3.7.4
        platform: Linux-4.19.90-17.ky10.aarch64-aarch64-with-glibc2.17
        pip: 18.0
        setuptools: 63.1.0
        setuptools_rust: 1.3.0
        =============================DEBUG ASSISTANCE=============================
    
    error: Rust 1.29.1 does not match extension requirement >=1.41.0
    
    ----------------------------------------

pip3 install --upgrade pip

pip install pyOpenSSL

 yum install python-openssl

 "summary": {
            "message": "Module 'dashboard' has failed dependency: No module named 'jwt'"
        },
        "detail": [
            {
                "message": "Module 'dashboard' has failed dependency: No module named 'jwt'"
            }
        ]

"MGR_MODULE_DEPENDENCY": {
        "severity": "HEALTH_WARN",
        "summary": {
            "message": "Module 'dashboard' has failed dependency: No module named 'bcrypt'"
        },
        "detail": [
            {
                "message": "Module 'dashboard' has failed dependency: No module named 'bcrypt'"
            }
        ]
    },

cmd:
  - ceph
  - --version
  delta: '0:00:00.058951'
  end: '2022-07-12 18:17:39.682224'
  msg: non-zero return code
  rc: 1
  start: '2022-07-12 18:17:39.623273'
  stderr: |-
    Traceback (most recent call last):
      File "/usr/local/bin/ceph", line 140, in <module>
        import rados
    ImportError: librados.so.2: cannot open shared object file: No such file or directory
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

yum install librados2

stderr: |-
    Traceback (most recent call last):
      File "/usr/local/bin/ceph", line 140, in <module>
        import rados
    ImportError: /usr/local/lib64/python3.7/site-packages/rados.cpython-37m-aarch64-linux-gnu.so: undefined symbol: rados_getaddrs
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

'chown failed: failed to look up user ceph'

ceph-volume --cluster ceph lvm batch --bluestore /dev/dm-0 --report --format=json
  - --cluster
  - ceph
  - lvm
  - batch
  - --bluestore
  - /dev/dm-0
  - --report
  - --format=json

  TASK [ceph-mon : start the monitor service] ***************************************************************************************************
Wednesday 13 July 2022  09:16:28 +0800 (0:00:00.020)       0:00:31.680 ******** 
fatal: [ceph1]: FAILED! => changed=false 
  msg: 'Could not find the request
  

 journalctl -u ceph-mgr@ceph1
-- Logs begin at Mon 2022-07-11 18:36:55 CST, end at Thu 2022-07-14 18:00:03 CST. --
7月 14 17:39:31 ceph1 systemd[1]: Started Ceph cluster manager daemon.
7月 14 17:39:31 ceph1 ceph-mgr[366951]: Could not find platform independent libraries <prefix>
7月 14 17:39:31 ceph1 ceph-mgr[366951]: Could not find platform dependent libraries <exec_prefix>
7月 14 17:39:31 ceph1 ceph-mgr[366951]: Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
7月 14 17:39:31 ceph1 ceph-mgr[366951]: Fatal Python error: initfsencoding: Unable to get the locale encoding
7月 14 17:39:31 ceph1 ceph-mgr[366951]: ModuleNotFoundError: No module named 'encodings'
7月 14 17:39:31 ceph1 ceph-mgr[366951]: Current thread 0x0000ffff58480010 (most recent call first):
7月 14 17:39:31 ceph1 ceph-mgr[366951]: *** Caught signal (Aborted) **
7月 14 17:39:31 ceph1 ceph-mgr[366951]:  in thread ffff58480010 thread_name:ceph-mgr

 "message": "Module 'volumes' has failed: Not found or unloadable"

制作rpm包
https://blog.csdn.net/qq_29493353/article/details/90205415

https://blog.csdn.net/ygtlovezf/article/details/81333956

部署方案
https://blog.csdn.net/qq_29493353/article/details/90205415