yum 报错 ZLIB_1.2.3.3 not defined in file libz.so.1

这篇记录工作中发现的，库文件被修改导致 yum 无法正常使用的问题排查过程

问题描述

1）执行yum 报错说python2.7.5 结构异常，发现/usr/bin/yum 的解释器被修改过，恢复成/usr/bin/python即可

2）恢复后，发现执行yum依旧报错：

# yum
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   /lib64/libxml2.so.2: symbol gzopen64, version ZLIB_1.2.3.3 not defined in file libz.so.1 with link time reference

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
2.7.5 (default, Oct 30 2018, 23:45:53) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

If you cannot solve this problem yourself, please go to 
the yum faq at:
  http://yum.baseurl.org/wiki/Faq

处理过程

1、关于yum 报错python2.7.5 结构异常，因为yum 和 dnf 其实都是一个用Python写的脚本，所以如果有异常可以先看看/usr/bin/yum 或 /usr/bin/dnf 看看文件是否被修改过，这里通过文件修改时间可以判断这个文件被动过

2、恢复/usr/bin/yum ，另外看看解释器是哪个，是否正常

3、恢复/usr/bin/yum 后，发现第一个报错没了，另外有个新的报错，问题并不简单

4、尝试通过/usr/bin/yum 指定的 Python 解释器去复现一下报错

# more /usr/bin/yum
大概能看到实际就是导入了 yum 等几个模块，尝试用指定的解释器 import 看看
# /usr/bin/python
Python 2.7.5 (default, Oct 30 2018, 23:45:53) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import yum
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 62, in <module>
    import rpmsack
  File "/usr/lib/python2.7/site-packages/yum/rpmsack.py", line 38, in <module>
    import yum.depsolve
  File "/usr/lib/python2.7/site-packages/yum/depsolve.py", line 30, in <module>
    from transactioninfo import TransactionMember
  File "/usr/lib/python2.7/site-packages/yum/transactioninfo.py", line 32, in <module>
    from sqlitesack import YumAvailablePackageSqlite
  File "/usr/lib/python2.7/site-packages/yum/sqlitesack.py", line 26, in <module>
    import yumRepo
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 35, in <module>
    import sqlitecachec
  File "/usr/lib64/python2.7/site-packages/sqlitecachec.py", line 19, in <module>
    import _sqlitecache
ImportError: /lib64/libxml2.so.2: symbol gzopen64, version ZLIB_1.2.3.3 not defined in file libz.so.1 with link time reference

5、这里大致就清楚了，多半是模块或者库文件也被动过，尝试将相同版本正常机器的 yum 模块复制过来，在正常机器import yum ，打印 yum 看看在什么位置

>>> import yum
>>> yum
<module 'yum' from '/usr/lib/python2.7/site-packages/yum/__init__.pyc'>

6、将模块文件导入了以后，仍然没有恢复，这时候排除了模块文件被修改后，考虑软件版本或者相关文件被修改，对比未发现异常

# rpm -qa|grep -i ZLIB
zlib-1.2.7-18.el7.x86_64
# rpm -ql zlib-1.2.7-18.el7.x86_64
/usr/lib64/libz.so.1
/usr/lib64/libz.so.1.2.7
/usr/share/doc/zlib-1.2.7
/usr/share/doc/zlib-1.2.7/ChangeLog
/usr/share/doc/zlib-1.2.7/FAQ
/usr/share/doc/zlib-1.2.7/README

7、考虑库文件被修改，发现上述库报错/lib/libz.so.1 这个库异常

查看共享库依赖
# ldd /lib64/libxml2.so.2
/lib64/libxml2.so.2: /lib/libz.so.1: no version information available (required by /lib64/libxml2.so.2)
/lib64/libxml2.so.2: /lib/libz.so.1: no version information available (required by /lib64/libxml2.so.2)
linux-vdso.so.1 =>  (0x00007ffe5ebfa000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fd741cd7000)
libz.so.1 => /lib/libz.so.1 (0x00007fd741ac1000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007fd74189b000)
libm.so.6 => /lib64/libm.so.6 (0x00007fd741599000)
libc.so.6 => /lib64/libc.so.6 (0x00007fd7411cc000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd742245000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fd740fb0000)

与正常库对比
# ldd /lib64/libxml2.so.2
linux-vdso.so.1 =>  (0x00007ffe7c7d4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002aac91eea000)
libz.so.1 => /lib64/libz.so.1 (0x00002aac920ee000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00002aac92304000)
libm.so.6 => /lib64/libm.so.6 (0x00002aac9252a000)
libc.so.6 => /lib64/libc.so.6 (0x00002aac9282c000)
/lib64/ld-linux-x86-64.so.2 (0x00002aac9195c000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aac92bfa000)

# ldd /lib/libz.so.1
ldd: /lib/libz.so.1: No such file or directory
# ll /lib/libz.so.1
lrwxrwxrwx. 1 root root 7 May 28  2020 /lib/libz.so.1 -> libz.so

8、依据对比，可以发现库文件被指向了32位的库文件目录以及一个错误的库文件，参考正常的服务器配置即可，问题解决

# rm /lib/libz.so.1
# ln -s /lib64/libz.so.1 /lib/libz.so.1

9、回溯，这个就不继续写了，大致就是看看last / history操作记录

总结

1、这次主要是了解了ldd 对动态共享库的信息分析

2、这种动态库修改的不太好发现，需要依据报错，逐步分析出现异常的动态库

3、nm -C -D 库文件路径 可以查看库文件信息，不过没研究内容啥意思