Running Root.Sh On Second Node PRKN-1008 Unable to Load Shared Library

Running Root.Sh On Second Node Fails PRKN-1008 Unable to Load the Shared Library srvmhas10 (Doc ID 342750.1)

In this Document
  Symptoms
  Changes
  Cause
  Solution
     Scalability RAC Community
  References

Applies to:

Oracle Server - Enterprise Edition - Version: 10.2.0.1 and later [Release: 10.2 and later ]
Linux x86-64

Symptoms

On 64bit linux, Starting vipca through root.sh at install time returns following error
PRKH-1010 : Unable to communicate with CRS services.
[PRKH-1000 : Unable to load the SRVM HAS shared library
[PRKN-1008 : Unable to load the shared library "srvmhas10"
or a dependent library, from
LD_LIBRARY_PATH="/opt/oracle/product/10.2.0.1/crs/jdk/jre/lib/i386/client:
/opt/oracle/product/10.2.0.1/crs/jdk/jre/lib/i386:/opt/oracle/product/10.2.0.
1/crs/jdk/jre/../lib/i386:/opt/oracle/product/10.2.0.1/crs/lib32:
/opt/oracle/product/10.2.0.1/crs/srvm/lib32:/opt/oracle/product/10.2.0.
1/crs/lib:/opt/oracle/product/10.2.0.1/crs/srvm/lib:/opt/oracle/product/10.2.0.1/crs/lib"

Changes

New install

Cause

Missing library "libclntsh.so.10.1" under $ORA_CRS_HOME/lib32

Bug 4189481 Vipca/Srvctl: Prkn-1008 : Unable To Load The Shared Library "Libsrvmhas.So"

This is often caused by missing 32bit rpm for glibc-devel-2*.
Perform rpm -qa against glibc rpm, it should return two lines for each rpm which one stands for 64bit and one stands for 32bit library:

$ rpm -qa |grep glibc-2
glibc-2.3.4-2.13
compat-glibc-2.3.2-95.30
compat-glibc-2.3.2-95.30
glibc-2.3.4-2.13
$ rpm -qa |grep glibc-devel
glibc-devel-2.3.4-2.13
glibc-devel-2.3.4-2.13

If you have only 1 line output, then often it is the 32bit library missing. During relink of generating libclntsh.so, it reports:
error: can not find crti.o
This file is part of glibc-devel-* 32bit rpm.

Solution

Confirm that libclntsh.so.10.1 under under $CRS_HOME/lib32 is missing.
Check make log under $CRS_HOME/install for the error during relink.

Make sure that rpm glibc-2*.i686.rpm and glibc-devel-2*.i386.rpm are installed.

ie) rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n"|grep glibc-devel

Then
Either : Reinstall CRS.
Or : Try the following:

1. Login as root and shutdown crs on the node having problem.
      # crsctl stop crs
2. Backup files under $ORA_CRS_HOME/lib and $ORA_CRS_HOME/lib32
3. Log in as root user.
4. Set ORACLE_HOME to $ORA_CRS_HOME ie:
      # export ORACLE_HOME=/<full path of $ORA_CRS_HOME>
5. # cd $ORACLE_HOME/network/lib
6. # make -f ins_net_client.mk client_sharedlib
7. For files under the $ORA_CRS_HOME/lib32 directory, Change owner, group, permission on files generated by step 6 to oracle:oinstall, 755 (check files generated by checking timestamp of files).
8. Check if symbolic link libclntsh.so pointing to $CRS_HOME/lib/libclntsh.so.10.1 exists under $ORA_CRS_HOME/lib as before. If not, create one.
9. Startup crs as root:
   # crsctl start crs
10. rerun vipca if needed

------------------------------------------------
In some ct's case, error on nnfgt.o shows also.
-----------------------------------------------
$CRS_HOME/install/make.log:
Building 32bit version of nnfgt.o
In file included from /usr/include/features.h:352,
from /usr/include/sys/types.h:27,
from nnfgt.c:7:
/usr/include/gnu/stubs.h:7:27: error: gnu/stubs-32.h: No such file or
directory
mv: cannot stat `nnfgt.o': No such file or directory
/usr/bin/ar: /u01/app/oracle/product/10.2.0/clusterware/lib32/nnfgt.o: No
such file or directory

--------------------------------------------------------

Then, you may perform following steps BEFORE running
make -f ins_net_client.mk client_sharedlib again (that's from make.log):

gcc -m32 -c nnfgt.c
mv nnfgt.o $CRS_HOME/lib32/
/usr/bin/ar rv $CRS_HOME/lib32/libn10.a $CRS_HOME/lib32/nnfgt.o

Scalability RAC Community

To discuss this topic further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Scalability RAC Community.