How to install Nvidia driver on a Xen kernel, CentOS 5.5(Final)

We have already discussed the installation of Nvidia driver on CentOS. But this method only applies to regular kernels. Xen, the virtual machine monitor, requires you to install a modified linux kernel and guess what, nVidia officially doesn't support it. The regular installation is aborted when this kernel is detected and working without the driver is not possible because the performance degradation is unbearable(at least on my system). So, after some searching and trying out different methods, I finally found this work around to make nvidia drivers work on Xen kernels.

Lets start by updating the system. Run the following commands.

[root]# yum -y update

[root]# yum clean all

[root]# reboot

Now, lets keep only the latest kernels and remove all the old ones. In order to see what kernel versions are already installed, execute the following command.

[root]# rpm -qa | grep kernel

The output should look something like this

kernel-headers-2.6.18-194.11.3.el5
kernel-2.6.18-194.11.3.el5
kernel-xen-2.6.18-194.11.3.el5
kernel-devel-2.6.18-194.11.3.el5
kernel-xen-devel-2.6.18-194.11.3.el5

This is the output on my system, all updated. Before starting its better to install the latest kernel-xen & kernel-xen-devel also kernel & kernel-devel

Just run the following commands to install the latest kernels and gcc.

[root]# yum install kernel kernel-devel kernel-xen kernel-xen-devel gcc

Its better to keep the latest versions. If you want to remove any older ones, then you can use the following command to remove them

[root]# rpm -e kernel-xen-2.6.18-164.el5 kernel-xen-devel-2.6.18-164.el5

The installation method I have used and mentioned here seems to work with this version. NVIDIA-Linux-x86_64-256.53.run. If you don't have the driver package yet, then you may use this link on nvidia's website to download the appropriate driver for your card. You may use the instructions given on the previous article in case you are wondering how to proceed.

Now, before starting make sure that you are running the xen kernel. Run the following command to check it.

[shredder12]$ uname -r
2.6.18-194.11.3.el5xen

If this is the case then move on, otherwise shutdown and boot into the xen kernel(select the xen kernel in the grub menu)

Follow the steps below

• First of all we will need the drivers, use the steps mentioned in the previous article to download the driver from Nvidia's website.
• Now, move the driver file to the /tmp directory

  [root]# mv NVIDIA-Linux-x86_64-256.53.run /tmp

• Now, make the file executable by changing the permissions

  [root]# chmod a+x NVIDIA-Linux-x86_64-256.53.run

• Run the following command to just extract the bundle

  [root]# ./NVIDIA-Linux-x86_64-256.53.run --extract-only

• Now, based upon the driver file, you may either have to move to [extracted-dir]/usr/src/nv folder or [extracted-dir]/kernel. Both don't exist in the same driver file, so there shouldn't be a conflict. In my case it was kernel dir.

  [root]# cd NVIDIA-Linux-x86_64-256.53.run/kernel/

• Run the following command

  IGNORE_XEN_PRESENCE=y make SYSSRC=/lib/modules/`uname -r`/build module

• You may get a lot of warnings and error after this command but at the end you should see

  NVIDIA: left KBUILD

• Its time to move the built kernel module in to Xen-kernel library. First we will create the directory for nvidia drivers and then will copy the nvidia.ko file into that folder. The following couple of commands should do so.

  [root]# mkdir -p /lib/modules/`uname -r`/kernel/drivers/video/nvidia/

  [root]# cp -i nvidia.ko /lib/modules/`uname -r`/kernel/drivers/video/nvidia/

  Now, we will build the .dep module files using depmod and finally load the module into the kernel using modprobe

  [root]# depmod -a

  [root]# modprobe nvidia

So, we have finally loaded the nvidia module into the running Xen kernel, but things still won't work because we haven't installed most of the nvidia software yet. Now, the next step is to install the driver software fully on the non-Xen linux kernel. For that, you can follow the small guide on basic installation of Nvidia driver on CentOS (non-Xen kernel).

Once you are done with the installation, finally boot into the xen kernel and you should see the nvidia splash screen. In order to check if the driver module has been uploaded, you may run the following command

[root]# dmesg | grep -i nvidia

You should see the following output

nvidia: module license 'NVIDIA' taints kernel.
NVRM: loading NVIDIA UNIX x86_64 Kernel Module  256.53  Fri Oct 01 20:27:48 IST 2010

Now, you may even check the driver's performance using glxgears

[root]# glxgears

You should see the frame output, which you can use to compare the performance on non-xen kernels. On my system the output was around 15000 frames in 5 seconds. Apparently, the driver installed on Ubuntu(through the default way) gives around 11000 frames every 5 second. I wonder why is that so .

P.S. Just like regular kernels, you will have to perform the same thing everytime a new xen kernel update comes in. I would be happy to know some easy way to handle such issues. If you know any other way, do suggest :).