http://blog.brettalton.com/2010/04/28/installing-guest-additions-in-virtualbox-for-an-ubuntu-server-guest/
check the above link...
the key point is to grab essential headers from repository like:
sudo apt-get install build-essential linux-headers-`uname -r`
&
sudo apt-get install xserver-xorg xserver-xorg-core
in case desktop isn't available.