Eucalyptus
- 官网 http://open.eucalyptus.com/downloads
- 手册 http://open.eucalyptus.com/wiki
- 服务器角色
- Cloud controller(CLC),控制调度云的资源,同时提供web界面以及兼容EC2的SOAP服务
- Walrus,实现了基于桶的存储,同时提供兼容S3的SOAP服务以及REST接口
- Cluster controller(CC),实现以集群为单位的调度
- Storage controller(SC),基于块的存储控制器,EBS-style
- Node controller(NC),虚拟节点,运行xen及虚拟机的机器
版本
- CentOS 6 Minimal install
- Xen 4.1.1 NC服务器上面需要完全安装,其余服务器需要几个编译好的rpm,参考 http://blog.csdn.net/phoenixie/article/details/6885313
- Eucalyptus 2.0.3,下载offline源码包, http://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-offline.tar.gz , 依赖的库 http://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gz
- euca2ools 1.3.1, 使用fedora的,下载 http://eucalyptussoftware.com/downloads/releases/euca2ools-1.3.1-fedora-x86_64.tar.gz
- dhcpd 3.0.5, CC上需要安装, 使用旧版本rhel5的, eucalyptus 2.0.3对稍微高版本的ISC dhcpd支持有问题, 下载http://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/dhcp-3.0.5-29.el5_7.1.src.rpm
准备工作
1. 下载各个软件包
2. 除了NC以外的服务器安装CentOS 6 Minimal方式
3. NC服务器按照参考文档安装配置Xen环境,注意在Xen的内核配置里面添加一项
max_loop=256
4. 配置好所有机器的NTP,使它们时钟保持同步
安装过程
1. 创建 /etc/yum.repos.d/euca.repo[euca]
name=Eucalyptus
baseurl=http://www.eucalyptussoftware.com/downloads/repo/eucalyptus/2.0.3/yum/centos/x86_64
gpgcheck=0
2. 通过yum安装必须的软件包 (所有机器上)
yum -y install perl-Convert-ASN1.noarch scsi-target-utils httpd java-1.6.0-openjdk-devel ant ant-nodeps libvirt-devel curl-devel httpd-devel apr-devel openssl-devel libxml2 libxml2-devel gnutls gnutls-devel libgcrypt-devel zlib-devel perl-Convert-ASN1 perl-Crypt-OpenSSL-RSA perl-Crypt-OpenSSL-Random chkfontpath fuse-libs swig gcc make patch compat-expat1 compat-db43 lzo2 vblade m2crypto lvm2 rsync ntp vconfig groff rpm-build
3. 安装之前编译好的Xen的rpm包(给控制服务器,节点服务器已经装过xen的就不需要了)
rpm -Uvh xen-licenses-*.x86_64.rpm xen-devel-*.x86_64.rpm xen-libs-*.x86_64.rpm
4. 安装dhcpd
wget http://ftp.redhat.com/redhat/linux/enterprise/5Server/en/os/SRPMS/dhcp-3.0.5-29.el5_7.1.src.rpm
rpm -i dhcp-3.0.5-29.el5_7.1.src.rpm
cd ~/rpmbuild/SPECS/
rpmbuild -bb dhcp.spec # 如果编译不过去,可能需要手动去掉-Werror编译选项
cd ~/rpmbuild/RPMS/x86_64/
rpm -Uvh dhcp-3.0.5-29.el6.1.x86_64.rpm # 如果已经装了dhcp,先手动删除掉原来的
5. 安装Eucalyptus的依赖软件(eucalyptus-2.0.3-src-deps.tar.gz)
export EUCALYPTUS=/home/public/eucalyptus-2.0.3
export APACHE_INCLUDES=/usr/include/httpd/
export APR_INCLUDES=/usr/include/apr-1/
export AXIS2C_HOME=${EUCALYPTUS}/packages/axis2c-1.6.0
mkdir -p ${EUCALYPTUS}/packages/
tar xzf eucalyptus-2.0.3-src-deps.tar.gz
cd eucalyptus-src-deps
tar xzf axis2-1.4.tgz -C ${EUCALYPTUS}/packages/
cd ${EUCALYPTUS}/packages/
ln -sf axis2-1.4 axis2
cd -
tar xzf axis2c-src-1.6.0.tar.gz
cd axis2c-src-1.6.0
CFLAGS="-w" ./configure --prefix=${AXIS2C_HOME} -with-apache2=${APACHE_INCLUDES} --with-apr=${APR_INCLUDES} --enable-multi-thread=no
make
make install
cd -
rm -rf axis2c-src-1.6.0
export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib
tar xzf rampartc-src-1.3.0-0euca2.tar.gz
cd rampartc-src-1.3.0
./configure --prefix=${AXIS2C_HOME} --enable-static=no --with-axis2=${AXIS2C_HOME}/include/axis2-1.6.0/
make
make install
rm -rf rampartc-src-1.3.0
cd ${EUCALYPTUS}/packages
ln -sf axis2c-1.6.0 axis2c
cd -
sed -i "s;<\!--phase name=\"Security\"/-->;<phase name=\"Security\"/>;g" ${AXIS2C_HOME}/axis2.xml
cd ..
rm -rf eucalyptus-src-deps
6. 安装Eucalyptus (eucalyptus-2.0.3-src-offline.tar.gz)
export EUCALYPTUS=/home/public/eucalyptus-2.0.3
export AXIS2_HOME=${EUCALYPTUS}/packages/axis2
export AXIS2C_HOME=${EUCALYPTUS}/packages/axis2c
export JAVA_HOME="/usr/lib/jvm/java-openjdk/"
export JAVA="$JAVA_HOME/JRE/bin/java"
tar xzf eucalyptus-2.0.3-src-offline.tar.gz
cd eucalyptus-2.0.3-src-offline
./configure --with-axis2=${AXIS2_HOME} --with-axis2c=${AXIS2C_HOME} --enable-debug --prefix=${EUCALYPTUS}
make
make install
cd -
rm -rf eucalyptus-2.0.3-src-offline
cd /home/public/
ln -sf eucalyptus-2.0.3 eucalyptus
export EUCALYPTUS=/home/public/eucalyptus
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc
ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc
chkconfig eucalyptus-cloud off
chkconfig eucalyptus-cc off
chkconfig eucalyptus-nc off
7. 安装工具(euca2ools-1.3.1-fedora-x86_64.tar.gz)
tar xzf euca2ools-1.3.1-fedora-x86_64.tar.gz
cd euca2ools-1.3.1-fedora-x86_64
rpm -Uvh euca2ools-1.3.1-1.x86_64.rpm
cd -
rm -rf euca2ools-1.3.1-fedora-x86_64
配置
1. 创建一个专属用户adduser eucalyptus
2. 初始化
export EUCALYPTUS=/home/public/eucalyptus
$EUCALYPTUS/usr/sbin/euca_conf \
-d $EUCALYPTUS \
--hypervisor xen \
--instances /home/eucalyptus/instances \
--user eucalyptus \
--setup
# -d: Eucalyptus路径
# --hypervisor: 使用的hypervisor类型,可以是kvm或xen
# --instances: instance存储路径
# --user: 服务使用的用户
# --setup: 初始化
service eucalyptus-nc start
chkconfig eucalyptus-nc on
3. 编辑 /home/public/eucalyptus/etc/eucalyptus/eucalyptus.conf,按自己的需求修改,其中 VNET_BRIDGE 是xen要使用的bridge名字,其余的VNET_MODE等请参考配置文件的说明,里面写的很详细。
4. 复制eucalyptus安装目录到所有机器上,建立init.d的链接,添加service的服务,然后还需要运行一次初始化
5. 启动NC
service eucalyptus-nc start # NC的Web service默认端口8775
chkconfig eucalyptus-nc on
6. 启动CC
service eucalyptus-cc start # CC的Web service默认端口8774
chkconfig eucalyptus-cc on
7. 启动相关服务,不同的服务可以部署在不同机器上面
export EUCALYPTUS=/home/public/eucalyptus
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --enable cloud
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --enable walrus
$EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS --enable sc
service eucalyptus-cloud start # 三个服务都通过这个脚本启动,http端口8080,https端口8443,Web service端口8773
chkconfig eucalyptus-cloud on
8. 注册服务
$EUCALYPTUS/usr/sbin/euca_conf --register-walrus $IP
$EUCALYPTUS/usr/sbin/euca_conf --register-cluster OurCloud $IP # 注册一个Cluster,名字叫OurCloud,默认端口8774
$EUCALYPTUS/usr/sbin/euca_conf --register-sc OurCloud $IP # 给Cluster添加一个SC
9. 注册节点
$EUCALYPTUS/usr/sbin/euca_conf --register-nodes $IP
10. 访问Cloud(CLC)所在IP的https://$IP:8443,用户名admin,密码admin,第一次登陆成功后会要求马上修改密码
11. 在管理界面的导航栏找到Credentials,进入,点击按钮"Download Credentials",会下载一个zip文件
12. 把zip文件传到安装了euca2ools的服务器上,建一个目录解压进去,以后每次使用euca2ools前需要执行一次下面的命令
source eucarc
13. 安装完成
实例1: 使用官网提供的Ubuntu image
1. 下载地址 http://open.eucalyptus.com/wiki/EucalyptusUserImageCreatorGuide 2. 添加 (euca-ubuntu-9.04-x86_64.tar.gz)tar xzf euca-ubuntu-9.04-x86_64.tar.gz
cd euca-ubuntu-9.04-x86_64
# 进入运行环境
source ~/.euca/eucarc
# 上传内核,得到一个eki-XXXXXXXX
euca-bundle-image -d ./tmp/ -i xen-kernel/vmlinuz-2.6.27.21-0.1-xen --kernel true
euca-upload-bundle -b ubuntu9-bucket -m ./tmp/vmlinuz-2.6.27.21-0.1-xen.manifest.xml
euca-register ubuntu9-bucket/vmlinuz-2.6.27.21-0.1-xen.manifest.xml
# 上传ramdisk,得到一个eri-XXXXXXXX
euca-bundle-image -d ./tmp/ -i xen-kernel/initrd-2.6.27.21-0.1-xen --ramdisk true
euca-upload-bundle -b ubuntu9-bucket -m ./tmp/initrd-2.6.27.21-0.1-xen.manifest.xml
euca-register ubuntu9-bucket/initrd-2.6.27.21-0.1-xen.manifest.xml
# 上传根分区,用上面两个命令得到的数字执行
euca-bundle-image -d ./tmp/ -i ubuntu.9-04.x86-64.img --kernel eki-XXXXXXXX --ramdisk eri-XXXXXXXX
euca-upload-bundle -b ubuntu9-bucket -m ./tmp/ubuntu.9-04.x86-64.img.manifest.xml
euca-register ubuntu9-bucket/ubuntu.9-04.x86-64.img.manifest.xml
3. 创建一个登录虚拟机用的key
euca-add-keypair euca-key > ~/euca-key.private
chmod 0600 ~/euca-key.private
4. 使用刚才上传虚拟机时候得到的emi值启动一个虚拟机实例
euca-run-instances -k euca-key -n 1 emi-XXXXXXXX
5. 查看运行状态,没有dhcpd的话,这时拿不到IP
>euca-describe-instances
RESERVATION r-4F030A57 admin default
INSTANCE i-35C505C3 emi-XXXXXXXX 0.0.0.0 0.0.0.0 running euca-key 0 m1.small 2011-11-01T10:48:56.831Z OurCloud eki-XXXXXXXX eri-XXXXXXXX
实例2: 使用dhcpd分配IP
0. CC服务器, IP eth3 192.168.11.3/24; NC服务器, IP eth3 192.168.11.4/24, Xen的bridge xenbr0; 实现从NC上面运行的虚拟机可以通过CC访问到外面的网络1. 配置CC的/home/public/eucalyptus/etc/eucalyptus/eucalyptus.conf
VNET_PUBINTERFACE="eth3"
VNET_PRIVINTERFACE="eth3"
VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_DHCPUSER="eucalyptus"
VNET_MODE="MANAGED-NOVLAN"
VNET_SUBNET="192.168.101.0"
VNET_NETMASK="255.255.255.0"
VNET_DNS="192.168.11.2"
VNET_ADDRSPERNET="16" # 这个数字如果太大会无法创建虚拟机
VNET_PUBLICIPS="192.168.11.80-192.168.11.120"
VNET_CLOUDIP="192.168.11.3"
#VNET_MODE="SYSTEM"
2. 配置NC的/home/public/eucalyptus/etc/eucalyptus/eucalyptus.conf
VNET_PUBINTERFACE="xenbr0"
VNET_PRIVINTERFACE="xenbr0"
VNET_BRIDGE="xenbr0"
VNET_MODE="MANAGED_NOVLAN"
#VNET_MODE="SYSTEM"
3. 重新启动CC
service eucalyptus-cc cleanrestart
4. 创建一个虚拟机看看
>euca-run-instances -k euca-key -n 1 emi-XXXXXXXX
RESERVATION r-502007DA admin admin-default
INSTANCE i-440907AE emi-XXXXXXXX 0.0.0.0 0.0.0.0 pending euca-key 2011-11-04T11:01:17.656Z eki-XXXXXXXX eri-XXXXXXXX
>euca-describe-instances
RESERVATION r-502007DA admin default
INSTANCE i-440907AE emi-XXXXXXXX 192.168.11.101 192.168.101.131 running euca-key 0 m1.small 2011-11-04T11:01:17.656Z OurCloud eki-XXXXXXXX eri-XXXXXXXX
>ssh -i ~/euca-key.private 192.168.11.101
root@ubuntu:~#
>ip addr show eth0
UNKNOWN qlen 1000
link/ether d0:0d:44:09:07:ae brd ff:ff:ff:ff:ff:ff
inet 192.168.101.131/28 brd 192.168.101.143 scope global eth0
inet6 fe80::d20d:44ff:fe09:7ae/64 scope link
valid_lft forever preferred_lft forever
>ping -c 1 192.168.11.2
PING 192.168.11.2 (192.168.11.2) 56(84) bytes of data.
64 bytes from 192.168.11.2: icmp_seq=1 ttl=63 time=1.26 ms
--- 192.168.11.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.264/1.264/1.264/0.000 ms
实例3: 自己建立CentOS6虚拟机
1. 使用virtinst以minimal模式安装一个CentOS6虚拟机,硬盘弄一个分区就行了,其他分区都可省,在虚拟机里面安装配置好需要的软件,关掉虚拟机(安装参数可参考
http://blog.csdn.net/phoenixie/article/details/6885313)
2. 提取虚拟机 / 分区,假设硬盘文件名为sda.img,2GB
> parted sda.img
GNU Parted 2.1
Using /home/vms/centos6/sda.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) U
Unit? [compact]? b
(parted) p
Model: (file)
Disk /home/vms/centos6/sda.img: 2147483648B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 1048576B 2147483647B 2146435072B primary ext3 boot
(parted) quit
> dd if=sda.img of=rootfs.img bs=1M skip=1 count=2047
2047+0 records in
2047+0 records out
2146435072 bytes (2.1 GB) copied, 37.3425 s, 57.5 MB/s
3. 配置根分区文件
# 加载根分区
mkdir /mnt/rootfs
mount -o loop rootfs.img /mnt/rootfs/
# 复制出来kernel和initramfs
cp /mnt/rootfs/boot/vmlinuz-* ./kernel
cp /mnt/rootfs/boot/initramfs-* ./ramdisk
# 配置网卡
rm udev/rules.d/70-persistent-net.rules
vi /mnt/rootfs/etc/udev/rules.d/60-net.rules
ACTION=="add", SUBSYSTEM=="net", IMPORT{program}="/lib/udev/rename_device"
SUBSYSTEM=="net", RUN+="/etc/sysconfig/network-scripts/net.hotplug"
vi /mnt/rootfs/etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet
# 关闭selinux
vi /mnt/rootfs/etc/sysconfig/selinux
SELINUX=disabled
# 修改fstab,去掉所有的UUID方式
vi /mnt/rootfs/etc/fstab
/dev/xvda1 / ext3 defaults,errors=remount-ro 0 0
/dev/xvda2 /mnt ext3 defaults 0 0
/dev/xvda3 swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
vi /mnt/rootfs/etc/mtab
/dev/sda1 / ext3 rw,errors=remount-ro 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0
# 启动时自动下载可登录的ssh-rsa的public key
vi /mnt/rootfs/etc/rc.local
# simple attempt to get the user ssh key using the meta-data service
mkdir -p /root/.ssh
echo >> /root/.ssh/authorized_keys
curl -m 10 -s http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key | grep 'ssh-rsa' >> /root/.ssh/authorized_keys
echo "AUTHORIZED_KEYS:"
echo "************************"
cat /root/.ssh/authorized_keys
echo "************************"
# 删除密码
vi /mnt/rootfs/etc/shadow
root:!:15280:0:99999:7:::
# 卸载
umount /mnt/rootfs
4. 修改NC上面的 /home/public/eucalyptus/usr/share/eucalyptus/gen_libvirt_xml, 把所有的sda改为xvda, 这个修改后,目前官网提供的image将不能使用
5. 最后, 上传
source ~/.euca/eucarc
# 上传内核,得到一个eki-KKKKKKKK
euca-bundle-image -d ./tmp/ -i ./kernel --kernel true
euca-upload-bundle -b centos6-bucket -m ./tmp/kernel.manifest.xml
euca-register centos6-bucket/kernel.manifest.xml
# 上传ramdisk,得到一个eri-RRRRRRRR
euca-bundle-image -d ./tmp/ -i ./ramdisk --ramdisk true
euca-upload-bundle -b centos6-bucket -m ./tmp/ramdisk.manifest.xml
euca-register centos6-bucket/ramdisk.manifest.xml
# 上传根分区
euca-bundle-image -d ./tmp/ -i ./rootfs.img --kernel eki-KKKKKKKK --ramdisk eri-RRRRRRRR
euca-upload-bundle -b centos6-bucket -m ./tmp/rootfs.img.manifest.xml
euca-register centos6-bucket/rootfs.img.manifest.xml
6. 现在可以试试了,需要注意的即将创建的虚拟机的磁盘,不能小于根分区文件的字节数,否则不能建立虚拟机;而且设定的虚拟机的内存,不能小于内核的最低需要;这两个参数可以在管理界面里面调整。