Greenplum安装
评估存储能力
计算可用的空间
步骤1:初始存储能力=硬盘大小*硬盘数
步骤2:配置RAID10,格式化磁盘空间=(初始存储能力*0.9)/2
步骤3:可用磁盘空间=格式化磁盘空间*0.7
步骤4:用户数据使用空间
使用镜像:(2*用户数据)+用户数据/3=可用磁盘空间
不使用镜像:用户数据+用户数据/3=可用磁盘空间
计算用户数据大小
平均来说,实际占用磁盘空间大小=用户数据*1.4
页面开销:32KB页面需要20 bytes
行开销:每行24 bytes,'append-only'表需要4bytes
索引开销:
B-tree:唯一值*(数据类型大小+24bytes) Bitmap:(唯一值*行数*1bit*压vi缩比率/8)+(唯一值*32)
为元数据和日志计算空间需求
系统元数据:20M
预写日志(WAL):WAL被拆分成多个64M的文件,WAL文件数最多为
2*checkpoint_segments+1,checkpoint_segments默认值为8。也就意味着每个实例需要1088MB的WAL空间
GP数据库日志文件:日志轮转
性能监控数据
实验环境
硬件环境
VMware虚拟机软件10.0
三台Linux虚拟机:Red Hat Enterprise Linux Serverrelease 5.4
2数据库:greenplum-db-4.2.8.0-build-1-RHEL5-x86_64.zip
网卡设置
虚拟机配置
基本信息配置
节点 |
IP |
内存 |
硬盘 |
master |
192.168.80.200 |
1024M |
20G |
slave1 |
192.168.80.201 |
1024M |
20G |
slave2 |
192.168.80.202 |
1024M |
20G |
设置IP地址的命令行方式
vi /etc/sysconfig/network-scripts/ifcfg-eth0 |
填写ip地址、子网掩码、网关、DNS等
DEVICE=eth0 HWADDR=00:0c:29:b7:7d:69 ONBOOT=yes BOOTPROTO=static DNS1=192.168.80.200 IPV6INIT=no USERCTL=no IPADDR=192.168.80.200 NETMASK=255.255.255.0 GATEWAY=192.168.80.1 |
关闭防火墙
[root@localhost ~]# service iptables status 表格:filter Chain INPUT (policy ACCEPT) num target prot opt source destination 1 RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT) num target prot opt source destination 1 RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT) num target prot opt source destination
Chain RH-Firewall-1-INPUT (2 references) num target prot opt source destination 1 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 2 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 255 3 ACCEPT esp -- 0.0.0.0/0 0.0.0.0/0 4 ACCEPT ah -- 0.0.0.0/0 0.0.0.0/0 5 ACCEPT udp -- 0.0.0.0/0 224.0.0.251 udp dpt:5353 6 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:631 7 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:631 8 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 9 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:21 10 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22 11 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:23 12 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:80 13 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:443 14 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
[root@localhost ~]# service iptables stop 清除防火墙规则: [确定] 把 chains 设置为 ACCEPT 策略:filter [确定] 正在卸载 Iiptables 模块: [确定] |
修改主机名和IP映射:
[root@master ~]# vi /etc/sysconfig/network NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=master [root@localhost ~]# vi /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 192.168.80.200 master |
系统参数配置
包括共享内存、网络、用户限制等,修改或添加/etc/sysctl.conf # Kernel sysctl configuration file for Red Hat Linux # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and # sysctl.conf(5) for more details.
# Controls source route verification net.ipv4.conf.default.rp_filter = 1
#for greenplum begin xfs_mount_options = rw,noatime,inode64,allocsize=16m # Controls the maximum shared segment size, in bytes kernel.shmmax = 500000000 # Controls the minimum shared segment size, in bytes kernel.shmmni = 4096 # Controls the maximum number of shared memory segments, in pages kernel.shmall = 4000000000 kernel.sem = 250 512000 100 2048 # Controls the System Request debugging functionality of the kernel kernel.sysrq = 1 # Controls whether core dumps will append the PID to the core filename # Useful for debugging multi-threaded applications kernel.core_uses_pid = 1 # Controls the maximum size of a message, in bytes kernel.msgmnb = 65536
# Controls the default maxmimum size of a mesage queue kernel.msgmax = 65536 # Controls the default minmimum size of a mesage queue kernel.msgmni = 2048 # Controls the use of TCP syncookies net.ipv4.tcp_syncookies = 1 # Controls IP packet forwarding net.ipv4.ip_forward = 0 # Do not accept source routing net.ipv4.conf.default.accept_source_route = 0 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_max_syn_backlog = 4096 net.ipv4.conf.all.arp_filter = 1 net.ipv4.ip_local_port_range = 1025 65535 net.core.netdev_max_backlog = 10000 vm.overcommit_memory = 2 #for greenplum end |
系统设置(所有节点)
配置/etc/security/limits.conf文件
# for greenplum begin * soft nofile 65536 * hard nofile 65536 * sof |