Cassandra系列-3-集群搭建

最新推荐文章于 2024-07-02 00:59:54 发布

cuian7084

最新推荐文章于 2024-07-02 00:59:54 发布

阅读量220

点赞数

文章标签： java 网络 runtime

5. 搭建集群
5.1 创建用户,目录
--logon as root
mkdir -p /nosql/cassandra

groupadd nosqlg
useradd nosql -g nosqlg

chown -R nosql:nosqlg /nosql/cassandra
chmod -R 755 /nosql/cassandra

passwd nosql

5.2 复制JRE,复制Cassandra
ftp jre,cassandra软件到/nosql/cassandra/目录下:
[root@bocnet95 cassandra]# tar -zxf jre-7u4-linux-i586[1].tar.gz
[root@bocnet95 cassandra]# tar -zxf apache-cassandra-1.0.10-bin.tar.gz

chown -R nosql:nosqlg /nosql/cassandra
chmod -R 755 /nosql/cassandra

5.3 配置用户.bash_profile
vi .bash_profile
-----------add following lines
JAVA_HOME=/nosql/cassandra/jre1.7.0_04
export JAVA_HOME

PATH=$JAVA_HOME/bin:$PATH:$HOME/bin
export PATH
------------

. ./.bash_profile
[nosql@bocnet95 ~]$ java -version
java version "1.7.0_04"
Java(TM) SE Runtime Environment (build 1.7.0_04-b20)
Java HotSpot(TM) Client VM (build 23.0-b21, mixed mode)

5.4 配置Cassandra
5.4.1配置cassandra.yaml
[nosql@rhel5 conf]$vi cassandra.yaml
----------------------------------------------------------
# directories where Cassandra should store data on disk.
data_file_directories:
- /nosql/cassandra/data

# commit log
commitlog_directory: /nosql/cassandra/commitlog

# saved caches
saved_caches_directory: /nosql/cassandra/saved_caches

cluster_name: 'MyCluster'

# TCP port, for commands and data
storage_port: 7000

# port for Thrift to listen for clients on
rpc_port: 9160

seeds: "22.11.97.96,22.11.97.96"

listen_address: 22.11.97.96
rpc_address: 22.11.97.96
----------------------------------------------------------

同时修改节点1上cassandra.yaml
seeds: "22.11.97.96,22.11.97.95"
listen_address: 22.11.97.95
rpc_address: 22.11.97.95

5.4.2配置log4j-server.properties
[nosql@rhel5 conf]$ vi log4j-server.properties
----------------------------------------------------------
# Edit the next line to point to your logs directory
log4j.appender.R.File=/nosql/cassandra/system.log
----------------------------------------------------------

5.4.3配置cassandra-env.sh
[nosql@rhel5 conf]$ vi cassandra-env.sh
MAX_HEAP_SIZE根据系统物理内存设置;HEAP_NEWSIZE设置为MAX_HEAP_SIZE/4;这两个参数会自动设置,可以去掉注释指定具体值.
----------------------------------------------------------
MAX_HEAP_SIZE="400M"
HEAP_NEWSIZE="100M"
----------------------------------------------------------

5.5 启动Cassandra新增节点
在集群内已经有节点启动的情况下，启动：

[nosql@rhel5 bin]$ ./cassandra
本节点system.log:
INFO [GossipStage:1] 2015-06-29 09:59:33,869 Gossiper.java (line 838) Node /22.11.97.96 is now part of the cluster
INFO [GossipStage:1] 2015-06-29 09:59:33,870 Gossiper.java (line 804) InetAddress /22.11.97.96 is now UP
...
INFO [FlushWriter:1] 2015-06-29 09:59:34,124 Memtable.java (line 283) Completed flushing /nosql/cassandra/data/system/LocationInfo-hd-13-Data.db (89 bytes)
INFO [CompactionExecutor:4] 2015-06-29 09:59:34,225 CompactionTask.java (line 220) Compacted to [/nosql/cassandra/data/system/LocationInfo-hd-14-Data.db,]. 749 to 343 (~45% of original) bytes for 3 keys at 0.000997MB/s. Time: 328ms.
INFO [MigrationStage:1] 2015-06-29 09:59:35,229 Migration.java (line 119) Applying migration e2998350-e468-11e3-0000-242d50cf1fff Add keyspace: test, rep strategy:NetworkTopologyStrategy{}, durable_writes: true
INFO [MigrationStage:1] 2015-06-29 09:59:35,231 ColumnFamilyStore.java (line 705) Enqueuing flush of Memtable-Migrations@15306383(6899/8623 serialized/live bytes, 1 ops)
...
INFO [MigrationStage:1] 2015-06-29 09:59:35,509 Migration.java (line 119) Applying migration 060ae350-e46a-11e3-0000-242d50cf1fff Add column family: org.apache.cassandra.config.CFMetaData@10ab6fa[cfId=1000,ksName=test,cfName=sessioninfo,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@72fc8c,mergeShardsChance=0.1,keyAlias=,column_metadata={},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={},bloomFilterFpChance=]
...
INFO [MigrationStage:1] 2015-06-29 09:59:35,776 Migration.java (line 119) Applying migration 37900280-e478-11e3-0000-242d50cf1fff Add column family: org.apache.cassandra.config.CFMetaData@1e2a2c0[cfId=1001,ksName=test,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@2b2bd1,mergeShardsChance=0.1,keyAlias=,column_metadata={java.nio.HeapByteBuffer[pos=0 lim=10 cap=10]=ColumnDefinition{name=62697274685f64617465, validator=org.apache.cassandra.db.marshal.LongType, index_type=KEYS, index_name='users_birth_date_idx'}, java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]=ColumnDefinition{name=66756c6c5f6e616d65, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=null, index_name='null'}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={},bloomFilterFpChance=]
...
INFO [MigrationStage:1] 2015-06-29 09:59:36,043 SecondaryIndexManager.java (line 183) Creating new index : ColumnDefinition{name=62697274685f64617465, validator=org.apache.cassandra.db.marshal.LongType, index_type=KEYS, index_name='users_birth_date_idx'}
INFO [Creating index: users.users_birth_date_idx] 2015-06-29 09:59:36,060 SecondaryIndex.java (line 150) Submitting index build of users.users_birth_date_idx for data in
INFO [MigrationStage:1] 2015-06-29 09:59:36,064 Migration.java (line 119) Applying migration 23e610b0-e47a-11e3-0000-242d50cf1fff Update column family to org.apache.cassandra.config.CFMetaData@1b83ca2[cfId=1001,ksName=test,cfName=users,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=200000.0,readRepairChance=1.0,replicateOnWrite=true,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=14400,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@94e6ee,mergeShardsChance=0.1,keyAlias=java.nio.HeapByteBuffer[pos=0 lim=3 cap=3],column_metadata={java.nio.HeapByteBuffer[pos=0 lim=10 cap=10]=ColumnDefinition{name=62697274685f64617465, validator=org.apache.cassandra.db.marshal.LongType, index_type=KEYS, index_name='users_birth_date_idx'}, java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]=ColumnDefinition{name=66756c6c5f6e616d65, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=null, index_name='null'}, java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=7374617465, validator=org.apache.cassandra.db.marshal.UTF8Type, index_type=KEYS, index_name='users_state_idx'}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={},bloomFilterFpChance=]

启动比较早的节点system.log:
INFO [GossipStage:1] 2014-05-27 10:15:35,336 Gossiper.java (line 838) Node /22.11.97.95 is now part of the cluster
INFO [GossipStage:1] 2014-05-27 10:15:35,336 Gossiper.java (line 804) InetAddress /22.11.97.95 is now UP
INFO [GossipStage:1] 2014-05-27 10:15:35,349 ColumnFamilyStore.java (line 705) Enqueuing flush of Memtable-LocationInfo@14929107(35/43 serialized/live bytes, 1 ops)
INFO [FlushWriter:2] 2014-05-27 10:15:35,350 Memtable.java (line 246) Writing Memtable-LocationInfo@14929107(35/43 serialized/live bytes, 1 ops)
INFO [FlushWriter:2] 2014-05-27 10:15:35,359 Memtable.java (line 283) Completed flushing /nosql/cassandra/data/system/LocationInfo-hd-30-Data.db (89 bytes)

[nosql@rhel5 bin]$ ./nodetool -h rhel5 ring
Address         DC          Rack        Status State   Load            Owns    Token
                                                                               159495876378727078405590336465058029385
22.11.97.95     datacenter1 rack1       Up     Normal 36.02 KB        76.93% 120248406591972837751075008114949334676
22.11.97.96     datacenter1 rack1       Up     Normal 92.22 KB        23.07% 159495876378727078405590336465058029385

5.6 客户端验证
--查询数据
--插入，跨节点查询
[nosql@bocnet95 bin]$ ./cassandra-cli -h 22.11.97.95 -p 9160
Connected to: "MyCluster" on 22.11.97.95/9160
Welcome to Cassandra CLI version 1.0.10

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown] use test;
Authenticated to keyspace: test
[default@test] get users[utf8('userc')];
Returned 0 results.
Elapsed time: 99 msec(s).
[default@test] get users[utf8('usera')];
=> (column=birth_date, value=1975, timestamp=1401069384452000)
=> (column=full_name, value=Brandon Sanderson, timestamp=1401069384437000)
=> (column=state, value=FL, timestamp=1401069751547000)
Returned 3 results.
Elapsed time: 40 msec(s).

[default@test] set users[utf8('userc')][utf8('full_name')]='Howard Tayler';
Value inserted.
Elapsed time: 57 msec(s).
[default@test] set users[utf8('userc')][utf8('birth_date')]=1968;
Value inserted.
Elapsed time: 4 msec(s).
[default@test] get users[utf8('userc')];
=> (column=birth_date, value=1968, timestamp=1435545861556000)
=> (column=full_name, value=Howard Tayler, timestamp=1435545861495000)
Returned 2 results.
Elapsed time: 6 msec(s).

在节点1查询：
[default@test] get users[utf8('userc')];
=> (column=birth_date, value=1968, timestamp=1435545861556000)
=> (column=full_name, value=Howard Tayler, timestamp=1435545861495000)
Returned 2 results.
Elapsed time: 15 msec(s).

5.7 停止本节点服务
[nosql@bocnet95 bin]$ ps -ef|grep cass
nosql 13402 1 0 09:59 pts/2 00:00:02 /nosql/cassandra/jre1.7.0_04/bin/java -ea -javaagent:./../lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms400M -Xmx400M -Xmn100M -XX:+HeapDumpOnOutOfMemoryError -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dlog4j.configuration=log4j-server.properties -Dlog4j.defaultInitOverride=true -cp ./../conf:./../build/classes/main:./../build/classes/thrift:./../lib/antlr-3.2.jar:./../lib/apache-cassandra-1.0.10.jar:./../lib/apache-cassandra-clientutil-1.0.10.jar:./../lib/apache-cassandra-thrift-1.0.10.jar:./../lib/avro-1.4.0-fixes.jar:./../lib/avro-1.4.0-sources-fixes.jar:./../lib/commons-cli-1.1.jar:./../lib/commons-codec-1.2.jar:./../lib/commons-lang-2.4.jar:./../lib/compress-lzf-0.8.4.jar:./../lib/concurrentlinkedhashmap-lru-1.2.jar:./../lib/guava-r08.jar:./../lib/high-scale-lib-1.1.2.jar:./../lib/jackson-core-asl-1.4.0.jar:./../lib/jackson-mapper-asl-1.4.0.jar:./../lib/jamm-0.2.5.jar:./../lib/jline-0.9.94.jar:./../lib/json-simple-1.1.jar:./../lib/libthrift-0.6.jar:./../lib/log4j-1.2.16.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/slf4j-api-1.6.1.jar:./../lib/slf4j-log4j12-1.6.1.jar:./../lib/snakeyaml-1.6.jar:./../lib/snappy-java-1.0.4.1.jar org.apache.cassandra.thrift.CassandraDaemon
nosql 20083 18237 0 10:46 pts/2 00:00:00 grep cass
[nosql@bocnet95 bin]$ kill -9 13402

[nosql@bocnet95 conf]$ ../bin/nodetool -h bocnet95 ring
Error connection to remote JMX agent!
java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: bocnet95; nested exception is:
        java.net.ConnectException: 拒绝连接]
        at javax.management.remote.rmi.RMIConnector.connect(Unknown Source)
        at javax.management.remote.JMXConnectorFactory.connect(Unknown Source)
        at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:141)
        at org.apache.cassandra.tools.NodeProbe.(NodeProbe.java:111)
        at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:623)
Caused by: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: bocnet95; nested exception is:
        java.net.ConnectException: 拒绝连接]
        at com.sun.jndi.rmi.registry.RegistryContext.lookup(Unknown Source)
        at com.sun.jndi.toolkit.url.GenericURLContext.lookup(Unknown Source)
        at javax.naming.InitialContext.lookup(Unknown Source)
        at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(Unknown Source)
        at javax.management.remote.rmi.RMIConnector.findRMIServer(Unknown Source)
        ... 5 more
Caused by: java.rmi.ConnectException: Connection refused to host: bocnet95; nested exception is:
        java.net.ConnectException: 拒绝连接
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(Unknown Source)
        at sun.rmi.transport.tcp.TCPChannel.createConnection(Unknown Source)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source)
        at sun.rmi.server.UnicastRef.newCall(Unknown Source)
        at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
        ... 10 more
Caused by: java.net.ConnectException: 拒绝连接
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
        at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.(Unknown Source)
        at java.net.Socket.(Unknown Source)
        at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(Unknown Source)
        at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(Unknown Source)
        ... 15 more

--连接集群内存活的节点可以查询到集群状态
[nosql@bocnet95 conf]$ ../bin/nodetool -h rhel5 ring
Address         DC          Rack        Status State   Load            Owns    Token
                                                                               159495876378727078405590336465058029385
22.11.97.95     datacenter1 rack1       Down   Normal 82.99 KB        76.93% 120248406591972837751075008114949334676
22.11.97.96     datacenter1 rack1       Up     Normal 92.22 KB        23.07% 159495876378727078405590336465058029385

--从本节点连接到集群内存活的节点:
[nosql@bocnet95 bin]$ ./cassandra-cli -h 22.11.97.96 -p 9160
Connected to: "MyCluster" on 22.11.97.96/9160
Welcome to Cassandra CLI version 1.0.10

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknown] use test;

--对于仅由存活节点存储的数据,查询报错:
[default@test] get users[utf8('usera')];
=> (column=birth_date, value=1975, timestamp=1401069384452000)
=> (column=full_name, value=Brandon Sanderson, timestamp=1401069384437000)
=> (column=state, value=FL, timestamp=1401069751547000)
Returned 3 results.
Elapsed time: 25 msec(s).

--对于仅由失败节点存储的数据,查询报错:
[default@test] get users[utf8('userc')];
null
UnavailableException()
        at org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:7764)
        at org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:570)
        at org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:542)
        at org.apache.cassandra.cli.CliClient.doSlice(CliClient.java:467)
        at org.apache.cassandra.cli.CliClient.executeGet(CliClient.java:603)
        at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:209)
        at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:220)
        at org.apache.cassandra.cli.CliMain.main(CliMain.java:348)

--在节点1上查询：
[default@test] get users[utf8('userc')];
null
UnavailableException()
        at org.apache.cassandra.thrift.Cassandra$get_slice_result.read(Cassandra.java:7764)
        at org.apache.cassandra.thrift.Cassandra$Client.recv_get_slice(Cassandra.java:570)
        at org.apache.cassandra.thrift.Cassandra$Client.get_slice(Cassandra.java:542)
        at org.apache.cassandra.cli.CliClient.doSlice(CliClient.java:467)
        at org.apache.cassandra.cli.CliClient.executeGet(CliClient.java:603)
        at org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:209)
        at org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:220)
        at org.apache.cassandra.cli.CliMain.main(CliMain.java:348)

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/18922393/viewspace-731634/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/18922393/viewspace-731634/