Cassandra集群安装配置

Cassandra集群安装配置

官方网站：

http://cassandra.apache.org

http://docs.datastax.com/en/landing_page/doc/landing_page/current.html

http://www.datastax.com/dev

http://www.scylladb.com

http://www.oracle.com/technetwork/java/javase/downloads/index.html

NoSQL海量数据Hbase之外的另一开源选择---Cassandra,关于Cassandra的历史就不赘述,kvm的作者用C++重写了Cassandra项目取名为scylladb,有兴趣朋友也只可以看看

环境:

CentOS 7.1 x64

apache-cassandra-3.4

jdk-1.8

python-2.7

安装：

一.安装jdk

wget http://download.oracle.com/otn-pub/java/jdk/8u77-b03/jdk-8u77-linux-x64.tar.gz

tar -xvf jdk-8u77-linux-x64.tar.gz -C /opt

ln -s /opt/jdk-8u77-linux-x64 /opt/jdk

cat >>/etc/profile <<'HERE'

export JAVA_HOME=/opt/jdk

export PATH=$JAVA_HOME/bin:$PATH

HERE

source /etc/profile

或者用linux发行版自带的OpenJDK也可以，但cassandra-3.x己后的版本要求jdk8以上

root@jlive:~#java -version

java version "1.8.0_77"

Java(TM) SE Runtime Environment (build 1.8.0_77-b03)

 

Java HotSpot(TM) 64-Bit Server VM (build 25.77-b03, mixed mode)

root@jlive:~#python -V

Python 2.7.5

二.安装cassandra

wget http://apache.opencas.org/cassandra/3.4/apache-cassandra-3.4-bin.tar.gz

tar -xvf apache-cassandra-3.4-bin.tar.gz -C /opt

ln -s /opt/apache-cassandra-3.4/ /opt/cassandra

 

useradd -r  -m cassandra 

chown -R cassandra: /opt/cassandra

三.优化系统环境变量内核参数

http://docs.datastax.com/en/cassandra/3.x/cassandra/install/installRecommendSettings.html

1.时间同步(所有节点)

2.SSD参数调优

cat >>/etc/rc.d/rc.local <<HERE

echo never > /sys/kernel/mm/transparent_hugepage/enabled

echo noop > /sys/block/sda/queue/scheduler 

echo 0 > /sys/class/block/sda/queue/rotational

echo 8 > /sys/class/block/sda/queue/read_ahead_kb

echo 0 > /proc/sys/vm/zone_reclaim_mode

HERE

3.资源限制

cat >/etc/security/limits.d/99-cassandra.conf <<HERE

cassandra - memlock unlimited

cassandra - nofile 100000

cassandra - nproc 32768

cassandra - as unlimited

HERE

echo "vm.max_map_count = 131072" >>/usr/lib/sysctl.d/00-system.conf

sysctl -p /usr/lib/sysctl.d/00-system.conf

4.禁用swap

swapoff -a

5.RAID readhead优化

blockdev --setra 128 /dev/

blockdev --report /dev/

注意：避免readhead为65536

启动cassandra：

http://wiki.apache.org/cassandra/RunningCassandra

1.启动

su - cassandra -c /opt/cassandra/bin/cassandra -f

-f #foreground,不带该参数时是后台运行

2.停止

pkill -f CassandraDaemon

cqlsh测试：

1.创建KEYSPACE

CREATE KEYSPACE mykeyspace
WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };

2.创建表

USE mykeyspace;

CREATE TABLE mykeyspace.users (
  user_id int PRIMARY KEY,
  fname text,
  lname text
);

3.插入数据

INSERT INTO mykeyspace.users (user_id,  fname, lname)
  VALUES (1745, 'john', 'smith');
INSERT INTO mykeyspace.users (user_id,  fname, lname)
  VALUES (1744, 'john', 'doe');
INSERT INTO mykeyspace.users (user_id,  fname, lname)
  VALUES (1746, 'john', 'smith');

4.查询

SELECT * FROM users;

5.创建index

CREATE INDEX ON users (lname);

SELECT * FROM users WHERE lname = 'smith';

root@jlive:~#/opt/cassandra/bin/cqlsh

Connected to Test Cluster at 127.0.0.1:9042.

[cqlsh 5.0.1 | Cassandra 3.4 | CQL spec 3.4.0 | Native protocol v4]

Use HELP for help.

cqlsh> CREATE KEYSPACE mykeyspace

   ... WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };

cqlsh> USE mykeyspace;

cqlsh:mykeyspace> CREATE TABLE mykeyspace.users (

              ...   user_id int PRIMARY KEY,

              ...   fname text,

              ...   lname text

              ... );

cqlsh:mykeyspace> INSERT INTO mykeyspace.users (user_id,  fname, lname)

              ...   VALUES (1745, 'john', 'smith');

cqlsh:mykeyspace> INSERT INTO mykeyspace.users (user_id,  fname, lname)

              ...   VALUES (1744, 'john', 'doe');

cqlsh:mykeyspace> INSERT INTO mykeyspace.users (user_id,  fname, lname)

              ...   VALUES (1746, 'john', 'smith');

注意：这里只能分3条语句插入，一条语句插入多条数据不支持

cqlsh:mykeyspace> SELECT * FROM users;

 user_id | fname | lname

---------+-------+-------

    1745 |  john | smith

    1744 |  john |   doe

    1746 |  john | smith

(3 rows)

cqlsh:mykeyspace> CREATE INDEX ON users (lname);

cqlsh:mykeyspace> SELECT * FROM users WHERE lname = 'smith';

 user_id | fname | lname

---------+-------+-------

    1745 |  john | smith

    1746 |  john | smith

 

(2 rows)

语法和SQL非常相似，更多关键字可以在cqlsh下输入HELP获取

客户端：

客户端工具需要根据不同语言来封装，datastax公司也给出了主流的语言调用接口，

https://github.com/datastax

http://www.datastax.com/documentation/cql/3.0/webhelp/index.html

创建集群：

步骤同上，这里以OS X 10.11.4为例

1.安装jdk

请参看Tomcat9安装配置 OSX

2.安装cassandra

wget http://apache.opencas.org/cassandra/3.4/apache-cassandra-3.4-bin.tar.gz

tar -xvf apache-cassandra-3.4-bin.tar.gz -C /opt

ln -s /opt/apache-cassandra-3.4 /opt/cassandra

3.添加用户

dscl . -create /Users/cassandra UserShell /bin/bash

dscl . -create /Users/cassandra RealName "Cassandra"

dscl . -create /Users/cassandra UniqueID 1005

dscl . -create /Users/cassandra PrimaryGroupID 1005

dscl . -create /Users/cassandra HomeDirectory /Users/cassandra

dscl . -passwd /Users/cassandra foo.123

MacBook-Pro:opt root# id cassandra

uid=1005(cassandra) gid=1005 groups=1005,12(everyone),61(localaccounts),100(_lpoperator),701(com.apple.sharepoint.group.1)

 

4.修改cassandra目录的属主

chown -R cassandra: /opt/apache-cassandra-3.4

5.修改cassandra配置文件/opt/cassandra/conf/cassandra.yaml

listen_address: 192.168.130.1

rpc_address: 192.168.130.1

seeds: "192.168.130.154","192.168.130.1"

说明:默认是监听在localhost, 集群环境必须指定IP或interface,

因为是去中心集群，种子节点可以指定多个，GossIP协议会自动协商，配置文件很直白，就不多啰嗦，其它节点也要相应修改

6.启动cassandra

su cassandra -c /opt/cassandra/bin/cassandra -f

8.添加集群节点

http://wiki.apache.org/cassandra/Operations

添加集群节点就是在配置文件中指明seeds所在，上面已经修改过，不出问题的话，应该可以看到类似的如下结果

root@jlive:~#/opt/cassandra/bin/nodetool status

Datacenter: datacenter1

=======================

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

--  Address          Load       Tokens       Owns (effective)  Host ID                               Rack

UN  192.168.130.254  170.81 KB  256          52.1%             68e2f590-8c2e-4e06-8875-33d50cc1636a  rack1

 

UN  192.168.130.1    222.48 KB  256          47.9%             5d8f43e7-cdce-4516-bec6-fc9ffd3be2f2  rack1

/opt/cassandra/log/system.log

INFO  08:51:09 Node /192.168.130.254 state jump to NORMAL

INFO  08:51:09 Waiting for gossip to settle before accepting client requests...

INFO  08:51:09 Scheduling approximate time-check task with a precision of 10 milliseconds

INFO  08:51:09 Handshaking version with /192.168.130.1

INFO  08:51:09 Node /192.168.130.1 is now part of the cluster

INFO  08:51:09 InetAddress /192.168.130.1 is now UP

INFO  08:51:09 Updating topology for /192.168.130.1

INFO  08:51:09 Updating topology for /192.168.130.1

 

INFO  08:51:09 Handshaking version with /192.168.130.1

集群监控MX4J(可选)

http://mx4j.sourceforge.net

https://issues.apache.org/jira/browse/CASSANDRA-1068

http://mx4j.sourceforge.net/docs/ch05.html

root@jlive:~#unzip /mnt/hgfs/linux_soft/mx4j-3.0.2.zip mx4j-3.0.2/lib/mx4j-tools.jar

Archive:  /mnt/hgfs/linux_soft/mx4j-3.0.2.zip

  inflating: mx4j-3.0.2/lib/mx4j-tools.jar  

 

root@jlive:~#mv mx4j-3.0.2/lib/mx4j-tools.jar /opt/cassandra/lib/

重启cassandra,日志里会显示

• 
  HttpAdaptor version 3.0.2 started on port 8081
  

访问http://192.168.130.254:8081/即可调用,xml调用需要再添加一个参数&template=identity

本地集群管理---ccm(可选)

https://github.com/pcmanus/ccm

The goal of ccm and ccmlib is to make it easy to create, manage and destroy a small Cassandra cluster on a local box. It is meant for testing a Cassandra cluster.

brew install ccm

或

pip install ccm

或

git clone https://github.com/pcmanus/ccm.git

cd ccm

python setup install