Redis原理
Redis为单进程单线程模式，采用队列模式将并发访问变成串行访问，且多客户端对Redis的连接并不存在竞争关系。其次Redis提供一些命令SETNX，GETSET，可以方便实现分布式锁机制。且只需修改一个配置即可在RDB和AOF这两种持久化方案之间切换。
基本数据结构
总体来说以下这五种数据类型都是<K,V>的数据结构
-
k为String k,v
-
k为list k,{v,v,v,v,v} 有序
这里的list push()/pop() 的handler与MQ的操作方式非常相似
-
k为set k,{v,v,v,v,v} 无序
-
k为zset k,{v,v,v,v,v} 排序
-
k为hash k,[{field1,v},{field2,v},{field3,v}]
sentinel
sentinel独立于redis进程，一个redis进程一个sentinel，通过心跳等机制负责集群的容错和失效备援
锁
-
redis为单进程单线程模式，采用队列模式将并发访问变成串行访问，它采用乐观锁，适合存放读多写少的数据。redis有事物控制机制，但类似关系型数据库的stored procedure，并行发送多条命令才可生效。那么如果是单台redis则无须程序控制事物，但如果是集群，则需使用分布式锁，否则有出现数据覆盖的风险。然而这种风险出现的概率还是极低的，如果存放的数据不容丢失并且是通过先存入Redis再持久化，那么使用分布式锁的必要还是有的。
-
分布式锁简单说就是通过Redis提供的两个方法getset()和setnx()配合使用，最后再通过时间戳或其它机制确保数据不在更新时被覆盖。
-
getset()和setnx()
SETNX命令（SET if Not eXists）
语法：SETNX key value
功能：当且仅当 key 不存在，将 key 的值设为 value ，并返回1；若给定的 key 已经存在，则 SETNX 不做任何动作，并返回0。GETSET命令
语法：GETSET key value
功能：将给定 key 的值设为 value ，并返回 key 的旧值 (old value)，当 key 存在但不是字符串类型时，返回一个错误，当key不存在时，返回nil。 -
分布式并发导图
-
- 分布式锁整体流程图
- 代码
handler类
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.UnsupportedEncodingException;
import java.util.Date;
import com.zgg.group2.common.util.ExceptionUtil;
import com.zgg.group2.scheduler.redis.factory.RedisSourceSpringCluster;
public class RedisHandlerSpringCluster {
public static final int EXPIRE_SECOND_DEFAULT = 4*60*60;
private String lockTransaction(String key, String value) {
putObject(key, value);
long result = setnx(new StringBuffer(key).append(".lock").toString(), new Date().getTime() + "");
if (result == 1) {
return doTransaction(key, value);
}
Object obj = getObject(new StringBuffer(key).append(".lock").toString());
if (obj == null)
return "get obj = null.";
String keyLockValue = obj.toString();
if ("nil".equals(keyLockValue))
this.lockTransaction(key, value);
try {
String getsetResult = getSet(new StringBuffer(key).append(".lock").toString(), new Date().getTime() + "");
if ("nil".equals(getsetResult))
return doTransaction(key, value);
if (keyLockValue.equals(getsetResult) && new Date().getTime() - Long.valueOf(keyLockValue) > 1000)
return doTransaction(key, value);
if (keyLockValue.equals(getsetResult) && new Date().getTime() - Long.valueOf(keyLockValue) < 1000)
Thread.sleep(100);
this.lockTransaction(key, value);
if (!keyLockValue.equals(getsetResult)){
Thread.sleep(100);
this.lockTransaction(key, value);
}
return "error";
} catch (UnsupportedEncodingException e) {
return ExceptionUtil.exceptionStack(e);
} catch (InterruptedException e) {
return ExceptionUtil.exceptionStack(e);
} finally {
remove(new StringBuffer(key).append(".lock").toString());
}
}
private String doTransaction(String key, String value) {
putObject(key, value);
String s = (String) getObject(key);
remove(new StringBuffer(key).append(".lock").toString());
return s;
}
/**
* Set the string value as value of the key. The string can't be longer than 1073741824 bytes (1GB).
* Time complexity: O(1)
* @param key
* @param fieldVal
* @return Status code reply
*/
public static String putObject( final String key , Object fieldVal){
return putObject(key,fieldVal,EXPIRE_SECOND_DEFAULT);
}
public static String putObject( final String key, final Object fieldVal, Integer expireTime) {
expireTime = expireTime==null?EXPIRE_SECOND_DEFAULT:expireTime;
byte[] keyBytes =getBytes(key);
String res = RedisSourceSpringCluster.getInstance().getRedisClient().set(keyBytes,serialize(fieldVal));
RedisSourceSpringCluster.getInstance().getRedisClient().expire(keyBytes,expireTime);
return res;
}
public static Object getObject( final String key){
Object resultObj = unserialize(RedisSourceSpringCluster.getInstance().getRedisClient().get(getBytes(key)));
return resultObj;
}
/**当且仅当 key 不存在，将 key 的值设为 value ，并返回1；若给定的 key 已经存在，则 SETNX 不做任何动作，并返回0。
* @param key
* @param value
* @return
*/
public static Long setnx( final String key, String value) {
Long ll = RedisSourceSpringCluster.getInstance().getRedisClient().setnx(getBytes(key), getBytes(value));
return ll;
}
public static String getSet( final String key, String value) throws UnsupportedEncodingException {
byte[] b = RedisSourceSpringCluster.getInstance().getRedisClient().getSet(getBytes(key), getBytes(value));
return unserialize(b)!=null ? unserialize(b).toString():new String();
}
/**
* Remove the specified key
* Time complexity: O(1)
* @param key
*/
public static Long remove( final String key) {
return RedisSourceSpringCluster.getInstance().getRedisClient().del(getBytes(key));
}
public static byte[] serialize(Object object) {
ObjectOutputStream oos = null;
ByteArrayOutputStream baos = null;
try {
baos = new ByteArrayOutputStream();
oos = new ObjectOutputStream(baos);
oos.writeObject(object);
byte[] bytes = baos.toByteArray();
return bytes;
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if(baos!=null){
baos.close();
}
if(oos!=null){
oos.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
return null;
}
/**
* 反序列化
* @param bytes
* @return Object
*/
public static Object unserialize(byte[] bytes) {
if (bytes == null) {
return null;
}
ByteArrayInputStream bais = null;
ObjectInputStream ois = null;
try {
bais = new ByteArrayInputStream(bytes);
ois = new ObjectInputStream(bais);
return ois.readObject();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
if(bais!=null){
bais.close();
}
if(ois!=null){
ois.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
return null;
}
public static byte[] getBytes(String str){
return str==null?null:str.getBytes();
}
}
DataSource类
import java.io.IOException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import redis.clients.jedis.JedisCluster;
import com.zgg.group2.scheduler.redis.config.RedisConfigSpringCluster;
/**
* redis 集群
*/
public class RedisSourceSpringCluster {
private static RedisSourceSpringCluster instance = new RedisSourceSpringCluster();
private static JedisCluster jedisCluster;
static{
@SuppressWarnings("resource")
ApplicationContext ctx = new AnnotationConfigApplicationContext(RedisConfigSpringCluster.class);
if(jedisCluster == null){
jedisCluster = ctx.getBean(JedisCluster.class);
}
}
public JedisCluster getRedisClient() {
return jedisCluster;
}
public void destorySource() {
try {
jedisCluster.close();
} catch (IOException e) {
e.printStackTrace();
}
}
public static RedisSourceSpringCluster getInstance() {
return instance;
}
}
Config类
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.ImportResource;
/**
* 加载配置文件
* @author
*
*/
@Configuration
@ImportResource({"classpath*:redis-context.xml"})
public class RedisConfigSpringCluster {
}
redis-context.xml配置
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:aop="http://www.springframework.org/schema/aop"
xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-4.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-4.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-4.0.xsd">
<context:property-placeholder location="classpath*:redis.properties"/>
<bean name="genericObjectPoolConfig" class="org.apache.commons.pool2.impl.GenericObjectPoolConfig" >
<property name="maxWaitMillis" value="3000" /> <!-- 获取连接时的最大等待毫秒数(如果设置为阻塞时BlockWhenExhausted),如果超时就抛异常, 小于零:阻塞不确定的时间, 默认-1-->
<property name="maxTotal" value="50" /> <!-- 最大连接数, 默认8个-->
<property name="minIdle" value="8" /> <!-- 最小空闲连接数, 默认0 -->
<property name="maxIdle" value="10" /> <!-- 最大空闲连接数 -->
</bean>
<bean id="jedisCluster" class="redis.clients.jedis.JedisCluster" >
<constructor-arg index="0">
<set>
<bean class="redis.clients.jedis.HostAndPort" >
<constructor-arg name="host" value="${jedisCluster1.ip}" index="0"/>
<constructor-arg name="port" value="${jedisCluster1.port}" index="1"/>
</bean>
<bean class="redis.clients.jedis.HostAndPort" >
<constructor-arg name="host" value="${jedisCluster2.ip}" index="0"/>
<constructor-arg name="port" value="${jedisCluster2.port}" index="1"/>
</bean>
<bean class="redis.clients.jedis.HostAndPort" >
<constructor-arg name="host" value="${jedisCluster3.ip}" index="0"/>
<constructor-arg name="port" value="${jedisCluster3.port}" index="1"/>
</bean>
<bean class="redis.clients.jedis.HostAndPort" >
<constructor-arg name="host" value="${jedisCluster4.ip}" index="0"/>
<constructor-arg name="port" value="${jedisCluster4.port}" index="1"/>
</bean>
<bean class="redis.clients.jedis.HostAndPort" >
<constructor-arg name="host" value="${jedisCluster5.ip}" index="0"/>
<constructor-arg name="port" value="${jedisCluster5.port}" index="1"/>
</bean>
<bean class="redis.clients.jedis.HostAndPort" >
<constructor-arg name="host" value="${jedisCluster6.ip}" index="0"/>
<constructor-arg name="port" value="${jedisCluster6.port}" index="1"/>
</bean>
</set>
</constructor-arg>
<constructor-arg value="5000" index="1"/>
<constructor-arg value="1000" index="2"/>
</bean >
</beans>
redis.properties配置
redis.password=${maven.redis.password}
redis.timeout=100000
redis.maxIdle=100
redis.minIdle=50
redis.maxTotal=300
redis.maxWait=1000
redis.testOnBorrow=true
redis.testOnReturn=true
jedisCluster1.ip=${maven.redisCluster1.host}
jedisCluster2.ip=${maven.redisCluster2.host}
jedisCluster3.ip=${maven.redisCluster3.host}
jedisCluster4.ip=${maven.redisCluster4.host}
jedisCluster5.ip=${maven.redisCluster5.host}
jedisCluster6.ip=${maven.redisCluster6.host}
jedisCluster1.port=${maven.redisCluster1.port}
jedisCluster2.port=${maven.redisCluster2.port}
jedisCluster3.port=${maven.redisCluster3.port}
jedisCluster4.port=${maven.redisCluster4.port}
jedisCluster5.port=${maven.redisCluster5.port}
jedisCluster6.port=${maven.redisCluster6.port}
pom.xml配置
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>zgg-group2</groupId>
<artifactId>zgg-group2-level1</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>group2-level3-common-config</artifactId>
<build>
<finalName>${project.artifactId}</finalName>
<resources>
<resource>
<directory>src/main/resources</directory>
<targetPath>${project.build.directory}/classes</targetPath>
<includes>
<include>**/*</include>
</includes>
<filtering>true</filtering>
</resource>
</resources>
</build>
<profiles>
<!-- 本地环境 -->
<profile>
<id>local</id>
<properties>
<tttt>local</tttt>
<!-- redis -->
<maven.redisCluster1.host>192.168.89.221</maven.redisCluster1.host>
<maven.redisCluster1.port>7000</maven.redisCluster1.port>
<maven.redisCluster2.host>192.168.89.221</maven.redisCluster2.host>
<maven.redisCluster2.port>7001</maven.redisCluster2.port>
<maven.redisCluster3.host>192.168.89.221</maven.redisCluster3.host>
<maven.redisCluster3.port>7002</maven.redisCluster3.port>
<maven.redisCluster4.host>192.168.89.221</maven.redisCluster4.host>
<maven.redisCluster4.port>7003</maven.redisCluster4.port>
<maven.redisCluster5.host>192.168.89.221</maven.redisCluster5.host>
<maven.redisCluster5.port>7004</maven.redisCluster5.port>
<maven.redisCluster6.host>192.168.89.221</maven.redisCluster6.host>
<maven.redisCluster6.port>7005</maven.redisCluster6.port>
<maven.redis.password></maven.redis.password>
</properties>
</profile>
</profiles>
</project>
持久化
-
RDB
- 描述：RDB持久化是指在指定的时间间隔内将内存中的数据集快照写入磁盘，实际操作过程是fork一个子进程，先将数据集写入临时文件，写入成功后，再替换之前的文件，用二进制压缩存储。
- 配置：redis配置\关键文件\redis-conf.txt save 900 1
- 手动dump：client中执行 ，然后备份dump下的rdb文件。如果要恢复到dump好的rdb文件状态，只需将备份好的rdb文件移动到指定的文件夹，重启服务，会自动加载rdb文件内容
-
AOF:
- 描述：AOF持久化以日志的形式记录服务器所处理的每一个写、删除操作，查询操作不会记录，以文本的方式记录，可以打开文件看到详细的操作记录。
- 配置：appendonly no改为yes，当有更新操作，或每过一秒就会有appendonly.aof文件产生。
865
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
