memcached缓存

    什么是缓存

        凡是位于速度相差较大的两种硬件/软件之间的，用于协调两者数据传输速度差异的结构，均可称之为缓存

    分类

            操作系统磁盘缓存，减少磁盘机械操作
            数据库缓存，减少文件系统IO
            应用程序缓存，减少数据库操作
            WEB服务器缓存，减少应用服务器请求
            客户端浏览器缓存，减少网站访问

    web相关

            数据库缓存
            系统配置参数缓存
            ORM提供的对象缓存、查询缓存
            缓存服务器 EHCache、OSCache、JBossCache
            通用缓存 memcached、radis
            页面缓存（动态页面静态化、Servlet缓存、页面局部缓存）
            ajax查询结果缓存
            浏览器缓存
            代理服务器缓存
            CDN加速

    适用范围

        频繁访问且时效性不高的数据。例如：用户信息、菜单权限、功能权限、历史数据查询等

    memcached缓存

        Memcached 是一个高性能的分布式内存对象缓存系统，用于动态Web应用以减轻数据库负载。它通过在内存中缓存数据和对象来减少读取数据库的次数，从而提高动态、数据库驱动网站的速度。Memcached基于一个存储键/值对的hashmap。其守护进程（daemon ）是用C写的，但是客户端可以用任何语言来编写，并通过memcached协议与守护进程通信。[http://baike.baidu.com/view/794242.htm]

    alisoft Memcached Client

        尽管是“分布式”缓存服务器，但服务器端并没有分布式功能。各个memcached不会互相通信以共享信息。那么，怎样进行分布式呢？这完全取决于客户端的实现。所以，alisoft Memcached Client是一个支持分布式集群的管理客户端。

    单客户端配置

<?xml version="1.0" encoding="UTF-8"?>
<memcached>
    <client name="mclient0" compressEnable="true" defaultEncoding="UTF-8" socketpool="pool0">
        <errorHandler>com.alisoft.xplatform.asf.cache.memcached.MemcachedErrorHandler</errorHandler>
    </client>
    <socketpool name="pool0" failover="true" initConn="5" minConn="5" maxConn="250" maintSleep="5000" nagle="false"
        socketTO="3000" aliveCheck="true">
        <servers>127.0.0.1:11211,127.0.0.1:11212</servers>
        <weights>3,7</weights>
    </socketpool>
</memcached>

   
    创建memcached的标签

    创建 client的标签

        name 属性是程序中使用Cache的唯一标识。
        socketpool 属性将会关联到后面的socketpool配置。
        errorHandler 可选，用来处理出错情况。注意在Tag中不要使用空格或者Tab键。

        创建socketpool的标签

            name 属性和client 配置中的socketpool 属性相关联。
            maintSleep属性是后台线程管理SocketIO池的检查间隔时间，如果设置为0，则表明不需要后台线程维护SocketIO线程池，默认需要管理。
            socketTO 属性是Socket操作超时配置，单位ms。
            aliveCheck 属性表示在使用Socket以前是否先检查Socket状态。

        创建 servers 标签作为socketPool的子标签.设置memcache服务端实例地址，支持多个地址设置，例如“127.0.0.1:11211” 或 “127.0.0.1:11211, 127.0.0.1:11212”

        创建 weights 标签作为socketPool的子标签(可选),它表明了上面设置的服务器实例的Load权重. 例如 <weights>3,7</weights>表示30% load 在 127.0.0.1:11211, 70% load 在 127.0.0.1:11212
            单客户端代码

/**
 * lincl
 * 2016年7月12日 下午4:56:35
 * 
 */
package com.lezic.core.cache.memcached;

import java.util.Calendar;

import org.junit.AfterClass;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;

import com.alisoft.xplatform.asf.cache.ICacheManager;
import com.alisoft.xplatform.asf.cache.IMemcachedCache;
import com.alisoft.xplatform.asf.cache.memcached.CacheUtil;
import com.alisoft.xplatform.asf.cache.memcached.MemcachedCacheManager;

/**
 * 单客户端
 * 
 * @author lincl
 * 
 */
public class MemcachedClient {
    static ICacheManager<IMemcachedCache> manager;

    /**
     * 初始化上下文，加载配置文件
     * 
     * @throws Exception
     */
    @BeforeClass
    public static void setUpBeforeClass() throws Exception {
        manager = CacheUtil.getCacheManager(IMemcachedCache.class, MemcachedCacheManager.class.getName());
        manager.setConfigFile("memcached0.xml");
        manager.setResponseStatInterval(5 * 1000);
        manager.start();
    }

    /**
     * 摧毁管理器
     * 
     * @throws java.lang.Exception
     */
    @AfterClass
    public static void tearDownAfterClass() throws Exception {
        manager.stop();
    }

    /**
     * 测试获取
     * 
     */
    @Test
    public void testGet() {
        try {
            IMemcachedCache cache = manager.getCache("mclient0");
            cache.remove("key1");
            cache.remove("key2你好");
            cache.put("key1", "1");
            cache.put("key2你好", "你好123");
            Assert.assertEquals(cache.get("key1"), "1");
            Assert.assertEquals(cache.get("key2你好"), "你好123");
        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

    /**
     * 测试删除
     */
    @Test
    public void testRemove() {
        try {
            IMemcachedCache cache = manager.getCache("mclient0");
            cache.remove("key1");
            cache.put("key1", "value1");
            Assert.assertEquals(cache.get("key1"), "value1");
            cache.remove("key1");
            Assert.assertNull(cache.get("key1"));
        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

    /**
     * 测试是否包含key
     */
    @Test
    public void testContainsKey() {
        try {
            IMemcachedCache cache = manager.getCache("mclient0");
            cache.remove("key1");
            cache.put("key1", "value1");
            Assert.assertTrue(cache.containsKey("key1"));
        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

    @Test
    public void testAdd() {
        try {
            IMemcachedCache cache = manager.getCache("mclient0");
            cache.remove("key1");
            Assert.assertTrue(cache.add("key1", "value1"));
            Assert.assertFalse(cache.add("key1", "value1"));
            Assert.assertEquals(cache.get("key1"), "value1");
            cache.remove("key1");
        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

    /**
     * Test method for {@link com.alisoft.xplatform.asf.cache.ICache#clear()}.
     */
    @Test
    public void testClear() {
        try {
            IMemcachedCache cache = manager.getCache("mclient0");
            cache.remove("key1");
            cache.put("key1", "value1");
            Assert.assertEquals(cache.get("key1"), "value1");
            cache.clear();
            Assert.assertNull(cache.get("key1"));
            Thread.sleep(2000);
        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

    /**
     * 保存有有效期的数据 Test method for
     * {@link com.alisoft.xplatform.asf.cache.ICache#put(java.lang.Object, java.lang.Object, java.util.Date)} .
     */
    @Test
    public void testPutKVDate() {
        try {
            IMemcachedCache cache = manager.getCache("mclient0");
            Calendar calendar = Calendar.getInstance();
            calendar.add(Calendar.SECOND, 5);
            cache.remove("key1");
            cache.put("key1", "value1", calendar.getTime());
            Assert.assertEquals(cache.get("key1"), "value1");
            Thread.sleep(5 * 1000 + 1000);
            Assert.assertNull(cache.get("key1"));
        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }
}

    

9.集群配置

<?xml version="1.0" encoding="UTF-8"?>
<memcached>

    <client name="mclient" compressEnable="true" defaultEncoding="UTF-8" socketpool="pool0">
        <errorHandler>com.alisoft.xplatform.asf.cache.memcached.MemcachedErrorHandler</errorHandler>
    </client>

    <client name="mclient1" compressEnable="true" defaultEncoding="UTF-8" socketpool="pool1">
        <errorHandler>com.alisoft.xplatform.asf.cache.memcached.MemcachedErrorHandler</errorHandler>
    </client>

    <client name="mclient2" compressEnable="true" defaultEncoding="UTF-8" socketpool="pool2">
        <errorHandler>com.alisoft.xplatform.asf.cache.memcached.MemcachedErrorHandler</errorHandler>
    </client>

    <client name="mclient3" compressEnable="true" defaultEncoding="UTF-8" socketpool="pool3">
        <errorHandler>com.alisoft.xplatform.asf.cache.memcached.MemcachedErrorHandler</errorHandler>
    </client>

    <client name="mclient4" compressEnable="true" defaultEncoding="UTF-8" socketpool="pool4">
        <errorHandler>com.alisoft.xplatform.asf.cache.memcached.MemcachedErrorHandler</errorHandler>
    </client>


    <socketpool name="pool0" failover="true" initConn="5" minConn="5" maxConn="250" maintSleep="0" nagle="false"
        socketTO="3000" aliveCheck="true">
        <servers>127.0.0.1:11210</servers>
    </socketpool>

    <socketpool name="pool1" failover="true" initConn="5" minConn="5" maxConn="250" maintSleep="0" nagle="false"
        socketTO="3000" aliveCheck="true">
        <servers>127.0.0.1:11211</servers>
    </socketpool>

    <socketpool name="pool2" failover="true" initConn="5" minConn="5" maxConn="250" maintSleep="0" nagle="false"
        socketTO="3000" aliveCheck="true">
        <servers>127.0.0.1:11212</servers>
    </socketpool>

    <socketpool name="pool3" failover="true" initConn="5" minConn="5" maxConn="250" maintSleep="0" nagle="false"
        socketTO="3000" aliveCheck="true">
        <servers>127.0.0.1:11213</servers>
    </socketpool>

    <socketpool name="pool4" failover="true" initConn="5" minConn="5" maxConn="250" maintSleep="0" nagle="false"
        socketTO="3000" aliveCheck="true">
        <servers>127.0.0.1:11214</servers>
    </socketpool>

    <cluster name="cluster1" mode="active">
        //mode = active,standby
        <memCachedClients>mclient1,mclient2</memCachedClients>
    </cluster>

    <cluster name="cluster2" mode="standby">
        //mode = active,standby
        <memCachedClients>mclient3,mclient4</memCachedClients>
    </cluster>

</memcached>

   
    创建cluster标签
    创建memCachedClients

        标签作为cluster的子标签，然后将客户端配置到memCachedClients 标签中

    可以配置cluster mode（如果没有设置mode属性，默认采用active）

    集群当前的特性

            集群中多节点软负载均衡。（当前采用简单的Hash算法加取余来分发数据）
            数据在多节点上异步冗余存储。（防止数据丢失最基本要求）
            节点不可用切换功能。（当根据算法分发到某一失败节点时可以转向到其他可用节点）
            节点恢复可用后数据Lazy复制。（当A,B两台机器作为集群的时候，如果A出现了问题，系统会去B获取数据，当A正常以后，如果应用在A中没有拿到数据可以去B获取数据，并且复制到A上，这种方式也是一种lazy的复制。）

        集群代码

/**
 * 
 */
package com.alisoft.xplatform.asf.cache;

import org.junit.AfterClass;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Ignore;
import org.junit.Test;

import com.alisoft.xplatform.asf.cache.memcached.CacheUtil;
import com.alisoft.xplatform.asf.cache.memcached.MemcachedCacheManager;

/**
 * 集群测试类
 * @author wenchu.cenwc
 * 
 */
public class MemcachedClusterTest {

    static ICacheManager<IMemcachedCache> manager;

    /**
     * @throws java.lang.Exception
     */
    @BeforeClass
    public static void setUpBeforeClass() throws Exception {
        manager = CacheUtil.getCacheManager(IMemcachedCache.class, MemcachedCacheManager.class.getName());
        manager.setConfigFile("memcached_cluster.xml");
        manager.start();
    }

    /**
     * @throws java.lang.Exception
     */
    @AfterClass
    public static void tearDownAfterClass() throws Exception {
        manager.stop();
    }

    @Test
    public void testActiveMode() {
        try {
            IMemcachedCache cache1 = manager.getCache("mclient1");
            IMemcachedCache cache2 = manager.getCache("mclient2");

            cache1.put("key1", "value1");
            cache1.put("key2", "value2");
            cache1.put("key3", "value3");
            cache1.put("key4", "value4");
            cache1.put("key5", "value5");
            cache1.put("key6", "value6");

            // 模拟mclient1失效(结束服务端)，有出错日志在控制台打印
            Assert.assertEquals(cache1.get("key1"), "value1");
            Assert.assertEquals(cache1.get("key2"), "value2");
            Assert.assertEquals(cache1.get("key3"), "value3");
            Assert.assertEquals(cache1.get("key4"), "value4");
            Assert.assertEquals(cache1.get("key5"), "value5");
            Assert.assertEquals(cache1.get("key6"), "value6");

            Assert.assertEquals(cache2.get("key1"), "value1");
            Assert.assertEquals(cache2.get("key2"), "value2");
            Assert.assertEquals(cache2.get("key3"), "value3");
            Assert.assertEquals(cache2.get("key4"), "value4");
            Assert.assertEquals(cache2.get("key5"), "value5");
            Assert.assertEquals(cache2.get("key6"), "value6");

            // 恢复mclient1，无出错日志在控制台打印
            Assert.assertEquals(cache1.get("key1"), "value1");
            Assert.assertEquals(cache1.get("key2"), "value2");
            Assert.assertEquals(cache1.get("key3"), "value3");
            Assert.assertEquals(cache1.get("key4"), "value4");
            Assert.assertEquals(cache1.get("key5"), "value5");
            Assert.assertEquals(cache1.get("key6"), "value6");

            Assert.assertEquals(cache2.get("key1"), "value1");
            Assert.assertEquals(cache2.get("key2"), "value2");
            Assert.assertEquals(cache2.get("key3"), "value3");
            Assert.assertEquals(cache2.get("key4"), "value4");
            Assert.assertEquals(cache2.get("key5"), "value5");
            Assert.assertEquals(cache2.get("key6"), "value6");

        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

    @Test
    @Ignore
    public void testStandByMode() {
        IMemcachedCache cache1 = manager.getCache("mclient3");
        IMemcachedCache cache2 = manager.getCache("mclient4");

        cache1.put("key1", "value1");
        cache1.put("key2", "value2");
        cache1.put("key3", "value3");
        cache1.put("key4", "value4");
        cache1.put("key5", "value5");
        cache1.put("key6", "value6");

        // 模拟mclient1失效(结束服务端)，有出错日志在控制台打印
        Assert.assertEquals(cache1.get("key1"), "value1");
        Assert.assertEquals(cache1.get("key2"), "value2");
        Assert.assertEquals(cache1.get("key3"), "value3");
        Assert.assertEquals(cache1.get("key4"), "value4");
        Assert.assertEquals(cache1.get("key5"), "value5");
        Assert.assertEquals(cache1.get("key6"), "value6");

        Assert.assertEquals(cache2.get("key1"), "value1");
        Assert.assertEquals(cache2.get("key2"), "value2");
        Assert.assertEquals(cache2.get("key3"), "value3");
        Assert.assertEquals(cache2.get("key4"), "value4");
        Assert.assertEquals(cache2.get("key5"), "value5");
        Assert.assertEquals(cache2.get("key6"), "value6");

        // 恢复mclient1，无出错日志在控制台打印
        Assert.assertNull(cache1.get("key1"));
        Assert.assertEquals(cache1.get("key2"), "value2");
        Assert.assertNull(cache1.get("key3"));
        Assert.assertEquals(cache1.get("key4"), "value4");
        Assert.assertNull(cache1.get("key5"));
        Assert.assertEquals(cache1.get("key6"), "value6");

        Assert.assertNull(cache2.get("key1"));
        Assert.assertEquals(cache2.get("key2"), "value2");
        Assert.assertNull(cache2.get("key3"));
        Assert.assertEquals(cache2.get("key4"), "value4");
        Assert.assertNull(cache2.get("key5"));
        Assert.assertEquals(cache2.get("key6"), "value6");
    }

    /**
     * 集群复制
     * 
     */
    @Test
    public void testClusterCopy() {
        try {
            IMemcachedCache cache = manager.getCache("mclient");
            IMemcachedCache cache1 = manager.getCache("mclient1");
            IMemcachedCache cache2 = manager.getCache("mclient2");

            cache.put("key1", "value1");
            cache.put("key2", "value2");
            cache.put("key3", "value3");
            cache.put("key4", "value4");
            cache.put("key5", "value5");
            cache.put("key6", "value6");

            cache1.remove("key1");
            cache1.remove("key2");
            cache1.remove("key3");
            cache1.remove("key4");
            cache1.remove("key5");
            cache1.remove("key6");

            cache2.remove("key1");
            cache2.remove("key2");
            cache2.remove("key3");
            cache2.remove("key4");
            cache2.remove("key5");
            cache2.remove("key6");

            manager.clusterCopy("mclient", "cluster1");

            Assert.assertEquals(cache1.get("key1"), "value1");
            Assert.assertEquals(cache1.get("key2"), "value2");
            Assert.assertEquals(cache1.get("key3"), "value3");
            Assert.assertEquals(cache1.get("key4"), "value4");
            Assert.assertEquals(cache1.get("key5"), "value5");
            Assert.assertEquals(cache1.get("key6"), "value6");

            Assert.assertEquals(cache2.get("key1"), "value1");
            Assert.assertEquals(cache2.get("key2"), "value2");
            Assert.assertEquals(cache2.get("key3"), "value3");
            Assert.assertEquals(cache2.get("key4"), "value4");
            Assert.assertEquals(cache2.get("key5"), "value5");
            Assert.assertEquals(cache2.get("key6"), "value6");

        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

    @Test
    public void testReload() {
        try {
            IMemcachedCache cache = manager.getCache("mclient");
            IMemcachedCache cache1 = manager.getCache("mclient1");
            IMemcachedCache cache2 = manager.getCache("mclient2");
            IMemcachedCache cache5 = manager.getCache("mclient5");
            IMemcachedCache cache6 = manager.getCache("mclient6");

            Assert.assertNull(cache5);
            Assert.assertNull(cache6);

            cache.clear();
            Thread.sleep(1000);
            cache1.clear();
            Thread.sleep(1000);
            cache2.clear();
            Thread.sleep(1000);

            cache.put("key1", "1");
            cache1.put("key2", "2");

            Assert.assertNull(cache.get("key2"));
            Assert.assertNull(cache1.get("key1"));

            Thread.sleep(2000);

            // manager.reload("http://10.2.226.41/sip/memcached_cluster2.xml");
            manager.reload("memcached_cluster2.xml");

            Thread.sleep(2000);

            cache1 = manager.getCache("mclient1");
            cache2 = manager.getCache("mclient2");
            cache5 = manager.getCache("mclient5");
            cache6 = manager.getCache("mclient6");

            Assert.assertNull(cache1);
            Assert.assertNull(cache2);
            Assert.assertNotNull(cache5);
            Assert.assertNotNull(cache6);

            cache5.clear();
            Thread.sleep(1000);
            cache6.clear();
            Thread.sleep(3000);

            manager.reload("memcached_cluster3.xml");

            Thread.sleep(2000);

            cache = manager.getCache("mclient");
            cache1 = manager.getCache("mclient1");
            cache2 = manager.getCache("mclient2");
            cache5 = manager.getCache("mclient5");
            cache6 = manager.getCache("mclient6");

            Assert.assertEquals(cache.get("key2"), "2");
            Assert.assertEquals(cache1.get("key1"), "1");

            Assert.assertEquals(cache2.get("key2"), "2");
            Assert.assertEquals(cache5.get("key2"), "2");
            Assert.assertNull(cache2.get("key1"));
            Assert.assertNull(cache5.get("key1"));
            Assert.assertNull(cache6.get("key1"));
            Assert.assertNull(cache6.get("key2"));

        } catch (Exception ex) {
            ex.printStackTrace();
            Assert.assertTrue(false);
        }
    }

}