Spring Boot 使用 Hazelcast 作为 CacheProvider 以及 Hibernate JPA 二级缓存

之前在项目中一直用到hazelcast,但是并没有系统的总结下,今天刚好总结下.

简介

关于Hazelcast的介绍我就不赘述了,今天主要总结下用法.这篇帖子写的还不错,要了解的话可以看这个.
https://blog.csdn.net/jiangbb8686/article/details/102499206

关于Spring-Boot 缓存的使用 可以看这篇Blog.
https://blog.csdn.net/u013262689/article/details/84347490
需要注意的一点就是,它里面 @Cacheable 里面的value属性,就是我之后代码中使用的cacheNames.

使用

POM

		<dependency>
            <groupId>com.hazelcast</groupId>
            <artifactId>hazelcast</artifactId>
        </dependency>
        <dependency>
            <groupId>com.hazelcast</groupId>
            <artifactId>hazelcast-hibernate53</artifactId>
        </dependency>
        <dependency>
            <groupId>com.hazelcast</groupId>
            <artifactId>hazelcast-spring</artifactId>
        </dependency>

YML

JPA二级缓存是默认关闭的,需要设置去开启.并且设置factory_class为HazelcastCacheRegionFactory

spring:
  jpa:
    properties:
      hibernate.cache.use_second_level_cache: true
      hibernate.cache.use_query_cache: false
      hibernate.cache.region.factory_class: com.hazelcast.hibernate.HazelcastCacheRegionFactory
      hibernate.cache.use_minimal_puts: true
      hibernate.cache.hazelcast.instance_name: HC-SAMPLE-MS-A
      hibernate.cache.hazelcast.use_lite_member: true

use_query_cache: 使用查询缓存
use_minimal_puts: 是否以频繁的读操作为代价，优化二级缓存，以实现最小化写操作。在 Hibernate 3 中，该属性对集群缓存非常有用.
use_lite_member: 是否使用嵌入式模式
user_native_client: 是否启动客户端模式

我是用的是jhipster项目,其实他注入HazelcastInstance Bean还是通过Configuration代码,只不过到时候会读取yml的配置.

jhipster:
  cache: # Cache configuration
    hazelcast: # Hazelcast distributed cache
      time-to-live-seconds: 3600
      backup-count: 1

Entity

使用@Cache注解实体,或者级联的字段,当开启了二级缓存后,例如执行了findAll查询了该实体会以包名加entity名加ID作为cache的key,例如com.hc.ms.generated.domain.Album#1.并且自动加入缓存.并且当再次针对该entity的使用findById时使用该缓存.关于缓存策略,则可以看我写的另外一篇blog.
Hibernate 缓存中 @Cache 的 CacheConcurrencyStrategy的使用

图片中红线上才是该Entity的Id,蓝线是com.hc.ms.generated.domain.Album#8这个key对应的value的id.

package com.hc.ms.generated.domain;

import org.hibernate.annotations.Cache;
import org.hibernate.annotations.CacheConcurrencyStrategy;

import javax.persistence.*;
import javax.validation.constraints.*;

import java.io.Serializable;
import java.util.HashSet;
import java.util.Set;

/**
 * A Album.
 */
@Entity
@Table(name = "album")
@Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
public class Album implements Serializable {

    private static final long serialVersionUID = 1L;

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @NotNull
    @Column(name = "name", nullable = false)
    private String name;

    @Column(name = "hc_desc")
    private String desc;

    @Column(name = "detail")
    private String detail;

    @OneToMany(mappedBy = "album")
    @Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
    private Set<Photo> photos = new HashSet<>();

    // jhipster-needle-entity-add-field - JHipster will add fields here
    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public Album name(String name) {
        this.name = name;
        return this;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getDesc() {
        return desc;
    }

    public Album desc(String desc) {
        this.desc = desc;
        return this;
    }

    public void setDesc(String desc) {
        this.desc = desc;
    }

    public String getDetail() {
        return detail;
    }

    public Album detail(String detail) {
        this.detail = detail;
        return this;
    }

    public void setDetail(String detail) {
        this.detail = detail;
    }

    public Set<Photo> getPhotos() {
        return photos;
    }

    public Album photos(Set<Photo> photos) {
        this.photos = photos;
        return this;
    }

    public Album addPhoto(Photo photo) {
        this.photos.add(photo);
        photo.setAlbum(this);
        return this;
    }

    public Album removePhoto(Photo photo) {
        this.photos.remove(photo);
        photo.setAlbum(null);
        return this;
    }

    public void setPhotos(Set<Photo> photos) {
        this.photos = photos;
    }
    // jhipster-needle-entity-add-getters-setters - JHipster will add getters and setters here

    @Override
    public boolean equals(Object o) {
        if (this == o) {
            return true;
        }
        if (!(o instanceof Album)) {
            return false;
        }
        return id != null && id.equals(((Album) o).id);
    }

    @Override
    public int hashCode() {
        return 31;
    }

    // prettier-ignore
    @Override
    public String toString() {
        return "Album{" +
            "id=" + getId() +
            ", name='" + getName() + "'" +
            ", desc='" + getDesc() + "'" +
            ", detail='" + getDetail() + "'" +
            "}";
    }
}

Configuration

• 首先要加入@EnableCaching开始Spring 的Cache
• 其中JHipsterProperties里面就是刚才展示的yml里面的配置,直接在这个Configuration里面 hardcode配置也可以.
• 至于里面的Registration,是Spring Cloud里面的类,他会从Registry里面读取所有的instance.这里面的逻辑是把相同的instanceName,加入同一个hazelcast集群.
• 这样做的前提是
  config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false)
  设置Multicast为false,因为只能有一种发现方式被用.如果你要启用Multicast,就要把
  if (this.registration == null) {
  这里面包括else的代码都comment掉.
• Multicast 会使用广播的方式,在IP段内进行广播,得到响应的instance都会加入集群.
• 代码面使用的是 TcpIpConfig 发现方式.
• 通过 config.getMapConfigs().put() 设置具体配置, 当key为default情况下,设置默认配置.
• com.hc.generated.domain.* 这个key是所有entity的缓存的前缀,所有entity的cache都会使用给该key设置的config.

package com.hc.generated.config;

import io.github.jhipster.config.JHipsterConstants;
import io.github.jhipster.config.JHipsterProperties;

import com.hazelcast.config.*;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.Hazelcast;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.springframework.boot.autoconfigure.web.ServerProperties;

import org.springframework.cache.CacheManager;

import org.springframework.boot.info.BuildProperties;
import org.springframework.boot.info.GitProperties;
import org.springframework.cache.interceptor.KeyGenerator;
import org.springframework.beans.factory.annotation.Autowired;
import io.github.jhipster.config.cache.PrefixedKeyGenerator;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.serviceregistry.Registration;
import org.springframework.context.annotation.*;
import org.springframework.core.env.Environment;
import org.springframework.core.env.Profiles;

import javax.annotation.PreDestroy;

@Configuration
@EnableCaching
public class CacheConfiguration {
    private GitProperties gitProperties;
    private BuildProperties buildProperties;

    private final Logger log = LoggerFactory.getLogger(CacheConfiguration.class);

    private final Environment env;

    private final ServerProperties serverProperties;

    private final DiscoveryClient discoveryClient;

    private Registration registration;

    public CacheConfiguration(Environment env, ServerProperties serverProperties, DiscoveryClient discoveryClient) {
        this.env = env;
        this.serverProperties = serverProperties;
        this.discoveryClient = discoveryClient;
    }

    @Autowired(required = false)
    public void setRegistration(Registration registration) {
        this.registration = registration;
    }

    @PreDestroy
    public void destroy() {
        log.info("Closing Cache Manager");
        Hazelcast.shutdownAll();
    }

    @Bean
    public CacheManager cacheManager(HazelcastInstance hazelcastInstance) {
        log.debug("Starting HazelcastCacheManager");
        return new com.hazelcast.spring.cache.HazelcastCacheManager(hazelcastInstance);
    }

    @Bean
    public HazelcastInstance hazelcastInstance(JHipsterProperties jHipsterProperties) {
        log.debug("Configuring Hazelcast");
        HazelcastInstance hazelCastInstance = Hazelcast.getHazelcastInstanceByName("HC-SAMPLE");
        if (hazelCastInstance != null) {
            log.debug("Hazelcast already initialized");
            return hazelCastInstance;
        }
        Config config = new Config();
        config.setInstanceName("HC-SAMPLE");
        config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);
        if (this.registration == null) {
            log.warn("No discovery service is set up, Hazelcast cannot create a cluster.");
        } else {
            // The serviceId is by default the instance's name
            String serviceId = registration.getServiceId();
            log.debug("Configuring Hazelcast clustering for instanceId: {}", serviceId);
            // In development, everything goes through 127.0.0.1, with a different port
            if (env.acceptsProfiles(Profiles.of(JHipsterConstants.SPRING_PROFILE_DEVELOPMENT))) {
                log.debug("Application is running with the \"dev\" profile, Hazelcast " +
                          "cluster will only work with localhost instances");

                System.setProperty("hazelcast.local.localAddress", "127.0.0.1");
                config.getNetworkConfig().setPort(serverProperties.getPort() + 5701);
                config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(true);
                for (ServiceInstance instance : discoveryClient.getInstances(serviceId)) {
                    String clusterMember = "127.0.0.1:" + (instance.getPort() + 5701);
                    log.debug("Adding Hazelcast (dev) cluster member {}", clusterMember);
                    config.getNetworkConfig().getJoin().getTcpIpConfig().addMember(clusterMember);
                }
            } else { // Production configuration, one host per instance all using port 5701
                config.getNetworkConfig().setPort(5701);
                config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(true);
                for (ServiceInstance instance : discoveryClient.getInstances(serviceId)) {
                    String clusterMember = instance.getHost() + ":5701";
                    log.debug("Adding Hazelcast (prod) cluster member {}", clusterMember);
                    config.getNetworkConfig().getJoin().getTcpIpConfig().addMember(clusterMember);
                }
            }
        }
        config.getMapConfigs().put("default", initializeDefaultMapConfig(jHipsterProperties));

        // Full reference is available at: https://docs.hazelcast.org/docs/management-center/3.9/manual/html/Deploying_and_Starting.html
        config.setManagementCenterConfig(initializeDefaultManagementCenterConfig(jHipsterProperties));
        config.getMapConfigs().put("com.hc.generated.domain.*", initializeDomainMapConfig(jHipsterProperties));
        return Hazelcast.newHazelcastInstance(config);
    }

    private ManagementCenterConfig initializeDefaultManagementCenterConfig(JHipsterProperties jHipsterProperties) {
        ManagementCenterConfig managementCenterConfig = new ManagementCenterConfig();
        managementCenterConfig.setEnabled(jHipsterProperties.getCache().getHazelcast().getManagementCenter().isEnabled());
        managementCenterConfig.setUrl(jHipsterProperties.getCache().getHazelcast().getManagementCenter().getUrl());
        managementCenterConfig.setUpdateInterval(jHipsterProperties.getCache().getHazelcast().getManagementCenter().getUpdateInterval());
        return managementCenterConfig;
    }

    private MapConfig initializeDefaultMapConfig(JHipsterProperties jHipsterProperties) {
        MapConfig mapConfig = new MapConfig();

        /*
        Number of backups. If 1 is set as the backup-count for example,
        then all entries of the map will be copied to another JVM for
        fail-safety. Valid numbers are 0 (no backup), 1, 2, 3.
        */
        mapConfig.setBackupCount(jHipsterProperties.getCache().getHazelcast().getBackupCount());

        /*
        Valid values are:
        NONE (no eviction),
        LRU (Least Recently Used),
        LFU (Least Frequently Used).
        NONE is the default.
        */
        mapConfig.setEvictionPolicy(EvictionPolicy.LRU);

        /*
        Maximum size of the map. When max size is reached,
        map is evicted based on the policy defined.
        Any integer between 0 and Integer.MAX_VALUE. 0 means
        Integer.MAX_VALUE. Default is 0.
        */
        mapConfig.setMaxSizeConfig(new MaxSizeConfig(0, MaxSizeConfig.MaxSizePolicy.USED_HEAP_SIZE));

        return mapConfig;
    }

    private MapConfig initializeDomainMapConfig(JHipsterProperties jHipsterProperties) {
        MapConfig mapConfig = new MapConfig();
        mapConfig.setTimeToLiveSeconds(jHipsterProperties.getCache().getHazelcast().getTimeToLiveSeconds());
        return mapConfig;
    }

    @Autowired(required = false)
    public void setGitProperties(GitProperties gitProperties) {
        this.gitProperties = gitProperties;
    }

    @Autowired(required = false)
    public void setBuildProperties(BuildProperties buildProperties) {
        this.buildProperties = buildProperties;
    }

    @Bean
    public KeyGenerator keyGenerator() {
        return new PrefixedKeyGenerator(this.gitProperties, this.buildProperties);
    }
}

这里可以配置instanceName

eureka:
  instance:
    appname: hc-sample
    instanceId: hc-sample:${spring.application.instance-id:${random.value}}

使用@Cacheable

cacheNames 表示给这个方法的cache定义名字,之后可以通过该name从CacheManager中取出来该name的所有缓存.

可以注解在Repository的方法上,他会把条件当作该条缓存的key,当service调用该方法时,传入相同的参数,并且在缓存时效之类时,会使用缓存进行返回,而不会去call db.当传入不同参数时,会缓存新的结果并且不会删除之前的缓存.

 	@Cacheable(cacheNames = "userByEmail")
    Optional<User> findOneWithAuthoritiesByEmailIgnoreCase(String email);

也可以注解在service的方法上.效果也是一样的,他也会把全部传入参数当成key.

 	@Transactional(readOnly = true)
    @Cacheable(cacheNames = "allStations")
    public Page<StationDTO> findByCriteria(StationCriteria criteria, Pageable page) {
        log.debug("find by criteria : {}, page: {}", criteria, page);
        final Specification<Station> specification = createSpecification(criteria);
        return stationRepository.findAll(specification, page)
            .map(stationMapper::toDto);
    }

可以看到,该类cache名字就是cacheNames定义的allStations,第一条缓存是以某次调用该方法时的传入参数作为key,返回值作为value的Cache.

也可以注解在Controller的方法上.

 	@GetMapping("/stations/{id}")
    @Cacheable(cacheNames = "getStationById")
    public StationDTO getStation(@PathVariable Long id) {
        log.debug("REST request to get Station : {}", id);
        return stationService.findOne(id);
    }

注解在哪个方法上,该方法的上层逻辑不受影响,不管传入参数是否一致,都不会使用缓存,只会影响当前方法以及下层逻辑.因为当使用缓存时,会直接把缓存当成该次方法的返回值返回了,并不会执行该方法内的任何代码.

但是要注意一点,注释的方法的返回值必须是实现了 Serializable 接口的,否则就会报该错.
这是我的handler处理过的response,我是用的ResponseEntity作为返回值,并且注解了Cacheable,就报错了.查看源码后,ResponseEntity确实没有实现Serializable接口.

集群加入时CMD LOG

启动起来之后会看到这段log,后面的this会标注哪一个是该instance,当前集群中只有一个的情况.

这是基于我configuration逻辑时,我启动了同一个microservice的另一个instance情况下,他把instance2也加入了集群中.

使用代码管理Cache

使用cacheManager 进行管理.

//import org.springframework.cache.CacheManager;
	@Autowired
    private CacheManager cacheManager;

这个name就是 @Cacheable(cacheNames = “name”) cacheNames 里面标记的name,可以获取出所有该name下的cache.

this.cacheManager.getCache(name)

这里id是具体的key,会返回出针对该key的缓存.

this.cacheManager.getCache(name).evict(id);

这是清楚该name下的所有缓存.

this.cacheManager.getCache(name).clear()

这是获取所有缓存,并返回一个List.

this.cacheManager.getCacheNames().stream().map(item -> this.cacheManager.getCache(item)).collect(Collectors.toList())

Management Center

Hazelcast 提供了management center,可以在官网上下载对应的版本.
Prod 情况下一般在docker中用image,但是我本地测试就直接下war包了.
https://hazelcast.org/imdg/download/archives/#management-center

我下载的是ZIP.

解压之后 在该目录的cmd里面执行 start.bat [port] [path] 例如 start.bat 8020 hazelcast-mancenter

这时候启动就会在 8020端口 . 这时候在浏览器输入http://localhost:8020/hazelcast-mancenter.填写个默认用户名密码之后就可以进入管理界面.

然后再代码中配置 management-center的 enable url updateInterval .这里面的JHipsterProperties 是从yml取出来的.你也可以直接hardcode在代码里.

    private ManagementCenterConfig initializeDefaultManagementCenterConfig(JHipsterProperties jHipsterProperties) {
        ManagementCenterConfig managementCenterConfig = new ManagementCenterConfig();
        managementCenterConfig.setEnabled(jHipsterProperties.getCache().getHazelcast().getManagementCenter().isEnabled());
        managementCenterConfig.setUrl(jHipsterProperties.getCache().getHazelcast().getManagementCenter().getUrl());
        managementCenterConfig.setUpdateInterval(jHipsterProperties.getCache().getHazelcast().getManagementCenter().getUpdateInterval());
        return managementCenterConfig;
    }

management-center: 
        enabled: true
        update-interval: 3
        url: http://localhost:8020/hazelcast-mancenter

config.setManagementCenterConfig(initializeDefaultManagementCenterConfig(jHipsterProperties));
return Hazelcast.newHazelcastInstance(config);

然后再启动项目 就可以了.

需要注意一点,一个management center只能监控一个集群,如果只有一个center的情况下启动了两个配置了该center的集群,监控里面的数据就会根据配置的UpdateInterval的周期进行变换,一会是 cluster A 的一会是cluster B的.而且一台机器只能启动一个 center 因为他会生成一个 mc.mv.db,作为center的数据库用来储存一开始的账号和密码,如果强行启动两个,就会报下面的错.