Multi Tiered Caching - Using in-process Cache in front of Distributed Cache

Why Multi Tiered Caching?
  To improve application's performance, we usually cache data in distributed cache like redis/memcached or in-process cache like EhCache. 

  Each have its own strengths and weaknesses:
  In-Process Cache is faster but it's hard to maintain consistency and can't store a lot of data; This can be easily solved when using a distributed cache, but it's slower due to network latency and serialization and deserialization.

  In some cases, we may want to use both: mainly use a distributed cache to cache data, but also cache data that is small and doesn't change often (or at all) such as configuration in in-process cache.

The Implementation

  Spring uses CacheManager to determine which cache implementation to use.

  We define our own MultiTieredCacheManager and MultiTieredCache like below.

public class MultiTieredCacheManager extends AbstractCacheManager {
    private final List<CacheManager> cacheManagers;
    /**
     * @param cacheManagers - the order matters, when fetch data, it will check the first one if not
     *        there, will check the second one, then back-fill the first one
     */
    public MultiTieredCacheManager(final List<CacheManager> cacheManagers) {
        this.cacheManagers = cacheManagers;
    }
    @Override
    protected Collection<? extends Cache> loadCaches() {
        return new ArrayList<>();
    }
    @Override
    protected Cache getMissingCache(final String name) {
        return new MultiTieredCache(name, cacheManagers);
    }
}

public class MultiTieredCache implements Cache {
    private static final Logger logger = LoggerFactory.getLogger(MultiTieredCache.class);

    private final List<Cache> caches = new ArrayList<>();
    private final String name;

    public MultiTieredCache(final String name, @Nonnull final List<CacheManager> cacheManagers) {
        this.name = name;
        for (final CacheManager cacheManager : cacheManagers) {
            caches.add(cacheManager.getCache(name));
        }
    }

    @Override
    public ValueWrapper get(final Object key) {
        ValueWrapper result = null;
        final List<Cache> cachesWithoutKey = new ArrayList<>();
        for (final Cache cache : caches) {
            result = cache.get(key);
            if (result != null) {
                break;
            } else {
                cachesWithoutKey.add(cache);
            }
        }
        if (result != null) {
            for (final Cache cache : cachesWithoutKey) {
                cache.put(key, result.get());
            }
        }
        return result;
    }

    @Override
    public <T> T get(final Object key, final Class<T> type) {
        T result = null;
        final List<Cache> noThisKeyCaches = new ArrayList<>();
        for (final Cache cache : caches) {
            result = cache.get(key, type);
            if (result != null) {
                break;
            } else {
                noThisKeyCaches.add(cache);
            }
        }
        if (result != null) {
            for (final Cache cache : noThisKeyCaches) {
                cache.put(key, result);
            }
        }

        return result;
    }
    // called when set sync = true in @Cacheable
    public <T> T get(final Object key, final Callable<T> valueLoader) {
        T result = null;
        for (final Cache cache : caches) {
            result = cache.get(key, valueLoader);
            if (result != null) {
                break;
            }
        }
        return result;
    }
    @Override
    public void put(final Object key, final Object value) {
        caches.forEach(cache -> cache.put(key, value));
    }
    @Override
    public void evict(final Object key) {
        caches.forEach(cache -> cache.evict(key));
    }
    @Override
    public void clear() {
        caches.forEach(cache -> cache.clear());
    }
    @Override
    public String getName() {
        return name;
    }
    @Override
    public Object getNativeCache() {
        return this;
    }
}

@Configuration
@EnableCaching
public class CacheConfig extends CachingConfigurerSupport {
  @Bean
  @Primary
  public CacheManager cacheManager(EhCacheCacheManager ehCacheCacheManager, RedisCacheManager redisCacheManager) {
      if (!cacheEnabled) {
          return new NoOpCacheManager();
      }
      // Be careful when make change - the order matters
      ArrayList<CacheManager> cacheManagers = new ArrayList<>();
      if (ehCacheEnabled) {
          cacheManagers.add(ehCacheCacheManager);
      }
      if (redisCacheEnabled) {
          cacheManagers.add(redisCacheManager);
      }
      return new MultiTieredCacheManager(cacheManagers);
  }

  @Bean(name = EH_CACHE_CACHE_MANAGER)
  public EhCacheCacheManager ehCacheCacheManager() {
      final EhCacheManagerFactoryBean ehCacheManagerFactoryBean = new EhCacheManagerFactoryBean();
      ehCacheManagerFactoryBean.setConfigLocation(new ClassPathResource("ehcache.xml"));
      ehCacheManagerFactoryBean.setShared(true);
      ehCacheManagerFactoryBean.afterPropertiesSet();

      final EhCacheManagerWrapper ehCacheManagerWrapper = new EhCacheManagerWrapper();
      ehCacheManagerWrapper.setCacheManager(ehCacheManagerFactoryBean.getObject());
      return ehCacheManagerWrapper;
  }

  @Bean(name = "redisCacheManager")
  public RedisCacheManager redisCacheManager(final RedisTemplate<String, Object> redisTemplate) {
      final RedisCacheManager redisCacheManager =
              new RedisCacheManager(redisTemplate, Collections.<String>emptyList(), true);
      redisCacheManager.setDefaultExpiration(DEFAULT_CACHE_EXPIRE_TIME_IN_SECOND);
      redisCacheManager.setExpires(expires);
      redisCacheManager.setLoadRemoteCachesOnStartup(true);
      return redisCacheManager;
  }
}

Misc
  Others things we can do when use multi (tiered) cache in CacheManager:

- We can use cache name prefix to determine which cache to use.
- We can add logic to only cache some kinds of data in specific cache.

TODO
- able to use only Distributed Cache or only in-process Cache