Multi Tiered Caching - Using in-process Cache in front of Distributed Cache


Why Multi Tiered Caching?
  To improve application's performance, we usually cache data in distributed cache like redis/memcached or in-process cache like EhCache. 

  Each have its own strengths and weaknesses:
  In-Process Cache is faster but it's hard to maintain consistency and can't store a lot of data; This can be easily solved when using a distributed cache, but it's slower due to network latency and serialization and deserialization.

  In some cases, we may want to use both: mainly use a distributed cache to cache data, but also cache data that is small and doesn't change often (or at all) such as configuration in in-process cache.
The Implementation
  Spring uses CacheManager to determine which cache implementation to use.
  We define our own MultiTieredCacheManager and MultiTieredCache like below.
public class MultiTieredCacheManager extends AbstractCacheManager {
    private final List<CacheManager> cacheManagers;
    /**
     * @param cacheManagers - the order matters, when fetch data, it will check the first one if not
     *        there, will check the second one, then back-fill the first one
     */
    public MultiTieredCacheManager(final List<CacheManager> cacheManagers) {
        this.cacheManagers = cacheManagers;
    }
    @Override
    protected Collection<? extends Cache> loadCaches() {
        return new ArrayList<>();
    }
    @Override
    protected Cache getMissingCache(final String name) {
        return new MultiTieredCache(name, cacheManagers);
    }
}

public class MultiTieredCache implements Cache {
    private static final Logger logger = LoggerFactory.getLogger(MultiTieredCache.class);

    private final List<Cache> caches = new ArrayList<>();
    private final String name;

    public MultiTieredCache(final String name, @Nonnull final List<CacheManager> cacheManagers) {
        this.name = name;
        for (final CacheManager cacheManager : cacheManagers) {
            caches.add(cacheManager.getCache(name));
        }
    }

    @Override
    public ValueWrapper get(final Object key) {
        ValueWrapper result = null;
        final List<Cache> cachesWithoutKey = new ArrayList<>();
        for (final Cache cache : caches) {
            result = cache.get(key);
            if (result != null) {
                break;
            } else {
                cachesWithoutKey.add(cache);
            }
        }
        if (result != null) {
            for (final Cache cache : cachesWithoutKey) {
                cache.put(key, result.get());
            }
        }
        return result;
    }

    @Override
    public <T> T get(final Object key, final Class<T> type) {
        T result = null;
        final List<Cache> noThisKeyCaches = new ArrayList<>();
        for (final Cache cache : caches) {
            result = cache.get(key, type);
            if (result != null) {
                break;
            } else {
                noThisKeyCaches.add(cache);
            }
        }
        if (result != null) {
            for (final Cache cache : noThisKeyCaches) {
                cache.put(key, result);
            }
        }

        return result;
    }
    // called when set sync = true in @Cacheable
    public <T> T get(final Object key, final Callable<T> valueLoader) {
        T result = null;
        for (final Cache cache : caches) {
            result = cache.get(key, valueLoader);
            if (result != null) {
                break;
            }
        }
        return result;
    }
    @Override
    public void put(final Object key, final Object value) {
        caches.forEach(cache -> cache.put(key, value));
    }
    @Override
    public void evict(final Object key) {
        caches.forEach(cache -> cache.evict(key));
    }
    @Override
    public void clear() {
        caches.forEach(cache -> cache.clear());
    }
    @Override
    public String getName() {
        return name;
    }
    @Override
    public Object getNativeCache() {
        return this;
    }
}

@Configuration
@EnableCaching
public class CacheConfig extends CachingConfigurerSupport {
  @Bean
  @Primary
  public CacheManager cacheManager(EhCacheCacheManager ehCacheCacheManager, RedisCacheManager redisCacheManager) {
      if (!cacheEnabled) {
          return new NoOpCacheManager();
      }
      // Be careful when make change - the order matters
      ArrayList<CacheManager> cacheManagers = new ArrayList<>();
      if (ehCacheEnabled) {
          cacheManagers.add(ehCacheCacheManager);
      }
      if (redisCacheEnabled) {
          cacheManagers.add(redisCacheManager);
      }
      return new MultiTieredCacheManager(cacheManagers);
  }

  @Bean(name = EH_CACHE_CACHE_MANAGER)
  public EhCacheCacheManager ehCacheCacheManager() {
      final EhCacheManagerFactoryBean ehCacheManagerFactoryBean = new EhCacheManagerFactoryBean();
      ehCacheManagerFactoryBean.setConfigLocation(new ClassPathResource("ehcache.xml"));
      ehCacheManagerFactoryBean.setShared(true);
      ehCacheManagerFactoryBean.afterPropertiesSet();

      final EhCacheManagerWrapper ehCacheManagerWrapper = new EhCacheManagerWrapper();
      ehCacheManagerWrapper.setCacheManager(ehCacheManagerFactoryBean.getObject());
      return ehCacheManagerWrapper;
  }

  @Bean(name = "redisCacheManager")
  public RedisCacheManager redisCacheManager(final RedisTemplate<String, Object> redisTemplate) {
      final RedisCacheManager redisCacheManager =
              new RedisCacheManager(redisTemplate, Collections.<String>emptyList(), true);
      redisCacheManager.setDefaultExpiration(DEFAULT_CACHE_EXPIRE_TIME_IN_SECOND);
      redisCacheManager.setExpires(expires);
      redisCacheManager.setLoadRemoteCachesOnStartup(true);
      return redisCacheManager;
  }
}
Misc
  Others things we can do when use multi (tiered) cache in CacheManager:
- We can use cache name prefix to determine which cache to use.
- We can add logic to only cache some kinds of data in specific cache.

TODO
- able to use only Distributed Cache or only in-process Cache

Labels

adsense (5) Algorithm (69) Algorithm Series (35) Android (7) ANT (6) bat (8) Big Data (7) Blogger (14) Bugs (6) Cache (5) Chrome (19) Code Example (29) Code Quality (7) Coding Skills (5) Database (7) Debug (16) Design (5) Dev Tips (63) Eclipse (32) Git (5) Google (33) Guava (7) How to (9) Http Client (8) IDE (7) Interview (88) J2EE (13) J2SE (49) Java (186) JavaScript (27) JSON (7) Learning code (9) Lesson Learned (6) Linux (26) Lucene-Solr (112) Mac (10) Maven (8) Network (9) Nutch2 (18) Performance (9) PowerShell (11) Problem Solving (11) Programmer Skills (6) regex (5) Scala (6) Security (9) Soft Skills (38) Spring (22) System Design (11) Testing (7) Text Mining (14) Tips (17) Tools (24) Troubleshooting (29) UIMA (9) Web Development (19) Windows (21) xml (5)