Writing High-Performance Application

The Problem
Some features/functions are time consuming or time sensitive, we want to make it as fast as possible.
The following are some tips that I learned form my experience and past mistakes.

Does performance really matter?
Whether the function is time consuming or time sensitive?
If so, do take time to measure performance and optimize it. 

Know the function
What it does? What expensive (db/remote)APIs/Services it calls? How many times it calls?
Can we cache results?

Measure the performance
Use Guava StopWatch to measure time consuming functions and analyze where it spends most of the time. 
Use AspectJ to log slow calls.

Test with real data
If the function handles a lot of data, then test it with a lot of data.

Test with real data can also uncover other issues/bugs in the code.
It would be good if we can get backup of production data to do test; if not, use JMeter, postman runner, newman or write test code to inject data.

Postman already generates code(use OK Http or unirest in java) for request, just introduce some randomness, use threadpool and run it x times.

Change/Optimize only one thing each step, then compare the performance

Use Cache aggressively
For example, to cache query from solr or db, we can split query to non-user related and cache the results, and do user related post filtering in the code.

Parallelization - Avoid Putting things in Series
Use Spring ThreadPoolTaskExecutor
-- Thread creation is expensive, don't create a new thread each time, instead use thread pool.
-- Use common shared thread pool, instead create a new thread pool and kill it after all tasks are done.

ThreadPool Configuration
Know what tasks the pool runs, are they CPU bound or I/O bound?Use different settings for different cases.
Understand how Java ThreadPool works
-- Rules of a ThreadPoolExecutor pool size

1. If the number of threads is less than the corePoolSize, create a new Thread to run a new task.
2. If the number of threads is equal (or greater than) the corePoolSize, put the task into the queue.
3. If the queue is full, and the number of threads is less than the maxPoolSize, create a new thread to run tasks in.
4. If the queue is full, and the number of threads is greater than or equal to maxPoolSize, reject the task.

new threads are only created when the queue fills up, so if you’re using an unbounded queue then the number of threads will not exceed corePoolSize.

Sample Code

@Configuration
public class TaskExecutorConfig implements AsyncConfigurer {
  @Bean @Override
  public Executor getAsyncExecutor() {
      final ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();

      final int cpus = Runtime.getRuntime().availableProcessors();
      final int corePoolSize = cpus * defaultScaleFactor;
      executor.setCorePoolSize(corePoolSize);
      final int maxPoolSize = cpus * maxScaleFactor;
      executor.setMaxPoolSize(maxPoolSize);
      executor.setQueueCapacity(queueCapacity);
      executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());

      executor.setThreadNamePrefix(ASYNC_EXECUTOR_PREFIX);
      executor.initialize();
      return executor;
  }
}

Use CompletableFuture/RX-Java

Use Hystrix to defend failures from third-party services and Fail Fast

Use profile tools(VisualVM)
-- Monitor memory usage and profile cpu
-- Use the method name filter to only show method we care.
This can give us a high level of the code, what methods it calls and how many times etc.