Using Java 8


Default Method

Using Optional
- as return value, field (Optional
Optional.ofNullable/orElse(T value)/orElseThrow
ifPresent(Consumer consumer)/orElseThrow(XException::new)

Map + ConcurrentHashMap
ConcurrentHashMap<String, Integer> wordMap = new ConcurrentHashMap<String, Integer>(16, 0.9f, 1);

compute, computeIfAbsent, computeIfPresent
the current (existing or computed) value associated with the specified key, or null if the computed value is null
targets.computeIfAbsent(ticket[0], k -> new PriorityQueue()).add(ticket[1]);
counters.computeIfAbsent(name, k -> new AtomicInteger()).incrementAndGet();
map.computeIfAbsent("key", k -> new LongAdder()).increment();

map.putIfAbsent(sum, i);
Instead of:
if (!map.containsKey(sum)) {
    map.put(sum, i);
}

map.getOrDefault(key, defaultValue)
wordMap.merge(word, occurrence, (key, value) -> wordMap.getOrDefault(key, 0) + value);
wordMap.reduceValues(1,(count1,count2) -> count1+count2);
concurrentHashMap.search(threshold, (k, v) -> v > 1000 ? k : null);

Lambda
iterable.forEach(x -> x.setXXX(null));
removeIf
removeIf is a default method in Collection.
Use removeIf for ArrayList O(n) instead of using Iterator.remove which is O(mn)
list.removeIf(s -> s.length() == 0);


list.replaceAll(s -> s.toUpperCase());
list.sort((x, y) -> x.length() – y.length());
Arrays.sort(intervals, (a, b) -> Integer.compare(a.start,b.start));
Arrays.sort(intervals, (a, b) -> a.start - b.start);
logger.finest(() -> complexMsg());
lineCount = Files.lines(path).count();


Collectors
toImmutable
.collect(collectingAndThen(toList(),Collections::unmodifiableList))

List to map:  
list.stream().collect(Collectors.toMap(x -> x.getId(), x -> x.getName())); - keyMapper, valueMapper
Convert to a different map - TreeMap
.collect(Collectors.toMap(keyMapper, valueMapper,(k,v) ->{ throw new RuntimeException(String.format("Duplicate key %s", k));}, TreeMap::new)); - mergeMapper, mapSupplier

stream().collect(Collectors.groupingBy(xxx));
mapStream.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));

long result=  list.stream().collect(Collectors.counting());

Collectors.joining joins the stream elements for a given delimiter, prefix and suffix. 
String result=  list.stream().collect(Collectors.joining(",","(",")"));

int result = list.stream().collect(Collectors.summingInt(i->i));
Map<String,String> map = streak.collect(Collectors.toMap(k->k, v->v+v));

stream.findFirst().ifPresent(System.out::println); 
streams cannot be reused. The stream is closed after you call any terminal operation.

Word Count:
groupBy aggregate
https://www.javacodegeeks.com/2015/03/jdk-8-streams-and-grouping.html
http://www.nurkiewicz.com/2014/07/grouping-sampling-and-batching-custom.html
Map collect = wordsList.stream().collect(groupingBy(Function.identity(), counting()));

Map map = "abcabc".chars().mapToObj(c -> (char) c).collect(Collectors.groupingBy(c -> c, Collectors.counting()));

Map countMap = Files.lines(file).flatMap(line -> SPLITTER.splitToList(line).stream())
                .collect(Collectors.groupingBy(Function.identity(), Collectors.counting()));

Method and Constructor References
XClass::aMethod, XClass::new
list.sort(String::compareToIgnoreCase);
stream.forEach(System.out::println)

Stream
List to map:
list.stream().collect( Collectors.toMap(Perseon::getId, Perseon::getAge));
toString

flatMap
Combination of map and flatten. The parameter mapper function returns a list of stream, flatMap flatten them into a Stream of single value.
slices.stream().map(slice -> slice.getReplicas()).flatMap(replicas -> replicas.stream())

    .map(replica -> replica.getCoreUrl()).collect(Collectors.toList())

ArrayList.toString() 
ArrayList class extends AbstractList class, which extends AbstractCollection class which contains a toString() method

String.join(", ", list);

Troubleshooting
When compiler complains: incompatible types: cannot infer type-variable(s),try to declare an explicit type to provide enough type information to compiler: 
Map<K1, V1> to Map<K2, V2>
new HashMap<String, ValueClass>().entrySet().stream().collect(Collectors.toMap((final Entry<String, ValueClass> e) -> LocalDate.parse(e.getKey()), e -> e.getValue()));

Java 8 Time API
LocalDate/LocalTime/LocalDateTime - no time zone, good for store birthday
ZonedDateTime
date1.until(date2, ChronoUnit.DAYS),
date1.plusDays(1)

DateTimeFormatter - thread safe
-- Prefer it than SimpleDateFormat

LocalDate.parse("2016-10-06")
LocalDateTime.parse("2016-10-06", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm"))


SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
sdf.setTimeZone(TimeZone.getTimeZone("UTC"));

Convert Date to ZonedDateTime
ZonedDateTime.ofInstant(date.toInstant(), ZoneOffset.UTC)

Month Enum
LongAdder vs AtomicLong

Related
Java Tips, Tricks and Traps

Keep alert of Abnormal things - Problem Solving Skills


In the post Read the Error Message - Problem Solving Skills, we mentioned that the first step of troubleshooting is to read and understand the error message.

During reading the error message/log, it's important to keep alert of unusual things/messages and guess possible causes.

Scenario 1 - In Memory Test User not work in Rest-Assured
After developed the feature Spring Security: Integrate In-Memory Authentication for Test Automation, another developer is using it to write automation test  - using Rest-Assured and reusing the session filter.

I was told that it didn't work and asked to help troubleshoot it.

First step: reproduce the problem
If we run the code that uses rest assured with the test user and session filter, it failed.
But I can use the test user to login admin ui in the browser. - This is weird, I should pay more attention to this unusual thing and think about possible causes, but I didn't at that time.

Then I found the following message in the log:
-- This is wired, as I am using the test user - spring should use the in-memory DaoAuthenticationProvider to verify it; it should only call ActiveDirectoryLdapAuthenticationProvider if it's not the in-memory user.

INFO 144038 [http-nio-0.0.0.0-8080-exec-10] org.springframework.security.ldap.authentication.ad.ActiveDirectoryLdapAuthenticationProvider   - Active Directory authentication failed: Supplied password was invalid

Now I check the code more carefully and found out that it uses wrong username: uses 'admin-123' instead of 'admin_123'. admin-123 is not in-memory user, so it calls ActiveDirectoryLdapAuthenticationProvider for authentication and it failed obviously.

Actually if I read rest-assured logs more carefully, I should have found the root cause early.
00:05:05.553 [main] DEBUG org.apache.http.headers - >> POST /xapp/j_spring_security_check HTTP/1.1
00:05:24.564 [main] DEBUG org.apache.http.wire -  << "HTTP/1.1 302 Found[\r][\n]"
00:05:24.568 [main] DEBUG org.apache.http.wire -  << "Set-Cookie: JSESSIONID=9C03D491F56D6CDC007FA931898F5504; Path=/xapp/; HttpOnly[\r][\n]"

00:05:24.568 [main] DEBUG org.apache.http.wire -  << "Location: http://localhost:8080/xapp/loginerror;jsessionid=9C03D491F56D6CDC007FA931898F5504[\r][\n]"

Eclipse: Be careful what class you import


In the following code, Eclipse shows warning in methodB: The type parameter T is hiding the type T.

private void methodA(final Map<String, T> map) {}
private <T> void methodB(final Map<String, T> map) {}

The root cause is that when write methodA, I made a mistake: I forgot to declare type variable.

As I configured Eclipse Save Action: it will automatically organize imports not declared classes;  in Map<String, T> map, the T is actually: org.apache.poi.ss.formula.functions.T, so methodA actually compiles.

In methodB, the T is actually type variable, so eclipse shows the warning: The type parameter T is hiding the type T.
This is configured in Preferences > Java > Compiler > Errors/Warnings > Type parameter hies another type > Choose Warning

Lesson Learned
During code review, don't ignore imports part.
Make sure what imports is actually we want.
Don't use T as type variables, use E, K, V etc.
Understand the generic syntax.

Mac Tips & Tricks - 2016



Series: Awesome Tips about Mac



Configuration
Use all F1, F2, etc. keys as standard function keys
System Preferences > Keyboard -> Use all F1, F2, etc. keys as standard function keys
Display fn keys by default in Touch Bar
System Preferences → Keyboard → Shortcuts, under Function Keys, add apps to the list to show the function keys by default.

Search in current folder instead of everywhere - link
Go to preference in finder, select "Search the Current Folder" in "When performing a search" and "Advanced" tab 

System Preferences > Keyboard > Shortcuts > All controls
- then we can use tab to move and use space to select the active button

System Preferences > Trackpad > Point & Click > Tap to click

Disable the new touchbar function and have the old F-keys with brightness, volume backlight
- Keybard gt; select "Expanded Control Strip" in the "Touch Bar Shows"

- Keyboard gt; Shortcuts tab gt; "Function Keys" gt; Click the + and add the applications where you want the F-keys to always show

Keychain Access > Manage passwords and certificates

Change default browser
System Preferences > General > Default web browser

Disable Slightly dim the display while on battery power under the Battery tab of Energy Saver.
- Otherwise after plugin power adapter, it will automatically be brighter.

Shortcuts
Go to Folder
/ or Command+Shift+G ->
- Use tab completion

CONTROL + a/e > Got to head/end - work in Terminal, Chrome, Atom

Command+` > cycle between open windows in selected application

xtrafinder support features:
Press Enter or Return to open selection
New File
Copy Path
Copy to, Move to
New Terminal Here

Finder
Command+i > Show info
Command+shift+deelete > Empty the trash

Rename a file or folder
- By default, we can press enter to change file name. 
- If we use XtraFinder and configure it to press Enter to open file, then we can press fn+enter to rename a file.

Close Windows
Undo/Redo:  Command + Z/Y
Command-Shift-W to close a window
Command-Option-W, Close all windows

Zoom out/in
System Preferences > Accessibility > Zoom
Check "Use keyboard shortcuts to zoom":
COMMAND + OPTION + 8 > Toogle Zoom
COMMAND + OPTION + = > zoom out
COMMAND + OPTION + - > zoom in

Check "Use scroll gesture with modifier keys to zoom"
Scroll scroll on the trackpad while pressing one of keys: CONTROL, OPTION, or COMMAND

Most applications supports: 
COMMAND + "+" > zoom out 
COMMAND + "-" > zoom in

Allow or disallow unidentified apps
Ctrl+click or right click then select Open from the contextual menu.

System Preferences > Privacy & Security > General > Allow applications downloaded from


Automatically Hide the Dock
System Preferences > Dock > Automatically hide and show the Dock
Command+Option+D: hide or show dock

Command + H > hide the current application

Force Quit Applications
Using shortcuts: Command + Option + Escape
Using Activity Monitor
Option + click application in the dock

Cmd + Ctrl + D > Dictionary

Activity Monitor

Spotlight Search: Command + Space
Show file location
- Hold Command key
Open file in Finder
- Hold Command key + enter
- Hold Command key + double-click on the result

Exclude folders from Spotlight Index

System Preferences -> Spotlight -> Privacy tab, add folder to exclude from index

Search in current folder rather than everywhere
Open Finder's preference, choose "Search the Current Folder" in Advanced tab

Install/Use GNU tools
brew install coreutils findutils gnu-tar gnu-sed gawk gnutls gnu-indent gnu-getopt
Search Contents of .jar Files for Specific String
gfind . -iname '*.jar' -printf "unzip -c %p | grep -q 'string_to_search' && echo %p\n" | sh



Homebrew
brew doctor
brew update && brew upgrade
brew cleanup     > remove older version
brew cleanup -ns >  Show what will be removed

brew tap caskroom/versions
brew cask install java7

Install GNU Command Line Tools
brew install coreutils
brew install wget

brew cask install iterm2
Reuse previous session's directory
Preferences > Profiles > Default > General > Working Directory > Reuse previous session's directory
⌘ + click to open url or file

Cmd+Shift+H   Paste History
Cmd+K             Clear buffer

Disable swap space
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist
sudo rm -rf /private/var/vm/swapfile*

launchctl
sudo launchctl stop|start com.openssh.sshd

Applications
How to make phone calls on your Mac
Messages 
- send/receive messages from mac
Apple Mail
Use rules to manage your inbox

Outlook
Do a basic search in Outlook: OPTION + COMMAND+ F
Do an advanced search in Outlook: SHIFT + COMMAND+ F
Find text within an item: COMMAND+ F

Install Programs from Unidentified Developers
Preferences > Security & Privacy  General  Allow apps downloaded from
To Manually Launch Application: Holding control key + choose open from dropdown menu

Kill Parental Controls
pgrep parentalcontrolsd 
sudo pkill -9 parentalcontrolsd

Resources
Making Sense of Mac Keyboard Symbols

Spring Security: Integrate In-Memory Authentication for Test Automation


Use Case
To automate test of our web services apis, we need add test users in test environments.

The Solution
We configure Spring Security AuthenticationManager: first add  in-memory DaoAuthenticationProvider in test environments then add other authentication providers.

The legacy code uses xml to configure authenticationManager and intercept-url rules, so I create org.springframework.security.authenticationManager bean. If you are using pure java config, please check Spring Security Java Config.
@Configuration
public class WebSecurityConfiguration {
    private static final String[] ENV_SUPPORT_TEST_USER = {PROFILE_LOCAL, PROFILE_XX};

    @Autowired
    private Environment environment;

    @Bean(name = "org.springframework.security.authenticationManager")
    public AuthenticationManager authenticationManager() {
        final List providers = new ArrayList<>();

        final String env = System.getProperty("env");
        if (StringUtils.isBlank(env)) {
            throw new BusinessException(ErrorCode.INTERNAL_ERROR, "env is empty");
        }
        // The order matters: don't change the order - it will first try to use the test user
        // if not succeed then use ldap
        // add test user for local, q1, e1 only
        if (ArrayUtils.contains(ENV_SUPPORT_TEST_USER, env)) {
            addTestUserAuthProvider(providers);
        }

        if (PROFILE_LOCAL.equals(env) || PROFILE_DOCKER.equals(env)) {
            addLdapProviderForLocal(providers);
        } else {
            // application is deployed to aws, use different LdapAuthenticationProvider
            addLdapProviderForAwsEnv(providers);
        }

        final AuthenticationManager authenticationManager = new ProviderManager(providers);
        return authenticationManager;
    }

    protected void addTestUserAuthProvider(final List providers) {
        final DaoAuthenticationProvider testUserAuthProvider = new DaoAuthenticationProvider();
        final Collection users = new ArrayList<>();
        // add admin user
        UserDetails user = new User(environment.getProperty("spring.security.test.user.adminOnly.name"),
                environment.getProperty("spring.security.test.user.adminOnly.password"),
                Lists.newArrayList(new SimpleGrantedAuthority(environment.getProperty("spring.security.adminGroup"))));
        users.add(user);
        // add provisioner
        user = new User(environment.getProperty("spring.security.test.user.provisionerOnly.name"),
                environment.getProperty("spring.security.test.user.provisionerOnly.password"), Lists.newArrayList(
                        new SimpleGrantedAuthority(environment.getProperty("spring.security.provisionGroup"))));
        users.add(user);

        // add user with admin and provisioner
        user = new User(environment.getProperty("spring.security.test.user.adminProvisioner.name"),
                environment.getProperty("spring.security.test.user.adminProvisioner.password"),
                Lists.newArrayList(new SimpleGrantedAuthority(environment.getProperty("spring.security.adminGroup")),
                        new SimpleGrantedAuthority(environment.getProperty("spring.security.provisionGroup"))));
        users.add(user);

        final UserDetailsService userDetailsService = new InMemoryUserDetailsManager(users);
        testUserAuthProvider.setUserDetailsService(userDetailsService);

        providers.add(testUserAuthProvider);
    }
}

Testing with Rest-Assured
Now we can use Rest-Assured to login and capture the session and reuse it in subsequent requests.
RestAssured.reset();
RestAssured.baseURI = baseURI;
RestAssured.basePath = basePath;
RestAssured.enableLoggingOfRequestAndResponseIfValidationFails();
RestAssured.config = config().logConfig(new LogConfig());
RestAssured.useRelaxedHTTPSValidation();

final SessionFilter sessionFilter = new SessionFilter();
given().auth()
.form(user, passwd,
        new FormAuthConfig("/j_spring_security_check", "j_username", "j_password"))
.filter(sessionFilter).expect().statusCode(200).when().get("/api");

// later...
given().filter(sessionFilter).contentType(ContentType.JSON).when().delete(productUrl).then()
        .statusCode(200);

Searching More Effectively - Programmer Skills


Search is an important skill to programmers, it helps us to quickly find root cause, solutions, resource and etc.

Searching Code
Ctrl+alt+h to check call hierarchy
-- When write code, give class/methods easy-to-search name.
Senario 1 
One colleague asked me to help trouble shoot one problem, he wanted to split part of code from one project, so he copied all XX-related code to another project, but found that it doesn't work.

The error message is that it can't inject XXBean:
void doSomething(@Context XXClass xxBean)
Next I tried to find how old project works. The XXClass is not a spring component, so next I checked who called its constructors, then I found out that the constructor is called in XXClassProvider.

Now the root cause is clear: he forgot to copy the XXClassProvider.
@Component @Provider
public class XXClassProvider extends InjectableProvider<Context, XXClass> {}

-- Another way is to use "Open Type", search "* XXClass*" to find XXClass related class.

Github Search
Check the user guide and search the source code from project's github.
How to searching code in Github
Git - Tips and Tricks
in:path, in:file
path:sub-project/sub-folder
filename:the-name
extension:yml

From How to ignore pojo annotations while using Jackson ObjectMapper? I know in codehause.jackson, we can configure objectMapper to ignore pojo annotation like below:
objectMapper.configure(org.codehaus.jackson.map.SerializationConfig.Feature.USE_ANNOTATIONS, false);

But how to do in Jackson2(fastxml)? So I go to Jackson GitHub, then search USE_ANNOTATIONS:
https://github.com/search?q=org%3AFasterXML+USE_ANNOTATIONS&type=Code
Now it's easy to see that I should use MapperFeature.USE_ANNOTATIONS.

Or we can try Google search: site:grepcode.com USE_ANNOTATIONS
https://github.com/jersey/jersey/search?utf8=%E2%9C%93&q=DisableWADL

Eclipse Search
Avoid duplication
If you think the constant or function defined in your class maybe used in other classes, search the codebase first to avoid duplicate code.

Give classes and methods easy-to-search names.
Command+h          -> search
Command+shift+t -> open type
Command+shift+r -> open resource
Command+o         ->  list all methods of this class
Command+o+o     ->  list all methods of this class including inherited methods
F4 to show all sub classes, then scan related code.

Searching Log
During trouble shoot, it's common that there are 200 + lines in error log that is related: first we can try to use project package name, class name to quickly locate and scan in the error log; if can't, before try other approaches, make sure first quickly scan through all error messages. 
- In most case the root cause is just hidden in the log.
- Check More at: Error creating bean with name 'org.springframework.security.config.authentication.AuthenticationManagerFactoryBean#0'

Command history search
To run command that you have ran before.
history | grep git | grep erase
history | grep solr | grep cd
history | grep solr | grep start

Google search(do some search) first
Google search (do some search) first before attend meeting or join discussion or ask others for help.

Know/Use internal websites
Know company’s internal resource – where to find them
Know some experts (in company you) can ask help from

Labels

adsense (5) Algorithm (69) Algorithm Series (35) Android (7) ANT (6) bat (8) Big Data (7) Blogger (14) Bugs (6) Cache (5) Chrome (19) Code Example (29) Code Quality (7) Coding Skills (5) Database (7) Debug (16) Design (5) Dev Tips (63) Eclipse (32) Git (5) Google (33) Guava (7) How to (9) Http Client (8) IDE (7) Interview (88) J2EE (13) J2SE (49) Java (186) JavaScript (27) JSON (7) Learning code (9) Lesson Learned (6) Linux (26) Lucene-Solr (112) Mac (10) Maven (8) Network (9) Nutch2 (18) Performance (9) PowerShell (11) Problem Solving (11) Programmer Skills (6) regex (5) Scala (6) Security (9) Soft Skills (38) Spring (22) System Design (11) Testing (7) Text Mining (14) Tips (17) Tools (24) Troubleshooting (29) UIMA (9) Web Development (19) Windows (21) xml (5)