Lessons Learned about Programming and Soft Skills - 2017

How to compare different approaches
- Always think about different approaches (even if you already finished/committed code)

- Don't just choose one that looks good
- List them and compare them
- Always ask you why choose this approache
- Try hard to find problems in your current approach, and how to fix them
For small coding
- Implement them if possible
- Then compare which makes code cleaner, less change etc
Example: Exclude source and javadoc from -jar
APP_BIN=$APP_BIN_DIR/$(ls $APP_BIN_DIR | grep -E 'jarName-version.*jar' | grep -v sources | grep -v javadoc | grep -v pom)

How to quickly scan/learn new classes
Sometimes we need quickly scan/check a bunch of related classes to check how to implement a function, use a method etc
- Check the class's Javadoc
- Check the class signature
- Check main methods:
  - static methods
  - using ctrol+o or outline view
- Check call hierarchy in source code
- Check test code/examples
- Google search code example


When refactor/change the code, also check/change/improve its related code.

Find related doc, check/read the doc carefully.
- Mark/Note the important part of the doc.

For some task, we can use the trial and error approach, just do it, then fix it.
But for some task(production or physical hardware related), it's better to figure out the right way to do it first.

Evaluate the outcome of the action. 
- best/worst outcome

How to implement/work on a feature
- what's goal, what to achieve
- how to test/verify/deploy/enable it in test or production environment easily 
  - useful tricks: dry-run, 
  - able to enable/disable default configuration automatically, but override it manually
- how to measure whether the change makes improvement

Think different/3+/more approaches
Compare them
Don't stop until find a solution that looks good to you
Use tools(notebook, whiteboard)
- Usually we are not happy with the first approach that comes to our minds, and find out a different approach: maybe it's a little better or maybe just be different. In some cases, we stop there: maybe we started to talk it with others or present it (to get others' ideas)
-Example: add Explanation to transform actions

Before ask a question
- Try to Solve it by yourself
- Make sure you read all related code/doc: from top to bottom(quickly, scan but don't ignore any code that may be important)

- Example: NightlyTestRunner

Verify the assumption
- Be aware of the the assumption we or others made in the design or the code.
- Verify whether it's true or not
- Example: one line one record in csv, one-to-one between tms id and bam_id

Alternatives
- Check and realize alternatives
Sometimes, we want A, but maybe B also works and is even better.
- Example: asset letter or account statement

Use tools
Write down in whiteboard or notebook or app
Take a picture now 
- always bring the phone

Check carefully and verify your claim before blame others or think others are wrong
We incline to think others are wrong or made a mistake even if someone told you he/she did that - we made a very brief search and didn't check carefully, then started to think they are wrong.

Prefer to use code to enforce the rule than documentation
Example: all tests must extends XbaseUnitTest.

Make API/feature easier
- to use
- to test/rollback in production (feature flag)

Realize your assumption/decision and verify it first
- Otherwise you may go farther but on the wrong path
Step by step and verify each step

Identify useful info and active quickly (or you may forget about it)
Don't let your past experience affect you
- Try it, it may be different this time
Example: big item delivery

When some thing totally doesn't make sense:
- maybe they are totally different things, you are comparing Apple with Banaba
Example: 12/15, 1/15

Related
Lessons Learned about Programming and Soft Skills - 2016

Lessons Learned about Programming and Soft Skills - 2016

Design schema carefully
The cost of changing/migrating old data is expensive, so it's important to try to make it right at first.
Give meaningful, easy-to-remember name
Use ID(instead of name) to reference other data.

Be schema-less
For example: in solr, only add explicit field if you need to search against it; for all other fields that we are sure we will never search, considering store all as one JSON or binary data field.
So it's easier to add/remove these fields.

Use public APIs as much as possible
-- For instance: when develop restful service: use apis from javax(HttpServletRequest) or javax.ws.rs.core if possible - not apis from com.sun.jersey or org.glassfish.jersey.
-- APIs from private apis is more likely to change.

Ensure You Can Wire On and Off Functions

Always Propose Different Solutions then Compare them
Use List and graph to compare pros and cons

Use Cases Analysis
How to implementation in high level 
What's the effort
Don't over-engineer

Pull form upstream(master) branch frequently
- Always work on latest code 

Build/Try simple workable code first

Solve Problem from Different Angles

What Could Possibly Go Wrong?
-- During coding, deployment

Soft Skills
How to find best solution 
Always try/propose different approaches
Talk with others/team members

Don't get too defensive for your solution
- Be open minded
What important is to:
- find best/easiest/robust solution - good for you to maintain code later
- Learn new things, new ways to tackle problems

Don't (just) try to convince others to take your approach
It's easy that we propose different approaches for feature other developer is working on.
But more important is to compare whether the new approach is better or whether there is other better solution.

Different doesn't means better
This is especially more important for senior engineers, because if you give not-well-thought solution, others may be (forced) convinced to take your approach.

More important is to carefully compare these different approaches in detail
-- How to implement all (important/basic) functions in each approach
-- How to handle new requirements later

Communication
Communicate early and often
Let others know what you are doing and your progress
-- Avoid duplicate work

Be open-minded
Learn aggressively

How to Build Better Common Library - Lesson Learned

Make it easy to use
Convention over configuration
Package common shared configuration in library
When we are using spring to build (common enterprise) library that will be used by other teams, it's better that we include common shared configuration- using @Configuration class or XML and all other needed stuff in the library.
The client just need import the configuration class and add needed properties file. 

It would be great if we also include basic property files such as thelib_common.properties, and client can provide custom property file such as thelib_overwrite.properties to overwrite it.

So all client need to do is just import the configuration class and add properties file if needed.
@Import({TheLibAppConfig.class})

Build client library - so client can only import necessary libs
When build a service that is going to be called by other teams, provide a client library that only includes classes, libs that are going to be used by client.

Usually client only need some model classes to build request and parse response, 

Some times, we found that we have to use xxService-common library, which includes too many classes, 3-rd libs that client is not used at all.

Check what libs imported

Remove unneeded dependencies

Build Sample Client Application to demo how to use it
Build sample application to demonstrate how client should call the service, check the built application whether it includes unneeded dependencies.

Split functions into multiple small libraries instead of Big Library

Don't declare same dependencies multiple times

For example: if the commons module import XX lib, the service module imports common, then don't import XX-lib again in service module.

Evolve the Library

-- To use newer framework: from codehaus.jackson to fastxml.jackson, jersey 1 to jersey 2, old spring to newer spring.

Don't constrain client teams choices because they use your library.

OutOfMemory due to incorrect compareTo

Lesson learned from Defect

OutOfMemory due to incorrect implementation of compareTo

Recently, our application hit out of memory error. Through digging the heapdump generated, we quickly found the suspected class that may cause memory leak.

The class would put one item to TreeMap, and later remove it from the TreeMap when this item is finished.
We found that in the heapdump this TreeMap is huge, and this should not happen.

Add more logs to the class, and then we recreated this problem.
We found one strange thing, when after we put one item to TreeMap, later we remove it from the TreeMap, the treemap.remove(item.key) returns null, strange.

We wrote an example, which would put these items into the treemap, and then call contains(item.key), it returns false, en, very strange.
Then we debug the code, and found out that root cause.

See the code as below, have you seen the problem?
public class ClientID implements Serializable
public int compareTo(Object o)
{
   if (this == o)
    {
       return 0;
   }        
   if (o instanceof ClientID)
    {
       return (id - ((ClientID) o).id);
    }
   return -1;
}

In this code, it compares the two clients by subtracting their id(int).
But the id is a randomly-generated Hexadecimal int. If big negative number(a) subtracts another big negative number(b), a < b, it may overflow, and return a positive number.
So the compareTo method would return 1.

This would cause this item exists in the TreeMap, but TreeMap.contains/get can't find it.
The code should be changed like this:
public int compareTo(Object o)
{
   if (this == o)
    {
       return 0;
   }       
   if (o instanceof ClientID)
    {
           int thatClientId = ((ClientID) o).id;
           return (id == thatClientId) ? 0 : (id > thatClientId) ? 1 : -1;
    }
   return -1;
}

Except the fix above, we also change one TreeMap variable in the class to HashMap - as it doesn't need to be sorted by key, another TreeMap variable to TreeSet, as its key and value are same.

Lesson learned 1:
Ensure correct implementation of compareTo, equals, and hashcode.
In compareTo method, compare their value, not subtract them, this may cause overflow.
Be care of addition/subtraction and other operations of numbers, it may overflow.
Use correct data structure.

Lesson learned 2:
Recreate your problem.
Add more logs to help you recreate your problems.
Write example, prototype to verify your thoughts.
Think and list all possible reasons, order them by possibility, and the ease it can be tested and verified.
Then analyze one by one.

Lesson Learned in Life

Lesson Learned in Life

Sensitive to change, and respond to the change.

Forget to take my wallet

Today when I was taking an afternoon nap, I received one message from my friend to remind me to transfer money to him. So I went out to the bank, but only found that I left my wallet in the office.

So I had to go back, took my wallet and went to bank again, this wasted me more than half an hour.

Lesson Learned

When you want to do anything, check whether you take all you need.

The most efficient way is to form some habits, such that I always put my cell-phone, wallet in my desk, every morning, before I left my room, I would look around my desk to ensure I don't leave anything behind.

In the office, usually I put all my stuff in my bag, so when go out to do some stuff, search my bag, and ensure I take all things needed such as cell-phone, wallet and etc.

Forget to take my Badge

This morning, I got office earlier, and found that I forgot to take my badge.

Why?

Because I usually put my badge in my pocket, but this morning I changed my clothes.

Lesson Learned:

Always take everything out of pocket and put them into my bag, or onto the desk.

Another stupid thing is that when I realized this, I decided to take the lift to security room to take one temporary badge, but on arriving, I immediately realized that it was too early that nobody came to work yet.

Lesson Learned:

Think more.

Before decide to do something, think it more, what to do, when, and how.

Whether it will succeed, why or why not it will succeed.

Lesson Learned at Work

Lesson Learned at Work

Here, I would like to write down the lessons I learned at my work.
Hope I can learn something, bear them in mind, and won't repeat similar mistakes twice.

Misleading API - Calendar.before
The parameter of Calendar is Object, so I mistakenly pass a Date object, and it just always returns false.
public boolean before(Object when)
Lesson Learned:
Ensure to use Calendar as the parameter for these operations.
These before, after APIs in Calendar are really bad.
Design your API wisely.
Add a new field DaysToExpire to User Class.
This value is calculated dynamically at server: Its formula is like this:
DaysToExpire of user = Expiration days of system password setting - {current time - the user's last password change time}

Simple? But there are several things that must be considered.
1. DaysToExpire of user < 0, what value it will show to user?
A value like -30 may confuse user.
2. What if Expiration days of system password setting is 0, means password never expires, what value it will show to user?
3. Compatibility issue
As this value is newly added, we must consider what will happen when client and server are in different version: old client - new server, new client - old server.

Especially, in new client - old server, as old server doesn't include this field, client would always get 0 for DaysToExpire? Is this suitable? If not, what value should be displayed?

Learn and know common problems/issues.
Think and consider more, things are always not easy as they appear.
Event: Innovation Idea Presentation
Need Improve Presentation Skills and Public Speaking Skills are important.
Improve the skills to Speaking without Notes

Improve English Listening and Speaking Skills
Need Improve Time management Skills
Give more time to important thing.

Make well preparation for the important stuff.
Take your time and effort to do important thing.

Be brave and take responsibility
Answer the question as possible as you can.
Lesson 1
We use Google WindowTester Pro to automate UI test for our Eclipse RCP test.

It runs well in my laptop, but can’t run in my colleague computer.
The UI test can be started, but it doesn't replay mouse click event.
We dug it for a while, and at last my colleague brought up that this may be because his left and right mouse buttons was swapped, so we tried to change it back, and it worked.

We both knew the fact he has changed the left and right mouse buttons.
But it took us a long time to realize that it may be the root cause of this problem.

Lesson learned:
Reason logically, and think.
For this kind of problem, think and compare what's the difference between my and his environment, consider all possible reasons, and guess, and exclude them one by one.
Run simple unit tests first
When debug and run unit test for your change, usually multiple test-cases should be covered, design your test cases carefully, categorize them and consider all cases.

And run unit tests that are simple, quick, safe and easy first. After they are passed, run the tests that are slow, hard or even danger later.

Miscs
Use correct jdk in your project; otherwise you may use java methods that don’t exist in JDK used in production environment.