Lesson learned from Defect
OutOfMemory due to incorrect implementation of compareTo
Recently, our application hit out of memory error. Through digging the heapdump generated, we quickly found the suspected class that may cause memory leak.The class would put one item to TreeMap, and later remove it from the TreeMap when this item is finished.
We found that in the heapdump this TreeMap is huge, and this should not happen.
Add more logs to the class, and then we recreated this problem.
We found one strange thing, when after we put one item to TreeMap, later we remove it from the TreeMap, the treemap.remove(item.key) returns null, strange.
We wrote an example, which would put these items into the treemap, and then call contains(item.key), it returns false, en, very strange.
Then we debug the code, and found out that root cause.
See the code as below, have you seen the problem?
public class ClientID implements Serializable
public int compareTo(Object o)
{
if (this == o)
{
return 0;
}
if (o instanceof ClientID)
{
return (id - ((ClientID) o).id);
}
return -1;
}
In this code, it compares the two clients by subtracting their id(int).
But the id is a randomly-generated Hexadecimal int. If big negative number(a) subtracts another big negative number(b), a < b, it may overflow, and return a positive number.
So the compareTo method would return 1.
This would cause this item exists in the TreeMap, but TreeMap.contains/get can't find it.
The code should be changed like this:
public int compareTo(Object o)
{
if (this == o)
{
return 0;
}
if (o instanceof ClientID)
{
int thatClientId = ((ClientID) o).id;
return (id == thatClientId) ? 0 : (id > thatClientId) ? 1 : -1;
}
return -1;
}
Except the fix above, we also change one TreeMap variable in the class to HashMap - as it doesn't need to be sorted by key, another TreeMap variable to TreeSet, as its key and value are same.
Lesson learned 1:
Ensure correct implementation of compareTo, equals, and hashcode.
In compareTo method, compare their value, not subtract them, this may cause overflow.
Be care of addition/subtraction and other operations of numbers, it may overflow.
Use correct data structure.
Lesson learned 2:
Recreate your problem.
Add more logs to help you recreate your problems.
Write example, prototype to verify your thoughts.
Think and list all possible reasons, order them by possibility, and the ease it can be tested and verified.
Then analyze one by one.
