Understanding Java Class Loader

Understanding Java Class Loader

Typically class loaders are arranged in a parent/child hierarchy.

When a class loading request is presented to a class loader, it first asks its parent class loader to fulfill the request.

The parent, in turn, asks its parent for the class until the request reaches the top of the hierarchy.

If the class loader at the top of the hierarchy cannot fulfill the request, then the child class loader that called it is responsible for loading the class.

If it can't load the class, a ClassNotFoundException is thrown by the class loader.

Characteristics

Delegation Model

Class loaders are arranged hierarchically in a tree, with the bootstrap class loader as the root of the tree.

Namespaces

A loaded class in a JVM is identified by its fully qualified name and its defining class loader, so class A defined by class loader X is not same class as class A defined by class loader B. In Eclipse, each plugin has its class loader.

Java default class loaders

Bootstrap class loader

Bootstrap classloader is the parent of all classloaders and loads the standard JDK classes in lib directory of JRE (rt.jar and i18n.jar). All the java.* classes are loaded by this classloader.

Extensions class loader (ExtClassLoader)

Extensions Classloader is the immediate child of Bootstrap classloader. This classloader loads the classes in lib\ext directory of the JRE.

System class loader (AppClassLoader)

It loads the classes and jars specified by the CLASSPATH environment variable, java.class.path system property, -cp or –classpath command line settings. If any of the jars specified in one of the above manner have a MANIFEST.MF file with a Class-Path attribute, the jars specified by the Class-Path attribute are also loaded.

Bootstrap class loader is pretty special, in that it is implemented in native code. All other class loaders are written in Java (apart from some native methods) and extend the java.lang.ClassLoader class.

A class is identified uniquely in the context of the associated classloader.

Two objects loaded by different classloaders are never equal.

Three principles of Classloader operation

Delegation Principle

If a class is not loaded already, the classloaders delegate the request to load that class to their parent classloaders. This delegation continues until the top of the hierarchy is reached and the primordial classloader tries to load the class.

Visibility Principle

Classes loaded by parent classloaders are visible to child classloaders but not vice versa, sibling classloaders cannot see each other’s classes. Requests can only go to a parent class loader; they cannot go to a child class loader.

Uniqueness Principle

When a classloader loads a class, the child classloaders in the hierarchy will never reload that class.

In most implementations getClassLoader() method returns null for the bootstrap class loader.

Classloaders and J2EE

J2EE classloader hierarchy and its implications

In J2EE, each application is packaged as an Enterprise ARchive (EAR). The EAR is a self-contained deployment unit having minimal dependencies on external classes (with the exception of application server classes).

An EAR file is composed of any number of following components: EJB-JAR, WAR, RAR, Dependency JAR, application.xml.

J2EE classloader hierarchy

When the parent classloader is below the System-Classpath Classloader in hierarchy, the child classloader can "see" the class only when it is specified in the manifest file for the child classloader. For instance, if an EJB application wants to reference util-a.jar and util-b.jar, then we can add the following entry into the EJB-JAR’s manifest file, MANIFEST.MF: Class-Path: util-a.jar util-b.jar

Every J2EE EAR gets its own classloader. This is the starting point for isolating application classes from one another. This classloader loads all the dependency libraries in the EAR, both EJB and the web module can see the class loaded by the EAR classloader.

An interesting question arises. Aren’t EJBs packaged as jars? How does the application server know to load the EJBs in a separate classloader? The simple answer is that it doesn’t, unless you tell it so. You have to explicitly specify which of the jars are EJB modules in application.xml – the deployment descriptor for the EAR.

WAR classloaders are the children of EJB classloader. The key difference between these classloaders is that while all EJB modules whether in the single or different EJB-JARs, share the same EJB classloader, each WAR gets its own classloader. All of the WAR classloaders however inherit from the same parent - EJB classloader. The rationale behind this hierarchy is that EJBs contain the core of the business logic and web applications have to "see" them to invoke their business methods. Of course to "see" them the WAR manifest file has to have an entry as shown earlier.

Understanding J2EE Application Server ClassLoading Architectures

WebLogic class loader

One EJB class loader is created as a child of the system class loaders. It is responsible for loading all EJB .jar classes for all EJB .jar files in the .ear.

One web application class loader is created for each .war in the .ear, and each of the web application class loaders is a child of the EJB class loader. The web application class loaders are responsible for loading the classes and jars in the WEB-INF/classes and WEB-INF/lib directories in the corresponding .war.   

.jar files listed in a Manifest Class-Path entry are loaded by the application's EJB class loader. This applies to Manifest Class-Path entries found in EJB .jar files and web application .war files within the .ear.

One advantage of this class loading architecture is the automatic availability of all EJB classes from the web application class loaders. However, classes found in the WEB-INF/classes or WEB-INF/lib directories of a .war are not available to any EJB classes.

Webshpere class loader

A WebSphere extensions class loader

The WebSphere extensions class loader loads the WebSphere Application Server classes that are required at run time. The extensions class loader uses a ws.ext.dirs system property to determine the path that is used to load classes. Each directory in the ws.ext.dirs class path and every Java archive (JAR) file or ZIP file in these directories is added to the class path used by this class loader.

One or more application module class loaders that load elements of enterprise applications running in the server

The application elements can be Web modules, enterprise bean (EJB) modules, resource adapter archives (RAR files), and dependency JAR files. Application class loaders follow Java EE class-loading rules to load classes and JAR files from an enterprise application. The product enables you to associate shared libraries with an application.

Zero or more Web module class loaders

By default, Web module class loaders load the contents of the WEB-INF/classes and WEB-INF/lib directories. Web module class loaders are children of application class loaders. You can specify that an application class loader load the contents of a Web module rather than the Web module class loader.

Class Loader Architecture Comparison – Java, J2EE and OSGi

Java class loader

This kind of tree like class loader supports delegation, visibility and uniqueness principle inherently.

Conflicting classes

You have a program, running on JRE 1.6, it needs latest and greatest xerces 2.9 explicitly. You have bundled the xerces jar with your program, and its included in the classpath. But during runtime, older version of xerces classes are getting loaded irrespective of your settings. You guessed it right! Its because JRE 1.6 ships xerces and it is already loaded by extension class loader. This is the primary problem of hierarchical class loader, at runtime all jars become a long list of files which is searched in a sequential order in the order of their loading.

Explicit dependencies can not be defined

Let’s say your program needs Apache commons-logging, xerces and a long list of popular open source components. When you ship your product, either you have to build the stack yourself and hand it over to the customer or communicate this through a user documentation. The so called logical unit of a java program, the jar, lacks the place holder in its meta-data to include information about its dependency.

Version dependency problem

Problem gets complicated when you explicitly need commons-logging 1.1.1, xerces 2.9 and so on, and your program is supposed to be part of a larger system which already has these libraries and their versions are not same as yours. You will have sleep less night in recompiling your program with those libraries to co-exist. Phew !! I have been through this many times.

Entire jar is exposed

When you are building java libraries you want the user to use only few classes which are meant be used. But today Java doesn't have any mechanism to define access specifiers at a class level. Once the jar is in the classpath all core classes can be seen and instantiated.

J2EE class loader

Class loader remains hierarchical but with support for isolation across multiple applications in the same container.

Application A, lets say an ear, has its own class loader and has loaded war and sar inside it. Another Application B, another ear, has its own class loader and can not see Application A's classes. Now if you have a need to share classes between these two applications, either you have to make them use the same class loader or push the shared classes up in the hierarchy to system class loader. By doing this the entire container will be able to see those shared classes even though other applications doesn't necessarily need them. JBoss has introduced unified class loader to address the same problem. But you have to be a pro to use that! Usually this is solved by duplicating jars.

OSGi class loader

Class loader in OSGi is a graph, unlike a hierarchical tree structure of Java and J2EE. Each bundle has its own class loader. For example bundle B defines an export list and bundle A imports it. When A requests a class to be loaded which is present in B, immediately call will be delegated to the class loader of B. As there is no long list of classes organized in a hierarchical structure anymore, it proves to be very fast and efficient. The process of resolving dependency based on import and export metadata is called resolution process. Its a complex procedure taken care by the OSGi framework.

Inherent advantage of having this kind of class loader is:

1. Export is defined with a version attribute, where as imports can specify a range of versions. This allows a bundle to say I need a library having a version between x and y.

2. All packages in a bundle need not stick to a version while exporting. Package org.gok can be exported under 0.9 and another package in the same bundle org.goh can be exported as 1.1.

3. Same library having different versions can be there in the application side by side

java.net.URLClassLoader

This class loader is used to load classes and resources from a search path of URLs referring to both JAR files and directories.

It can be used to download classes from other directories or remote server.

Usage

Class loading is fundamental to two of the most compelling features of Java: dynamic linking and dynamic extension.

Dynamic linking allows types to be incrementally incorporated into a running JVM.

Dynamic extension allows the decision as to which types are loaded into a JVM to be deferred until runtime. This means that an application hosted by the JVM can leverage types that were previously unknown, or perhaps did not even exist, when the application was compiled.

Custom class loader

Usually, J2EE servers implement its custom class loader to enable features such as hot redeployment and application independence.

Normally, the class loaded by custom class loader is unknown to current class loader, we can't cast the object to real type, we can refer it as an object or refer it using its interface.

Basic steps to implement a custom class loader

We usually just need override loadClass() or findClass() method.

1.       Check whether the class has already been loaded, if already loaded, just return it.

2.       If not, check to see whether its parent class loader can load this class.

3.       If not, implement the logic to load it by its own.

4.       Define the class (call parent's method directly).

5.       Check whether the class has to be resolved (linked), and in that case, resolve it (call parent's method directly)..

6.       Return the Class object to the caller.

Example:

SimpleClassLoader source code

A Java 2 Network Class Loader

Resource

The basics of Java class loaders

Understanding Network Class Loaders

Java Class Loading: The Basics

Class Loader Architecture Comparison – Java, J2EE and OSGi

Understanding J2EE Application Server ClassLoading Architectures

J2EE Class Loading Demystified

Understanding Java References

J2SE Knowledge Series - Part 2

Understanding Java References

There are four types of references in java, strong reference,

SoftReference, WeakReference and PhantomReference.

If an object has strong reference, garbage collector will never reclaim its storage.

The latter three reference types provide access to an object but without preventing it from being freed.

It provides a limited degree of interaction with the garbage collector.

SoftReference

The idea of a SoftReference is that your soft references will only be cleared, when system memory is low.

The garbage collector can arbitrarily free an object whose only reference is a soft reference.

This depends on the algorithm of the garbage collector as well as the amount of memory available while the collector is running.

The garbage collector is required to release any soft references before throwing an OutOfMemoryException.

Features

Keeps objects alive provided there’s enough memory.

Usage

SoftReference can be used as a memory-sensitive cache, but it can exacerbate low memory conditions.

WeakReference

Weak references are weaker than soft references. If the only references to an object are weak references, the garbage collector can reclaim the memory used by an object at any time(usually on the next collection). There is no requirement for a low memory situation.

Features

Compared with SoftReference, WeakReference will be GC-d by the JVM eagerly.

The weak references would be cleared as soon as no strong or soft refs remain, but usually it may take multiple runs of the garbage collector before it finds and frees a weakly reachable object.

Cleared ASAP, before the finalizer runs.

Not for caching! Use soft references, as intended

Usage

WeakReference are usually used to automatically reclaim resource when last stronger/soft reference is gone.

Most of the WeakReferences use cases needs a Map data structure, so JDK provides WeakHashMap for us to use.

WeakReference is used internally by the WeakHashMap class.

Use Weak Listeners to avoid Memory Leaks

We may forget to unregister a listener when it is no longer used or some, or it may be difficult to determine when to unregister it, this may cause memory leak.

As an option, we can use weak listener to make it automatically unregistered when no strong/soft reference to it.

PhantomReference

A phantom reference is quite different than either SoftReference or WeakReference.

Phantom Reference isn't meant to be used to access the object, but as a signal that the object has already been finalized, and the garbage collector is ready to reclaim its memory.

The PhantomReference class is useful only to track the impending collection of the referring object. As such, it can be used to perform pre-mortem cleanup operations. A PhantomReference must be used with the ReferenceQueue class.

The ReferenceQueue is required because it serves as the mechanism of notification. When the garbage collector determines an object is phantomly reachable, the PhantomReference object is placed on its ReferenceQueue.

Calling get() on a PhantomReference always returns null, so you must use it with a reference queue.

A PhantomReference is enqueued after finalization of the object. A WeakReference is enqueued before.

Usage

PhantomReferences can be used to determine exactly when an object is about to be removed from memory

Phantom references relate to pre-mortem cleanup tasks.

PhantomReferences can be used as a replacement of finalize() method. - Lets you clean up after finalization but before the space is reclaimed.

Problem with Finalization

Finalization is unpredictable and nondeterministic process - They're not guaranteed to run, especially not timely.

Finalizations can slowdown an application.

   - They can make allocation/reclamation 430X slower!(Effective Java)

   - Finalizers have an impact on the performance of the garbage collector since Objects with finalizers are slow to garbage collect, especially when the finalize method may run for a long time.

Undefined threading model; they can run concurrently, this can cause potential problems.

Exceptions thrown are ignored (per spec).

You should only use finalization only when it is absolutely necessary.

It can be used as a safety measure, but not necessary.

WeakHashMap

WeakHashMap associates key objects with values, it keeps weak refs to keys, strong refs to values. However, once the key object becomes inaccessible via stronger references it becomes eligible for garbage collection. When it is freed, the map entry magically disappears. The assumption here is that if you are not using the key anywhere other than in the map you will have no need to look it up, so it should be freed.

It is a storage-saving technique, WeakHashMap allows the garbage collector to automatically clean up the keys and values when there are no strong/soft reference to the keys outside the map.

WeakHashMap wraps the key as a WeakReference automatically.

HashMap is intended to replace Hashtable.

Map types

HashMap

LinkedHashMap

- When you iterate through it, you get the pairs in insertion order, or in least-recently-used (LRU) order.

TreeMap

- Implementation based on a red-black tree. When you view the keys or the pairs, they will be in sorted order (determined by Comparable or Comparator).

WeakHashMap

ConcurrentHashMap

- A thread-safe Map which does not involve synchronization locking.

IdentityHashMap

- A hash map that uses == instead of equals( ) to compare keys. Only for solving special types of problems; not for general use.

Resources

Thinking in Java (4th Edition)

Swing: Use Weak Listeners to avoid Memory Leaks

Using WeakHashMap for Listener Lists

Understanding Weak References

Java Basics

The Get and Put Principle
- "Producer Extends, Consumer Super"
- use an extends wildcard when you only get values out of a structure, use a super wildcard when you only put values into a structure, and don’t use a wildcard when you both get and put.
public static <T> void copy(List<? super T> dest, List<? extends T> src)

Type erasure
- fix it - pass class type
- no way to find out the runtime type of generic type parameters in Java
- pass the Class of the type parameter
- create(Class<T> type)
new ArrayList<Integer>().getClass() == new ArrayList<String>().getClass(); // true
Difference between ArrayList<? extends T> and ArrayList<? super T>

Unbounded Wildcards - Class<?>
printList(List<?> list)
List<Object> and List<?> are not the same.
- can only insert null into a List<?>.

Upper Bounded Wildcards -- List<? extends Foo> list
- List<Number> is more restrictive than List<? extends Number>
Lower Bounded Wildcards -- List<? super Integer> list

NoClassDefFoundError
- it's an Error - LinkageError
- class was present during time of compilation but not available at runtime when class loader is trying to load it
- error on static initializer block can also result in NoClassDefFoundError.

ClassNotFoundException
- checked exception - must handled in the code
- load a class in runtime using Reflection, such as Class.forName(), ClassLoader.loadClass()

prefer double over float
- float: single (32 bit) precision
- double: double (64 bit) precision

Static method
- static method is always resolved at compile time by using Type of reference variable
- can not access non static member inside static context
- static variables are also not serialized During Serialization

Why String is immutable
String Pool
Multithreading Benefits
Optimization and Performance
- hashcode is cached
HashMap keys

Service provider interface (SPI)
- an API intended to be implemented or extended by a third party
Service Provider
ServiceLoader
- Load drivermanager

Checked vs RuntimeException
- Use exceptions only for exceptional scenarios
- Use checked exceptions for recoverable conditions and runtime exceptions for programming errors
- Use checked exceptions when want to force to client to think how to handle the recoverable exceptional situation. Otherwise use RuntimeException

- Avoid unnecessary use of checked exceptions
- Favor the use of standard exceptions
- If a client can reasonably be expected to recover from an exception, make it a checked exception. If a client cannot do anything to recover from the exception, make it an unchecked exception.
- Don't use Exceptions for control flow
- Use a checked exception for conditions client code can reasonably be expected to handle.

RXjava Exceptions.propagate()
- throw a RuntimeException and Error directly or wrap any other exception type into a RuntimeException.
Thread.setDefaultUncaughtExceptionHandler(handler);

String.CASE_INSENSITIVE_ORDER

Java NIO
Stream Oriented vs. Buffer Oriented
Blocking vs. Non-blocking IO
- only get what is currently available, or nothing at all
- not wait for it to be fully written

Selectors
- an object that can monitor multiple channels for events
Channels
- FileChannel,FileChannel,DatagramChannel,ServerSocketChannel
Buffers
- ByteBuffer, CharBuffer, IntBuffer,LongBuffer,ShortBuffer

Why can't we define a static method in an interface?
Suppose we could do it, if interface A declares methodF, two class B, C both implement interface A. then if we can interfaceA.methodF(), what implementation it should call. JVM just can't figure it out.
Annotations
Annotations (also known as metadata) provide a formalized way to add information to the code so later other tools can easily use these metadata.
Annotations themselves are hardly more useful than comments, more valuable thing is the tools that read and process them.
Annotations are partly motivated by a general trend toward combining metadata with source-code files, instead of keeping it in external documents. In many cases, this can simplify maintenance significantly.
It can provide cleaner looking code, compile-time checking, IDE support, annotation API to get and process these added information.
Annotations are widely used by frameworks such as web services, EJB3, hibernate, that require some sort of additional information to accompany your source code.

Multithread
Why double checked locking doesn't work in Java?
Double checked locking in java is usually implemented like the following:
public class Singleton {
    private Singleton instance;
    public static Singleton getInstance() {
     if (instance == null) {
       synchronized(this) {
         if (instance == null) {
           instance = new Singleton ();
         }
       }
     }
     return this.theGadget;
    }
}
The main reason is that the new operation is not atomic.
At First, one thread pass the null check and tries to initialize the object, because the new operation is not atomic, the process - start allocating and writing the object to the field, contains several writes to memory without guaranteed ordering.
During this period, another thread can come in, see the partially written object.
It would pass the null check, and return the partially allocated object. This can happen with objects, longs type on a 32-bit platform.

ThreadLocal
Definition
ThreadLocal is used to create thread-local variable, each thread will have its own Thread Local variable. One thread can not access/modify other thread’s Thread Local variables, meanwhile they can be accessed from anywhere inside that thread.
ThreadLocal provides get and set accessor methods that maintain a separate copy of the value for each thread that uses it. When a thread calls ThreadLocal.get for the first time, initialValue is consulted to provide the initial value for that thread.
ThreadLocal can be used as an alternative to synchronization to improve scalability and performance. Classes encapsulated in ThreadLocal are automatically thread-safe in a pretty simple way, since it’s clear that anything stored in ThreadLocal is not shared between threads.
When to use ThreadLocal?
ThreadLocal can be used as an alternative to synchronization. In some cases, this can improve scalability and performance significantly.
ThreadLocal can be used to reuse non-trivial objects, just as a resource/object pool.
ThreadLocal is widely used in implementing application frameworks to maintain some context related to the current thread.
For example, J2EE containers use ThreadLocal to associate a transaction context with an executing thread for the duration of an EJB call.
Caveats
It is easy to abuse ThreadLocal by treating its thread confinement property as a license to use global variables or as a means of creating "hidden" method arguments.
Thread-local variables can detract from reusability and introduce hidden couplings among classes, and should therefore be used with care.


Resource
Think in Java (4th Edition)
Oracle JRockit: The Definitive Guide
Thread-local variables in Java
Java Thread Local – How to Use and Code Sample
When and how should I use a ThreadLocal variable?