1 Java Programming: First steps with ClassLoaders {anchor:Java Programming: First steps with ClassLoaders} Lately my interest in ClassLoaders and in how they work has been growing. This has happened because I'm curious by nature always wanting to learn more about stuff I use, and because I've been quite interested in implementing a hot deploy feature. Initially I was thinking in writing a massive single post but I chose to divide it in two posts. This is the first one which is about the java virtual machine class loaders, what they are, how they work and how to customize them. The second post will about hot deploy, what is it, what class loaders have to do with it - actually I'll answer this question at the end of this post - how to achieve it and some actual java code doing the trick. Even though this will be an introduction to the subject - after all I'm just learning this new cool tricks - I hope it will be interesting to everyone. __Java Class Loaders__ A Class Loader is an entity responsible to dynamically load into the java virtual machine (JVM) java bytecode transforming it in an actual java usable objects. In other words, a class loader object is one object that can read ~~class~~ files (java compiled files) and create a java.lang.Class Object from it. {image:img=start/2007-07-18/1/jvmLoaders-small.png|align=float-right|link=space/start/2007-07-18/1/jvmLoaders.png}The JVM default environment contains neither one nor two but actually three Class Loaders (which can be seen in the schematic at the right). Each one of these classes loaders as a different purpose: * The Bootstrap Class Loader, also known as the primordial class loader, is the only one written in native code - this means it's part of the JVM core instead of being an object in the java language - and is responsible to load the JVM core library which are found in ~~JAVA_HOME~~/lib. * The Extension Class Loader, is already a java object - an instance of sun.misc.Launcher$ExtClassLoader - and is responsible to load the extensions in ~~JAVA_HOME~~/lib/ext and any other that is specified in the __java.ext.dirs__ system property. * The System Class Loader is also a java object - instance of sun.misc.Launcher$AppClassLoader - and is responsible to load the classes that are in the classpath. When a new instance of an object is created here's what happens: 1. The ~~loadClass(String name)~~ method is invoked at the thread's context class loader (by default the System Class Loader) 1. The class loader tries different ways to find the class file happening one of two things: * The class __is__ found ** The bytecode is read and provided to the ~~defineClass~~ method ** Every other reference found in that bytecode will be also resolved using the ~~findClass(String name)~~ method ** A Class object is returned * The class __isn't__ found ** The task of loading into the JVM the class is delegates to the class loader's parent. The delegation performs an important role in the loading process, --when a class loader does not find a given class it delegates to its parent until the class is resolved or an exception is thrown-- a class loader always delegates the search of a given class to its parent - there are exceptions though, that will be seen in the 2nd part - and only if the class is not found tries to load it on its own. (correction by vinraj) Once a certain instance of the class loader defines a class, that instance cannot redefine the same class again. Although another class loader, or even another instance of the same class loader, will be allowed to define the class. {image:img=start/2007-07-18/1/instanceDifferences.png|align=float-left} This is an interesting fact about java class loaders, and it happens because the class loaders are part of class identification within the JVM. Normally for a beginner or the average intermediate java programmer a class is identified by it's package and class name, although the JVM also considers the class loader who has loaded it. This means that the JVM will see the same class loaded by different class loaders as actual different classes. Let's use an example - picture in the left - for better understanding: executing ~~D.getClass().equals(E.getClass())~~ would return __true__, although ~~C.getClass().equals(D.getClass())~~ would return __false__ even if C, D and E are instances of the same class file! Even more interesting is that custom class loaders can easily be created by extending the ~~java.lang.ClassLoader~~ class. __Custom Class Loaders__ Why should custom class loaders be created? Sometimes the class loaders that the JVM provides aren't enough. Below are listed some of the reasons that could lead to the creation of a new class loader: 1. Security 1. Non standard ways of retrieving class files 1. Dynamic generation 1. Hot deploy - Security\\ A simple example can be digital signed classes. Let's imagine a secure application where even the classes loaded into the JVM have to be trusted. Every class file would be signed using a public key system and the custom class loader would first check the signature before loading the class into the JVM. \\ \\ - Non standard ways of retrieving class files\\ Due to some strange reason instead of having classes in the file system or as a jar package, there was the need to have all the classes inside a single ~~.tar.bz2~~ compressed file. A custom class loader could simply detect the ~~special~~ compressed files, unpack them on-the-fly and load the classes into the JVM \\ \\ - Dynamic generation\\ This is definitely an interesting idea because it's here where we realised that almost anything can really be done. When I say dynamic generation I mean loading non-existing classes into the JVM, not everyone will have use to this but it's still pretty interesting. While talking with [m4ktub] about class loading an interesting scenario regarding snipsnap was suggested, here's the idea: \\ \\ Snipsnap would have a new snip label called java. It could then be associated with a snip that would only contain java code. A custom class loader that snipsnap would use, could then load that snip, wrap the content in a java file compile it and load the class into the JVM.\\ \\ Definitely an interesting idea, maybe not that useful though! ~~laugh~~\\ \\ - Hot Deploy\\ This is probably one of the main reasons that many reasons that some applications and frameworks create custom class loaders. The idea of hot deploy is to have a class loader that allows to load new bytecode versions of a class in runtime without the need of restarting the application. This topic will be covered in more depth in the next post, where the concept and an actual implementation will be presented. That's all for this first part. Comments, questions and general ideas about this subject are more than welcome. 2007-07-23 09:45:46.919 2007-07-18 23:20:39.509 I happen to know from a good source that Mr. Abrantes has done some interesting research on a concept called FileWatcher. It has very strong ties with the hot deploy concept that he is trying to implement, and I''m pretty sure that should be a teaser for us, code junkies, reading these posts. Just a tease for the next post in the series. Anyway, congratulations. It is a very good introspection into the mechanisms that make up the Java VM. I can honestly say I learned a lot from this. I would like, however, to ask (and I'm not so sure this relates to the subject at hand) about the bootstrap. If I understood correctly, this loads the standard VM classes. How does Java implement the low level operations (down the OS integration path) when dealing at this level? I mean... file handling differs from Windows to Linux... Is this integration done at this level or further down the hierarchy? 2007-07-19 10:08:54.826 2007-07-19 10:08:54.642 I'm glad you've learned something, that's the all point of writing this posts. {quote:jpmsi} I happen to know from a good source that Mr. Abrantes has done some interesting research on a concept called FileWatcher. It has very strong ties with the hot deploy concept that he is trying to implement, and I''m pretty sure that should be a teaser for us, code junkies, reading these posts. {quote} Yes it's true I've implemented a simple file watcher - I actually called file system watcher, since it's monitors a directory in the file system instead of a single file - and the source code along the explanation will be presented in the next post. Like you said it has strong ties with hot deploy, since it's the way we can understan that a class has been modified. {quote:jpmsi} I would like, however, to ask (and I'm not so sure this relates to the subject at hand) about the bootstrap. If I understood correctly, this loads the standard VM classes. How does Java implement the low level operations (down the OS integration path) when dealing at this level? I mean… file handling differs from Windows to Linux… Is this integration done at this level or further down the hierarchy? {quote} Well I still haven't found much information about that so I can't really point you a reference.\\ Although we know that for each OS we'll have a different JVM, being the bootstrap class loader written in native code what probably happens is that it's implementation also reflects integration with the low level directives of the OS. But I'm not sure about that, I'll try to find out about it, and I'll let you know. 2007-07-19 10:25:19.526 2007-07-19 10:25:19.353 I found myself thinking about a plugin architecture for a web application, using that hot deploy feature. I'm pretty sure you already done enough research concerning hot deploy based on the local file system. So here's what I started thinking about: imagine that you instead register with the web application a remote location, where you can get a certain plugin. You could then use a special class loader to load the remote plugin and incorporate it into runtime. Wouldn't that be great? You could even do some pretty neat stuff, such as dependency checking, remote dependency resolution... Even obtaining a plugin directly from CVS/SVN and compile it into runtime! And a simple low priority thread could check for the availability of updates, and present the results to any administrator logged in. What do you think about this? 2007-07-19 11:20:08.638 2007-07-19 11:20:08.465 {quote:jpmsi} (...) imagine that you instead register with the web application a remote location, where you can get a certain plugin. You could then use a special class loader to load the remote plugin and incorporate it into runtime. Wouldn't that be great? You could even do some pretty neat stuff, such as dependency checking, remote dependency resolution… Even obtaining a plugin directly from CVS/SVN and compile it into runtime! {quote} Well technically you can do this, unless you'll need to merge code which won't happen if it's a "production server". The low priority thread would just check for updates and synchronize the code. The class loader would detect changes on the local source files, compile them on the fly and load the new class versions.\\ Probably it would work fine on REST web applicatios although if there's the need to keep state you probably will have problems preserving the object states when loading the new versions. Also interface modifications still wouldn't be allowed, but it would definatly allow for example bug fix on at runtime. We could write such container, isn't it? :-) 2007-07-19 11:59:57.508 2007-07-19 11:59:57.343 Once again, I found myself thinking... Distributing plugins is all fine and stuff... Buy what if I want to assure a certain measure of security, should I choose to install a remotely maintained plugin on my server? Now, I'm pretty sure you know the concept of Java's SecurityManager. Do you know how that relates to classloaders? I mean, is it possible to have certain code, loaded by a certain classloader, run with a specified SecurityManager? Looking forward to ear anyone's thoughts... 2007-07-20 10:00:38.394 2007-07-20 10:00:38.228 {quote:jmpsi} . Do you know how that relates to classloaders? I mean, is it possible to have certain code, loaded by a certain classloader, run with a specified SecurityManager? {quote} First of all, you might wanna take a look at the {link:SecureClassLoader|url=http://java.sun.com/j2se/1.4.2/docs/api/java/security/SecureClassLoader.html}. The main idea is simple, the class loader uses a certain SecurityManager to determine the information about the class to know if that class should loaded or not. Such scenario happens, for example, in applets where they can't actually call any of the classes in the sun package. My suggestion is that you take a look at the Java Security book, which is {link:available online|url=http://www.unix.org.ua/orelly/java-ent/security/index.htm}. I know it's old but it talks about those scenarios. Give special attention to chapters 3,4 and 6. 2007-07-20 13:15:46.439 2007-07-20 13:15:46.18 An enlightening post indeed. I do have a question regarding the following paragraph: "The delegation performs an important role in the loading process, when a class loader does not find a given class it delegates to its parent until the class is resolved or an exception is thrown." As per my understanding, a class loader always delegates the search of a given class to its parent and only if the class is not found tries to load it on its own. This I believe is the fundamental paradigm ensuring security. For example, I cannot define a class called java.lang.Object on my own. Even if I do so, the class loaded will always be the Object class belonging JVM's system jars and not my custom class. Refer the following URL http://www.velocityreviews.com/forums/t148335-what-is-the-difference-between-classforname-and-classloaderdefineclass.html post 6 by Chris Smith. 2007-07-23 07:03:03.176 2007-07-23 07:03:03.016 {quote:vinraj} As per my understanding, a class loader always delegates the search of a given class to its parent and only if the class is not found tries to load it on its own. This I believe is the fundamental paradigm ensuring security. For example, I cannot define a class called java.lang.Object on my own. Even if I do so, the class loaded will always be the Object class belonging JVM's system jars and not my custom class. {quote} Hello vinraj, first of all thank you for your post. And yes you are right, thank you for your correction. I'll correct the entry. \\ I wrote that because at the time I was already thinking what I had done with my "hot deploy custom class loader" - you can read more about it on my latest post {link:Java Programming: Hot Deploy|url=http://www.pabrantes.net/blog/comments/start/2007-07-23/1} - which first would try to find the class by itself in order to be able to reload it (other the parent would already have it defined and the old version would be returned). Once again thank you for the correction and for the post. I'm glad you found it interesting. 2007-07-23 09:43:18.691 2007-07-23 09:38:13.904 Wed Jul 18 23:20:55 WEST 2007 Wed Jul 18 23:21:15 WEST 2007 Wed Jul 18 23:21:08 WEST 2007