Konstantin Ignatyev

Very simple AOP performance test for various frameworks.

Published 1-Jan-2004 - last edition::25-Dec-2004

Summary: This article describes my personal impression from dealing with various AOP frameworks. It has started as simple test for CGLIB based proxy approach, but as I have explored other frameworks I started adding them to this test to estimate convenience of configuring and dealing with them during development cycles, as well as to roughly compare their performance for per-instance Around type advice.
This test does not pretend to be a comprehensive and comparison, just some quick and dirty estimation. It still might be useful because before someone get intimately familiar with a tool he-or-she needs to do an initial quick-'n-dirty test, which might be crucial for framework adoption.

Sources: Download.

If you are eager to see results then jump to the end, but I suggest you reading throught to better understand where those results come from.

Compared frameworks:

CGLIB based home brewed solution;
Nanning;
JBossAOP;
HiveMind;
AspectWerkz v1;
AspectWerkz v2;
AspectWerkz Proxy on steroids;

Test description.

I have defined a very simple interface ( com.kgionline.tests.aop_perf.TestInterface ) and AOP advice will count number of invocations of a TestInterface method.
Because my goal is to evaluate frameworks for using as wrappers around DAO layer singleton classes I will configure Around advice with every tested AOP framework and run the advised instance by AOPTester class.

CGLIB.

It all has started from my experiments with CGLIB based home grown programmatic AOP. At the end of 2003 the approach was most performant, less obtrusive and most transparent than any of available at that time solutions. Well, it stays as most performant, less obtrusive, most transparent, and lightest AOP solution a year later at the end of 2004.

Applying of the method interceptor is very simple, our factory will take care of it by creating instance of our class via Enhancer:

Pros:

no magic;
everything is under strict developer's control;
very performant;

Cons:

All instances have to be instanciated via factory;
Advices have to be applied programmatically, that means that some development time advices are not that easy to apply;

Nanning.

Seem to be unmaintained and very slow AOP framework. I keep it here to show how quickly AOP systems mature. Nanning's use is very similar to the CGLIB

AspectWerkz v1.

NOTE: AspectWerkz v2 has significantly better prformance therefore I suggest using v2 and not to bother with v1. It is here just for history sake. Be sure to to compile Aspectwerkz 2.0 from CVS( There was some terrible bug in RC1 that caused the test run 100 slower!).

After a discussion on TSS I decided to try a full blown AOP system because I would like to experience a full power of AOP and have complete control over runtime system without necessity to code with using Factory pattern. No that I need that right now but it is nice to know how to do it just in case. AOP of choice was AspectWerkz because:

it does not alter Java syntax;

it allows various integration schemas;

comes with fairly good documentation;

have very professional team behind ( they provide even .so and .dll files for JVM hooks :) )

As I mentioned earlier AspectWerkz requires entire JVM worked within AOP system, which is scary but turned out being not that bad. There is (too) many ways to enable AOP for JVM which is kind of frightening for greenhorns like me but simultaneously provide some kind of assurance that entire affair is quite doable. I have decided to go along with #12 - Prepared_bootclasspath. For the sake of clarity I decided to use XML based definitions rather than annotation based because documentation says that those are the same things for runtime AOP engine. Such weaving scheme causes slight penalty of longer startup time but all changes in aop.xml are automatically applied as long as the file is accessible by classloader as META-INF/aop.xml.

Convenience: writing Aspects is simple and very similar to the writing of CGLib interceptor classes.

But we do not have to instantiate our target class with factory, it happens transparently by using primordial AspectWerkz classloader. All we need is to define when and how to apply this Aspect. We do this in the file META-INF/aop.xml:

Now let run our modified AOPTester class within AspectWerkz system: per instructions on AspectWerkz site we have to prepend bootclasspath with our prepared jar file with primordial classloader and append aspectwerkz-core and javassist jars to the bootclasspath.

like this: /usr/java/j2sdk1.4.2_05/bin/java -Xbootclasspath/p:/usr/java/lib/aspectwerkz/dist/aw_enhanced.jar -Xbootclasspath/a:/usr/java/lib/aspectwerkz/lib/aspectwerkz-core-1.0-beta2.jar -Xbootclasspath/a:/usr/java/lib/aspectwerkz/lib/javassist-3.0beta.jar ....

As we may see running our little test within AspectWerkz AOP:

does not affect classes which are not weaved (good thing );
does not affect Proxy based tests ( good thing )- but beware of versions: I ran into problems with some old versions of cglib;

I should also admit that I was wrong about difficulties which I expected debugger will have with weaved code. Debugger (in IntelliJ-IDEA 4.5.1 at least ) works surprisingly well, it gets lost for a moment but then steps into Aspect code, so debugging is possible and relatively easy. All is necessary is to make IDE aware of AspectWerkz system. I used instructions for NetBeans as guide and easily integrated AspectWerkz with IntelliJ-IDEA: all that was necessary is to specify those magic -Xbootclasspath parameters in execute of debug dialog like this:

Conclusion: full blown AOP system is certainly very powerful and almost convenient for use but at this moment I will probably continue to use Proxy based approach with CGLib as more explicit and performant. In the same time I will continue playing with AOP systems an as they quickly mature I am pretty sure that in the near future I will start using one for a real project.

AspectWerkz v2

Simply get AW from CVS, compile it with ant clean dist, prepared new classloader hook, and ran the same test as with AspectWerkx v1. AspectWerkz v2 demonstrates much better performance than v1.

Note(25-Dec-2004): AspectWerkz CVS head might be non-compilable at times, it is sad that AW team policy tolerates non-compillable code in CVS, have they heard about Continuous Integration?

AspectWerkz v2 Proxy

Running entire JVM under AW control seems to be easy, it might scare many people (I am among them) therefore AW team offers very interesting feature: Proxy 'on steroids'. Use wise this feature is very similar to CGLIB based approach but offers nearly full power of mature AOP and declarative advices configuration.

Unfortunately at this time AWProxy is significantly slower than ‘normal’ AW appliance and much slower than CGLIB proxies.

JBoss-AOP

Bill Burke ( JBoss-AOP lead) was kind enough to send me an example of JBoss-AOP bases test case, which I have slightly modified and added to this simple performance and convenience comparison.

JBoss-AOP test is the only test that uses additional build step to weave classes with aspects. I looked at JBoss-AOP documentation and it looks like it is possible to weave classes dynamically I did not figure out how to do it yet. (Send your example or link to the documentation if you know how to do runtime weaving with JBossAOP.) This looks like simple aopc task in Ant build file:

The task runs after compillation, pretty much like JDO postcompillation and I wonder if project becomes total mess of various postcompillations if AOP and JDO used together. ( And do not forget turning off compillation in IDE ).

aopc - is a bit confusing, classpath and src arguments should actually point to classes directory, and aopc does not complain if given src path has no classes at all.

JBoss-AOP interceptor class is very simple and straightforward:

and its binding looksstraightforward as well:

HiveMind

Note: HiveMind is not pure AOP system, it is rather InversionOfControl container with AOP capabilities.

Implementing HiveMind instrumenter is less simple and straightforward because we need to implement factory of instrumenters, but it is not that bad:

Then we define hivemodule registry:

Note: HiveMind is not true AOP system and more resemble my belowed cglib based approach because HiveMind requires us to instantiate our objects via registry calls like this:

Registry registry = RegistryBuilder.constructDefaultRegistry();
TestInterface ti = (TestInterface) registry.getService( TestInterface.class );
AOPTester.runtTests( "Hive Mind Instrumented call", ti );

The difference from CGLib based approach: we MUST have Interface and Implementation with HiveMind. CGLib allows instrumenting classes directly, which is more convenient. ( Note: having interface is a GOOD thing but might be omitted sometimes). I really wish HiveMind uses Proxy pased instrumentation and allowed using classes directly because it is 'simplest thing that works' and Mr.Lewis Ship many times said that coding should be as painless as possible I hope it will happen some day. Another annoying thing: constructor parameters have to be declared and specified in the exact sequence that matches constructor signature. Well, again, I do not see why do not adopt Pico's approach and by default use the greediest constructor that might be supplied by available services. Very often there is no ambiguity and such approach works really well.

Performance wise JBoss-AOP and HiveMind are very close and it is no wonder because they use exactly the same underlying technology: Javassist. I would speculate that difference in numbers might be caused by different Javassist versions: HiveMind uses 2.6 and JBoss-AOP uses 3.0RC1.

Results

all:
     [echo] Using Aspectwerkz 1.0
     [java] AspectWerkz - INFO - Pre-processor org.codehaus.aspectwerkz.transform.AspectWerkzPreProcessor loaded and initialized
     [java] Direct::    Test cycle took:000021   ms with one call takes: .000021 ms. Counter:1,000,000
     [java] CGLib::     Test cycle took:000135   ms with one call takes: .000135 ms. Counter:1,000,000
     [java] Nanning::   Test cycle took:001361   ms with one call takes: .001361 ms. Counter:1,000,000
     [java] AW v1::     Test cycle took:001771   ms with one call takes: .001771 ms. Counter:1,000,000
     [java] Hive Mind:: Test cycle took:000480   ms with one call takes: .000480 ms. Counter:1,000,000
     [java] JBoss AOP:: Test cycle took:000341   ms with one call takes: .000341 ms. Counter:1,000,000
     [echo] Using Aspectwerkz 2.0RC3
     [java] AspectWerkz - INFO - Pre-processor org.codehaus.aspectwerkz.transform.AspectWerkzPreProcessor loaded and initialized
     [java] AW v2::     Test cycle took:000265   ms with one call takes: .000265 ms. Counter:1,000,000
     [java] AspectWerkz - INFO - Pre-processor org.codehaus.aspectwerkz.transform.AspectWerkzPreProcessor loaded and initialized
     [java] AWProxy ::  Test cycle took:000823   ms with one call takes: .000823 ms. Counter:1,000,000

Conclusion:

AspectWerkz v2 is a clear winner among pure AOP systems.

CGLIB - still fastest and simplest way to apply AOP principles.

HiveMind - my personal favorite because it provides good set of IoC and AOP features. In my new projects I use it instead of CGLIB based approach, but have no desire to refactor existing projects because CGLIB solution is time-tested and works very dependably.

Happy coding,

25-Dec-2004

Konstantin