FreeMarker is on Twitter

Did you know you can follow FreeMarker project on Twitter?

FreeMarker on Google App Engine

We had several reports of issues in the past with use of FreeMarker on Google App Engine (GAE).

Analysis of these problems confirmed that runtime named "Java" in GAE is not, in fact, a Java-compliant runtime even though it's advertised as such, and looks like one at first sight. It doesn't provide access to some mandatory Java packages, and its reflection implementation has bugs; both of which issues affect FreeMarker.

We wish to stress that FreeMarker is being used in countless Java runtimes, many of them enterprise runtimes with strict security lockdowns, and it works in all of them, as long as these runtimes are conforming Java Runtime Environments. It is our belief that GAE is, unfortunately, non-conforming. Update: I have now reported the reflection bug in the GAE issue tracker.

This unfortunately doesn't help our users who would love to use FreeMarker on GAE. We can not quickly patch FreeMarker as some of the required workarounds would break backwards compatibility within the existing 2.3.x series, which is against our release policy; backwards compatible changes are only allowed when we increment first or second version numbers, that is, from current perspective, either in 2.4 or 3.0.

For this reason, I spent some of my weekend on this particular yak shaving, and have created a new version of FreeMarker's 2.3.16 JAR file that should run on GAE. I'm saying "should" because I don't use GAE myself, so it's up to you, the GAE users to verify it.

If you want to, please go to our project file releases, and grab "freemarker-gae-pre1.jar" from freemarker/2.3.16 directory. The file is labelled "pre1" as it's considered a prerelease; all you people who wish to use FreeMarker on GAE, please start using it and hammer at it, and report back any problems you can find. That said, I sincerely hope there will be none. The changes themselves are fairly light: I removed the dependency on Swing from AST classes, and worked around GAE's reflection security bugs. (If you see a different named file, it means we've updated it since, and please grab that more recent one.)

Update: There is now indeed a "freemarker-gae-pre3.jar"

This doesn't replace our longer term effort of having default FreeMarker be able to run under GAE, but is intended to help our users with an instant solution. There are no downsides to using this version instead of the official 2.3.16 release of FreeMarker, except for the backwards-incompatible change where the TemplateElement class no longer implements Swing TreeNode, but that incompatibility is probably exactly what you want under GAE. All other changes are under the hood - we had to expose few previously package-private implementation classes as being public to work around GAE's reflection security bugs. Since these classes are not documented in FreeMarker official JavaDoc anyway, they don't form the public API, so you shouldn't be using them even when you see them.

FreeMarker 2.3.16 released

FreeMarker 2.3.16 is out! Download it from here.

This is mainly a bugfix release; you can see the full changelog for details.

FreeMarker 2.3.12 released

FreeMarker 2.3.12 is out! Download it from here.

• We fixed one really ugly (and really stupid to make) bug that made the JSP 2.0 SimpleTag support introduced in 2.3.11 immediately unusable (ouch). Everything gets better at second attempt, I guess, so it works now; with my apologies to anyone bitten by it.


• FreeMarker's praised error reporting got even (a bit) better: parse-time exception messages now display the name of the template in addition to line and column number (I too can't believe we didn't have this earlier).


• Variable argument methods (introduced with Java 5) on Java objects can now be properly invoked (in case of overloaded methods, observing the same rules for method resolution that the Java language itself uses).


• Finally, Java 5 enum constants are now identified by their name, not their toString() representation (which can be overridden in a subclass). They still print their toString() representation, of course.

FreeMarker 2.3.11 released

We're pleased to announce that FreeMarker 2.3.11 is now released. This release contains several important bugfixes, performance improvements (both speed and memory), as well as some major feature improvements. It is almost completely backwards compatible with any other 2.3.x version and as such can be used as a drop-in replacement for them. (The only non-backwards compatible change is that ?c built-in formats numbers differently than it did earlier, but that is actually a bug fix).

Download it from SourceForge

You can look at the full list of changes, but I'll summarize it here by topic:

New features

• We now have much better JSP 2.0 and JSP 2.1 compliance. Most notably, JSP tags implementing SimpleTag interface will now work.


• We have better Rhino compliance as well: Rhino objects can be used in templates as scalars, numbers, and booleans, following the JavaScript conversion semantics for these types.


• We created a new interface, TemplateDirectiveModel that should make writing custom directives in Java much easier than with the previous only choice, TemplateTransformModel.

And then there's the small things, i.e. FileTemplateLoader can now load templates through symlinks pointing out of its root directory (it is disabled by default however).

Performance improvements

We eliminated a bunch of repeated instanceof tests in BeansWrapper whenever a new object was wrapped. They're now performed once per class instead of once per object. Also, the FreeMarker wrapper object created by BeansWrapper when wrapping a POJO now does not create an internal HashMap automatically, but only when really needed (and it is not needed often). Thus, we eliminated one HashMap per wrapped object in lots of use cases, noticeably reducing the runtime memory footprint.

Bug fixes

As mentioned, ?c built-in now prints numbers in the expected format always. Also, there is a serious file handle leak when loading resources from JAR files from local filesystem in Sun JVMs, and we now have a workaround for it. FreeMarker will now correctly find JRE-bundled Xalan when run on Java 5 and Java 6. Also, we corrected the lookup of JSP taglib JAR files to be fully compliant with the JSP specification. And there are some more I won't go into right now - see the full changelog page linked above.

Velocity or FreeMarker: Looking at 5 Years of Practical Experience

This month's Javaworld contains an article by Jeroen Van Bergen entitled "Velocity or FreeMarker". While it is, of course, nice to get such good publicity for our project, I have certain misgivings about the article and I feel it is necessary to make certain points about it. I have chosen to do so on this blog because this entry will provide a basic link to refer people to in the future, any time we feel these points need to be made.

Here is my central concern about the aforementioned article: typical readers, not very familiar with the template space, will come away with the idea that Velocity and FreeMarker are broadly comparable in functionality, and that it does not matter very much which one you choose. However, they would be extremely mistaken. There is now five years of practical experience that shows that, in the general case, Velocity is not a sufficiently capable tool for professional use. This is why so many projects -- among them some of the most visible projects in the java world, things like Hibernate, Webwork/Struts 2, and Netbeans -- used Velocity at some earlier stage, but have since migrated to FreeMarker.

Quite amazingly, Mr. Van Bergen's article makes no mention of any of this. So let me start by filling in some of this background.

Max's story about FreeMarker and Velocity

One of the better known cases of a project switching from Velocity to FreeMarker is Hibernate tools. In early 2006, the lead developer, Max Andersen, wrote a blog entry about this that generated some buzz. While Max outlined various issues he had with Velocity, his biggest problem, the central message, was that he simply could not get Velocity to provide useful error messages. If he misspelt a variable somewhere in a template, the default behavior was just to keep going as if nothing happened. Even if he configured Velocity to throw an exception at this point, he had no information about the line number or even the file in which the error occurred.

Now, to be fair, the most recent version of Velocity has some improvements on this front. However, Velocity is still quite weak in error reporting compared to FreeMarker, and real practical experience with both tools shows that the lack of good error messages can be a huge drag on developer productivity.

Error messages were also one of the two stated reasons that the Webwork project, in early 2005, switched from using Velocity by default to FreeMarker. The other was FreeMarker's support for JSP taglibs. Note that Webwork later was donated to the Apache Software Foundation and was renamed "Struts 2". At around this time, Matthew Porter wrote a blog entry describing his switch from Velocity to FreeMarker for all his development.

Now, meanwhile, other people undertook the same migration but for other reasons. Consider Matt Ward's blog entry from mid-2005. One of Matt's main problems with Velocity was that Velocity simply did not address the whole issue of outputting numbers, dates, and currencies, for different audiences, despite the fact that the information to do this is embedded in the core java libraries. Another issue for Matt was Velocity's lack of robust support for macro libraries. FreeMarker supports the notion of namespaces so that different sets of macros can define variables without any fear of them clobbering one another.

More recently, it has come to our attention that Netbeans 6 leverages FreeMarker for templating functionality. (Previous versions were using Velocity.) I have not seen any public explanation of their reasons, but I infer that they were running into various limitations of Velocity, and were looking for a better tool and opted to migrate towards FreeMarker. Consider this blog entry outlining how to adapt to the change to FreeMarker.

What about changes in the other direction?

Now, so far, I have provided examples of people switching from Velocity to Freemarker, but none of anybody going in the opposite direction. Well, feel free to look for examples of this. Google is your friend.

I can only find one example. In this blog entry, Howard Abrams, a long-time FreeMarker user, describes his experience switching to Velocity. Except that he finds Velocity to be too limited, and switches back to FreeMarker!


Final Points

In the foregoing, I highlighted examples of 5 years of practical experience that the java development community has had with Velocity and FreeMarker. Jeroen Van Bergen's article simply omits all of this. While that is surely the central problem, I should still point out other issues with the comparison carried out in the article.

For example, on the last page is a feature checklist chart, showing features that FreeMarker has and Velocity has. FreeMarker allows iteration, Velocity allows iterations, FreeMarker has macros, Velocity has macros. And so on. The chart is outrageously misleading, because it suggests that the two tools are comparable functionally. But that is simply because it omits all the features that FreeMarker has that Velocity lacks!

A more complete list would contain things like:

• Support for transparent i18n with numbers and dates shown according to the relevant locale: FreeMarker, yes. Velocity, no.
  
• Support for JSP taglibs: FreeMarker, yes. Velocity, no.
  
• Support for Jython and Rhino: FreeMarker, yes. Velocity, no.
  
• Special tags for stripping of extraneous whitespace: FreeMarker, yes. Velocity, no.
  
• Support for auto-escaping interpolations on blocks of text (converting problematic characters to HTML entities, for example): FreeMarker, yes. Velocity, no.
  

Also, just consider the statement that FreeMarker has macros and Velocity has macros, with no further elaboration. It is actually true, as far as that goes, but it is certainly quite misleading. Now, it is true that in paragraph below that feature checklist chart, the author does mention that FreeMarker's macro system has some additional capabilities. However, the basic impression given from the way the information is presented is that the difference is at most marginal. Surely, a more complete feature checklist of the tools' respective macro capabilities is called for and that would be something like:

• Support for importing a set of macros (macro library) in a separate namespace to avoid variable naming clashes: FreeMarker, yes, Velocity, no.
  
• Support for macros with an associated template block: FreeMarker, yes, Velocity, no.
  
• Support for optional and default parameters in macro calls: FreeMarker, yes. Velocity, no.
  
• Output of the macro call stack when an error is hit: FreeMarker, yes. Velocity, no.
  

Right below this, Mr. Van Bergen mentions Anakia, which is an add-on to Velocity for processing XML. He neglects to mention that FreeMarker provides similar XML processing functionality (though the implementation is much more complete, since it supports XML namespaces, for example) as part of its core feature set. Declarative XML processing is supported in FreeMarker via the #visit and #recurse directives, which are core directives in the FreeMarker language. One would infer from what the article says that XML processing is a point in favor of Velocity, when, really, quite the opposite is the case. The XML processing functionality available for Velocity is add-ons like Anakia and DVSL that are basically abandonware, where the XML processing support in FreeMarker is a core part of the product, and is clearly supported.

Still, the central problem with this article and the comparison it undertakes is that it ignores this huge body of evidence from real-world experience that all points to a very clear conclusion: quite frequently, people switch to FreeMarker after hitting Velocity's limitations. In Max Andersen's case, it was the poor to nonexistent error reporting, where for other people, it is Velocity's half-baked, rather buggy implementation of macros. And then, in other cases, the deal breaker is Velocity's lack of transparent support for internationalization. But regardless of the reasons in the specific case, it is quite clear that there has been a large migration from Velocity to FreeMarker and virtually no movement in the opposite direction. In the foregoing, I mentioned cases that are highly visible like Netbeans or Hibernate. We can only speculate about how many projects are out there, most of them not visible at all, because they are in a company's closed codebase, that have ended up undertaking a similar migration. In any of these cases, the people involved could have saved themselves a lot of time and effort by opting for FreeMarker in the first place, rather than starting with Velocity, discovering its limitations, and then having to migrate. So, it serves the readers of a publication like Javaworld rather poorly to have articles that foster the misconception that FreeMarker and Velocity are broadly comparable in functionality. They are not. There is an overwhelming body of evidence from real-world praxis that shows this quite clearly.

New features in the next release

We're nearing the release of FreeMarker 2.3.11. You can try out the latest freemarker.jar build right now if you can allow yourself to run experimental code and provide us with feedback. Just download it and drop it in place of your existing FreeMarker 2.3.x JAR file. Here's a nonexhaustive list of new and improved features:

• Probably the biggest news is that we support JSP 2.0 SimpleTag now. More precisely, you can now use JSP custom tags that implement the SimpleTag interface within FreeMarker.


• BeansWrapper's wrapping performance has been significantly improved. Previously, we did a bunch of instanceof checks on each object to figure out how to wrap it. Now, we perform only one lookup per class (instead of per object!), and subsequently quickly dispatch to wrapping code based on the class of the object. ^*


• Memory footprint of object wrappers using BeansWrapper has been reduced; we no longer create a per-object member map for each wrapper until you invoke a method through the wrapper. Property accesses don't count, so you get a reduced memory footprint if you use ${foo.prop} but you don't get it for ${foo.getProp()}. As in majority of cases, people tend to use properties, this has the potential to significantly reduce the load on GC.


• When using Rhino JavaScript objects as data models in template, they are recognized as boolean, number, and string values following the JavaScript conversion semantics to these types.

---

^* People who subclass BeansWrapper and override getInstance(Object, ModelFactory) will need to instead override the new getModelFactory(Class) method to take advantage of performance improvement. Overriding the old method still works (we promise to keep backwards compatibility within a major release), but doesn't realize the performance benefit.