The ColdFusion template cache is an interesting beast; one which taught me a great deal about caching. I’ll be discussing the design of the template cache in this post, along with the rationale behind the approach that was taken, and some explanation of interesting behaviours that the template cache exhibits. Before we venture further, please do read An Introduction to Caching - you’ll need that information to understand the rest of this post.
What is the template cache?
For those who don’t know, the template cache is where ColdFusion stores compiled CFM and CFC templates in memory. When a template is executed for the first time, it is compiled to Java bytecode, and then stored in the template cache. Subsequent requests for the template use the compiled form of the template stored in the template cache, saving the overhead of having to read a file from disk, or even having to recompile a template. Templates are typically recompiled only when the CFM or CFC is modified, though if you’ve configured CF to not store compiled templates to disk, they will be recompiled any time they’re not found in the template cache.
Template cache configuration is managed from the ColdFusion Administrator under Server Settings > Caching. You can manage cache size and a couple of other settings, which I’ll get to later, on this page.
Requirements
The template cache has very specific requirements that call for an interesting combination of caching strategies:
- It should maintain as many templates as possible in memory, so that templates can be retrieved rapidly, and so executed efficiently.
- In case increased traffic to a site causes a spurt in demand for templates across the site, it needs to be able to grow itself beyond its configured size. If it were not to grow itself, then it could end up thrashing as it rapidly moves templates in and out of the cache, to deal with high volumes of requests for different templates, degrading server performance, and possibly even causing a server crash.
- It must not grow itself so much that it causes memory consumption issues for the server as a whole.
- It must shrink in size when demand for templates across ths site reduces.
- It must not shrink below a specified size. If we were to allow the cache to shrink down to zero, even recently used templates would be dropped off. Leaving issues of memory aside, we want to keep at least some minimum number of recently used templates always available so that the server stays responsive to demand.
Sounds confusing? I’ll come back to these requirements as we go along, and then, hopefully, you’ll understand why we need things to be the way they are.
Caching Strategies
To fulfill the requirements I’ve outlined above, the template cache adopts a combination of caching strategies - in a nutshell, it uses a soft cache stacked on top of a LRU cache.
Why this approach? The LRU cache ensures that recently used templates are always maintained in memory for fast access. The soft cache stacked on top of it provides a means for the template cache to grow when it needs to, but ensures that it will never grow out of bounds, allowing the Java garbage collector to keep its size under control.
Contrary to what you might think, the template cache size that is configured via the CF Administrator does not specify the maximum number of templates that the cache may hold. It actually configures the maximum size of the LRU portion of the cache; there will always be at least this many templates available in the cache. As must be obvious by now, the template cache as a whole can grow beyond this configured size in times of high demand, provided memory is available. As demand drops, the template cache size will drop back to the size of the LRU cache.
Tuning the Template Cache
By default, the template cache is configured to hold 1024 templates at a minimum. This by no means indicates that it will always store 1024 templates - if your site has fewer than this number, that’s how many the cache will store. For most, there will never be any reason to change this setting. For those who do play with the cache size, here are some guidelines:
- If you turn it up too high (and you have an equivalent number of templates on your site), you run the risk of causing memory consumption problems in the JVM, since the LRU portion of the template cache will attempt to hold at least as many templates as you specify in memory, with no regard to overall memory consumption.
- If you turn it down too low, there is a possibility that your site could become less responsive as demand peaks, since it will have to load and/or compile templates to get them into the cache.
There are a couple of other settings apart from the LRU cache size you can use to control behaviour of the template cache, both of which will affect overall performance of your server:
- Trusted Cache: Turning this on saves the overhead of having to check whether templates have been modified. As the documentation suggests, this is ideal for production sites which don’t expect to see changes to templates.
- Save class files: Saves compiled template class files to disk. Turning this on saves the overhead of having to recompile templates which have been compiled once already, but dropped out of the cache.
That’s it! Go ahead and play with your template cache; see if you can’t get that server running that little bit smoother…
Simon Horwith | 12-Jul-06 at 4:20 pm | Permalink
Nice post! One thing worth mentioning regarding the ’save class files’ setting is that there’s a performance hit retrieving the class files from disk, perticularly if your code base results in a massive number of compiled classes. I’ve found that most OS/App Server environments have a major performance degredation when there are in excess of around 5,000 compiled class files on disk. Generally, I’ve found it’s better (especially on MX 6.1 and 7+) not to write classes to disk.
Of course, the flip side to this is that without writing the classes to disk, when the server instance(s) restart, all class files have to be recompiled - the one major drawback of persisting in memory alone.
Just thought I’d mention that…
Bob Everland | 12-Jul-06 at 5:44 pm | Permalink
We don’t do a lot of updates to our live enviroment, but if I set it to trusted cache how do I get it to recache my template if there is a change?
Phillip Senn | 12-Jul-06 at 6:07 pm | Permalink
Ray sent me.
Is there any way to subscribe to your blog other than through an RSS reader? I always read my email and hardly ever read my rss feeds.
Claude Betancourt | 12-Jul-06 at 6:21 pm | Permalink
Thanks for the post Aswin.
Do you have further details about how these settings affect performance under Linux? More specifically clustering of 4 or more instances.
charlie arehart | 12-Jul-06 at 8:52 pm | Permalink
Thanks, Ashwin. More detail from Adobe folks is of course always welcome.
But I do think it’s useful to point out also how one can measure and report on whether and when template cache misses occur. There are at least two.
First, it’s reported in the command-line CFSTAT tool as CP/Sec (for cache pops per second), and reports both a current and highwater mark. (Of course, you must enable CFSTAT support in the CF Admin, and the cfstat is in the cfusion/bin or cfusionmx/bin.)
It’s also reported in the Windows Performance Monitor, by way of the ColdFusion/ColdFusion MX “performance object” counter called “Cache Pops/Sec” (again, assuming that you’ve enabled Perfmon support in the CF Admin).
With perfmon’s ability to create logs and alerts, it should be easy for someone to create a mechanism to track if you ever have a non-zero value, which would suggest increasing the template cache size.
Hope that may help someone. :-)
charlie arehart | 12-Jul-06 at 8:59 pm | Permalink
And as a follow-up to Simon’s useful comment, I’d like to point out that the ability to “not to write classes to disk” is configurable in the CFMX Admin’s Caching page as a checkbox called “Save Class Files”. That may seem obvious to some, but since (as Simon mentioned) it’s new as 6.1, it seems worth mentioning.
Stake Five :: Tangled in the Template Cache | 13-Jul-06 at 9:10 am | Permalink
[…] I started writing responses to all the great comments I got from my last post, but those rapidly grew so large and… ummm… tangled that it just seemed more manageable to just write a new post to address all of them! […]
monkeywoo | 19-Jul-06 at 5:39 pm | Permalink
Bob Everland,
You can clear your template cache with some code like this:
createObject(”component”,”cfide.adminapi.administrator”).login(”ohnoyoudont”);
createObject(”component”,”cfide.adminapi.runtime”).clearTrustedCache();
Marc Esher | 08-Sep-06 at 11:37 am | Permalink
Is there an official whitepaper that describes the cache behavior, i.e. demonstrating that CF doesn’t actually access the .cfm file every time it is requested?
We’re trying to move all of our code off of our webservers and onto a central location (SAN), but the network people aren’t letting us because they claim it’ll result in much increased network overhead since “the webservers will have to travel the network every time to fetch the file from the SAN”.
Thanks.
ashwin | 08-Sep-06 at 1:42 pm | Permalink
No whitepaper, as far as I know, but I believe you can find the necessary information in the “Configuring and Administering ColdFusion MX” book that comes in the box with CF.
charlie arehart | 27-Feb-07 at 7:32 am | Permalink
Ashwin, I just observed that while the cp/sec stat is indeed still reported by CFSTAT in CFMX 6 and 7, I was working with a client running CFMX 6 and we couldn’t find it reported in his Perfomon’s CFMX6 performance object. And I couldn’t find it in my Perfmon’s CFMX 7 performance object. Do you know, or can anyone else report, if this is a known limitation in CFMX and above? It’s there in my CF5 performance object. Thanks.
ashwin | 28-Feb-07 at 5:14 am | Permalink
Charlie - I don’t have an answer for your myself, since I haven’t been around long enough. However, a quick poll in the team indicates that cache pops is probably not reported in Perfmon with CFMX.
charlie arehart | 01-Mar-07 at 5:57 am | Permalink
Thanks, but here’s a more pertinent question: would it even matter? Now that I think of it, on rereading your entry, it seems possible that you’re saying that the template cache is never popped, with the new soft cache you describe. So that would be the question to ask: can the template cache ever fill? If not, then there will never be a cache pop.
The real question, then, is simply how does one go about choosing the best size for the template cache? What measure do we have to know if we’ve made it too large or too small?
Also, Ashwin, can you configure your WordPress setup to enable us to choose to receive email notifications of new comments posted here? Since it asks for our email address I assumed it would, but I’m not finding that it does. Thanks
And I’ve seen other info (http://www.bpurcell.org/presentations/usergroups/ColdFusion_Performance_Tuning.pdf) to suggest that if one does increase it over the default of 1024 then one should also double the maxpermsize. Any thoughts on that?
ashwin | 01-Mar-07 at 6:53 am | Permalink
On the contrary, Charlie, the template cache is most definitely popped. The soft cache can grow only as large as the JVM will allow it to; templates that need to get compiled beyond the limits of the soft cache will trigger a cache pop of an older template so they can get in.
The more common case is that the soft cache will grow in size initially, and then shrink whenever the JVM needs memory, causing cache pops again.
Increasing the perm size, as Brandon suggests in his presentation, is probably a good idea if you’re sizing your template cache up. As he also points out, this is a trial and error kind of thing - it is also highly dependent on the size of the templates, and hence the generated class files.
Oh, and I’ll see about getting a Wordpress plugin to automatically send out email for comments.
charlie arehart | 01-Mar-07 at 8:06 am | Permalink
Ah, thanks for that clarification. Ok, so it does them seem we need a way to know when such pops are happening. That counter was the way before. Any thoughts on why it’s not there? And if an oversight, might it be put back in 8? :-)
ashwin | 01-Mar-07 at 8:23 am | Permalink
For CF 8, need I say - server monitoring? ;-)
As to why it’s not there, I don’t have a clue. Perhaps just one of those things that were dropped as we went from CF 5 to MX.
charlie arehart | 01-Mar-07 at 3:40 pm | Permalink
Thanks, Ashwin. Are you saying that the CF8 monitoring *will* offer this as a measure? That’s good to hear. It doesn’t help all those on 6 and 7 who really need a way to measure this statistic.
I’ve not yet tried it (reconfigured my machine and run tests to confirm) but since the CFSTAT still reports it, can anyone reading this indicate if they’ve seen it showing a value when indeed the template cache size is set lower than the number of templates requested?
I offer that (as well as the nod to CF8) to give some consolation to others who may come along here and ask/look for this. Thanks again.