Stake Five

Now that I’ve covered the server-side JSON support we have in place, it’s time to talk about something a little different - the client-side tools that CF8 provides to leverage the stuff we’ve put in on the server.

If you’re a CF developer, you’d have to be encased in carbonite not to have heard that CF8 adds a slew of new tags and functions to help you build AJAX applications. The first of these to be built was the CFAJAXPROXY tag.

Once automatic JSON<->CFML serialization and deserialization was built into CF’s remote CFC invocation layer, the next logical step was to provide a mechanism that would allow invocation of CFC functions remotely from a web browser in the simplest manner possible. The easy way out would have been to just write a JavaScript function that works like CFINVOKE, perhaps something like this:

ColdFusion.invoke(<Path to CFC>, <Function>, <Arguments>)

We did consider it, but it just didn’t feel natural enough… Continue Reading »

In my last post, I looked at the new JSON functions available in ColdFusion 8, and showed how they could be used to consume JSON web services. Today I’ll look at how CF8 lets you expose your applications as JSON web services.

Even with just the CF8 JSON functions, it is easy to create JSON web services - consume arguments from the caller (in the URL or FORM scopes) in JSON format, deserializeJSON() those to CFML types, run your logic on the CFML objects, then serializeJSON() your return type to get the data back to the caller in JSON format. Easy to do, from a CFC or a CFM.

But with our credo of making hard things easy (and easier!) that just wasn’t good enough.

Lost In Translation?

I’ve always liked the notion that the transformation from and to data formats for communication should be independent of application logic. Taking that a step further, I like to think that application designers should not need to worry about data formats (well, not too much, anyway!) - the client of a web service should be able to decide the data format in which it wishes to communicate, depending on its capabilities. Perhaps one client likes to talk SOAP web services, another prefers XML, and, perhaps, is willing to play with WDDX, and yet another chooses to chatter away in JSON. Once an application designer decides that a particular bit of logic should be exposed as a web service, why should it matter what format the data is exposed as?

RESTful Web Services

Before I get into what we’ve done with CF8 in this space, a little background first - did you know you could invoke CFC functions over HTTP? You should - to my mind, it’s one of the coolest things about CF, which I’ve written about before. Here’s how it works:

http://myhost/my/application/callme.cfc?method=echo&input1=yodel&input2=hey

Simple - form a URL to the location of your CFC, and pass it the function to invoke in the method request parameter (URL or FORM scope). Other request parameters are provided to the CFC as arguments. Of course, you have to be sure that the CFC function you’re invoking is marked access=remote, or CF will not allow you to call it remotely, as a web service. By default, the return format is WDDX - CF will automatically serialize the return value from your CFC to WDDX and return that to the caller. Arguments can be provided to the CFC in complex nested CFML types using the special request parameter argumentCollection, which can take a WDDX-formatted string, which must represent a CFML struct, and is deserialized to form the arguments scope for the CFC function to be invoked.

In the currency of the dynamic web, CF makes it simple to expose CFCs as REST web services. If you are planning to design web services into you applications, and don’t know what REST is, you should - REST, or REpresentational State Transfer, outlines a way of thinking about web services architecture using the mechanisms built into HTTP, rather than reinventing the wheel a la SOAP web services. For more see Roy Fielding’s Ph.D. dissertation (Roy Fielding originated the term, relevant chapter here), and for an easier, more fun, introduction, see Duncan Cragg’s REST dialogues. Personally, I’m a huge fan of REST web services - while SOAP has it’s place, I think it really is needlessly bloated and overly complex for the vast majority of use cases it is applied to.

Talking Tongues

Getting back on topic, CF8 introduces the new returnFormat attribute for HTTP invocations of CFC functions, which allows a caller to specify the format of data they’d like to get back. returnFormat supports the following options:

• wddx - the default, return data as WDDX
• json - serialize the return value from the function to JSON and return it
• plain - do nothing with the return value from the function, return it as-is. Note that the plain return format works only for simple types or XML documents. If you try to return a complex type (struct, array, query) with returnFormat=plain, CF will throw an error, since complex types cannot be returned as-is; they must be serialized to a string representation (WDDX or JSON) to transmit over HTTP. Also note that if you’re returning a XML document, you should remember to set the HTTP response content type appropriately - text/xml for most cases, or as appropriate for the particular XML dialect that you’re returning.

Taking my example above, a user who wants a JSON response would modify the URL like so:

http://myhost/my/application/callme.cfc?method=echo&input1=yodel&input2=hey&returnFormat=json

To flesh things out, CF8 can also accept JSON in the argumentCollection request parameter. For instance:

http://myhost/my/application/callme.cfc?method=echo&argumentCollection={"input1":"yodel","input2":"hey"}&returnFormat=json

For completeness, returnFormat is also available as a CFFUNCTION attribute - if you would rather your remote functions had a default other than WDDX you can set it on your functions, and save callers from have to specify it in their requests.

So what’s it good for?

For starters, you can now invoke CFCs directly from AJAX applications - there any any number of AJAX frameworks, including Spry, jQuery and Prototype, which provide support for consuming data from JSON web services. Oh, and lest we forget, CF8 offers up the cfajaxproxy tag to aid in this effort - but more on that later.

Or, if you buy into the “web as a platform” meme (I know I do!), you could choose to expose public JSON web services from your applications, as do del.icio.us, and many others. I particularly like the fact that services such as del.icio.us, built on the backs of their users’ tag gardening activity, open up the data for others to access and do with as they will. Then there’s the it-makes-commercial-sense approach, with companies such as Amazon leading the way - Amazon Web Services provide an astonishing array of functionality (XML rather than JSON, but that’s besides the point), providing pay-as-you-go services, such as S3, and the eCommerce Service, which allows developers to build their own storefronts on top of Amazon’s catalogues.

Think you could be the next Amazon? CF8 makes the technology end of things easier - by a lot - and it certainly doesn’t hurt that CF8 ups the ante on performance to let you operate at Amazon-scale that much more easily.

When we first decided to include AJAX functionality in CF8, well, that was just about all that we had decided! Which was fun, because it gave me a broad mandate to go out and come up with some crazy ideas, and scary, because I didn’t have a clue what to do about it.

So we started small, and built native support for JSON into the server, with a fast Java JSON parser of our own, which provided the basis for the new JSON functions in CF8:

serializeJSON(<CFML object>, <serializeQueryByColumns>)

deserializeJSON(<JSON string>, <strictTypes>)

isJSON(<JSON string>)

I’m not going to get into the specifics of these functions here - Ben Nadel’s done a great job covering them on his blog. Also, a note, these function signatures are not final - we’re tweaking them a bit (just a bit, applications you write now will not be broken!) for the final release. But more on that later.

JSON has been gaining currency as an alternative to XML for data interchange for a variety of reasons: it’s lighter, easier to read and write, maps well to basic constructs in many languages (such as arrays and structs), and, perhaps most importantly for the dynamic web, plays well with yer average browser’s language of choice, JavaScript.

With just these basic functions, it becomes easy to consume JSON-powered web services, mash them up as you will, and spit them back out again. For instance, here’s a tasty morsel of a CFC I wrote to get data from del.icio.us JSON web services:

<cfcomponent output="false">

	<!---
	Gets del.icio.us tags for a user.
	The web service returns a JSON object, mapping tag names to
	numbers of posts with that tag, which deserializes to a CFML Struct.
	More details of the web service available at:
	http://del.icio.us/help/json/tags
	--->
	<cffunction name="getTags" access="remote" output="false">
		<cfargument name="userid">
		<cfset tagsURL = "http://del.icio.us/feeds/json/tags/#userid#?raw">
		<cfhttp url="#tagsURL#">
		<cfreturn deserializeJSON(cfhttp.fileContent)>
	</cffunction>

	<!---
	Gets posts for a user, optionally for a specified set of tags.
	The web service returns a JSON array of objects, one object per
	post. Each post object has the keys:
	u - URL
	d - description
	t - tags
	This deserializes to a CFML Array of Structs, with one Struct per post.
	More details of the web service available at:
	http://del.icio.us/help/json/posts
	--->
	<cffunction name="getPosts" access="remote" output="false">
		<cfargument name="userid">
		<cfargument name="tags">
		<cfset var tagList = replace(tags, ",", "+")>
		<cfset tagsURL = "http://del.icio.us/feeds/json/#userid#/#tagList#?raw&count=30">
		<cfhttp url="#tagsURL#">
		<cfreturn deserializeJSON(cfhttp.fileContent)>
	</cffunction>

</cfcomponent>

CFHTTP to the service URL, deserializeJSON on the response, and, all of a sudden, you have native CFML types to extract data from. Doesn’t get much easier than that! I’ll show off an AJAX application that I wrote with this CFC in a little while.

Given CF’s ability to consume data from a variety of sources (databases, LDAP servers, FTP, MS Exchange, HTTP, etc.), along with native support for common data formats (XML, and now JSON), and the large variety of web services out there, you’re now limited only by your own imagination in the mashup applications you can build. Go forth and multiply!

Have you ever been stuck in a situation where you needed to find out where a particular Java class was loaded from? While this may not be the kind of problem the average CFer faces, I’m sure there are many gurus who can attest that they’ve been burned with issues of this sort. Robi Sen recently posted a bit of Java code to help solve problems like this, and reminded me of a time just a few months ago when I had to chase down a bizarre bug that turned out to be rooted in exactly this kind of JAR file hell.

On the ColdFusion team, we have a fairly rigorous checkin/build/test process. Every time anyone checks something in to the source control system, the build machine gets it, attempts to do a full build and runs a series of automated regression tests against it, just to make sure that nothing obvious has been broken. Those who break the build have the dubious honour of being named beer boy/girl, until someone else breaks the build, and claims the crown!

This has happened to me several times and, on most occasions, the problem was obvious, embarrassing, and easily fixed. However, in this particular case, I just could not figure out what was going on - I’d made changes to a couple of classes, with class A relying on a new method that I’d introduced in class B, but the build machine would constantly fail, with an error message which made it appear as though the new method on class B just did not exist. The build worked on my machine, and on other engineers’ machines, but constantly failed on the build machine.

It was Rupesh who eventually suggested that, perhaps, class B was getting loaded from somewhere other than it was supposed to be. As matters turned out, he was right. There was a dead JAR file that someone had placed on the build machine with a very old version of class B; the build machine was trying to build against that, rather than the newer version which I had checked in. Once the dead JAR file was removed, everything went through OK, and my beer boy crown passed on to the next hapless build-breaking engineer.

This bit of detective work was made possible by some interesting, albeit not very well known, Java APIs. All Java classes are defined, self-referentially, as instances of the class java.lang.Class. The java.lang.Class instance for a specific Java class can be loaded using java.lang.Class’s forName method. Every class maintains an object called ProtectionDomain, which defines the security access rules for the domain which that class belongs to. ProtectionDomain, in turn, maintains a CodeSource object that points to the location from which the code for that domain was loaded. The one gotcha to keep in mind is that the CodeSource object will be null for code which is loaded by Java’s bootstrap* classloader.

So here’s a bit of CF code which uses these APIs:

<cfset codeSource = createObject("java", "java.lang.Class").forName(url.classname).getProtectionDomain().getCodeSource()>
<cfif not isDefined("codeSource")>
<cfoutput>Code source is not defined; <b>#url.classname#</b> is probably being loaded by Java's bootstrap classloader.</cfoutput>
<cfelse>
<cfoutput>The class <b>#url.classname#</b> was loaded from <b>#codeSource.getLocation().toString()#</b></cfoutput>
</cfif>

Just pass your classname in (be sure to get the uppercase/lowercase characters right! Java’s not as forgiving as CF) with the classname URL parameter, and this little code snippet will tell you exactly where the application server loaded the class from.

There is one caveat to keep in mind, which can be good or bad, depending on where you’re coming from. The example that I’ve provided will show where a class is loaded from, from the perspective of the calling code’s classloader. 99% of the time, this is exactly the behaviour you’ll want, but for those with more advanced requirements, you may want to drop this code into a more appropriate location. This is an important consideration, since Java does allow parallel classloaders, which may load the same class from different locations. All application servers adopt this approach, since it is mandated by the JEE specifications. Sounds abstruse? Ignore this little detail then, it probably doesn’t matter to you anyway.

For those strange times when you have to debug a problem on a production system, or want to make sure that you’re using exactly the same classpath as your application server, you may want to try the approach that I’ve outlined. I hope that it will help you as much as it helped me.

Update: As Dan points out, Robi’s intention was quite different from mine - his code was written to list all classes in a given package, not to find where a class was loaded from.

* The Java bootstrap classloader is the mother of all classloaders, which loads up the Java core libraries. All application servers establish their own byzantine classloader hierarchies under the bootstrap classloader; I’ll navigate that labyrinth another time. For the curious, this theserverside.com article, although a bit dated, will give you an idea of what’s involved.

Sean put up his wishlist for CF recently, and one of the interesting things he was wishing for was the capability to call CF from the command line.

The thing is, you can already do this, and fairly easily. Let’s start by getting the right tool for the job - go pick up cURL, a tool for invoking URLs from the command line. Do note that if you choose to download the HTTPS-enabled version of cURL, you will need to also get the libeay.dll library for SSL support from OpenSSL.

That’s really all you need. You can invoke cURL like so: curl (CFM template URL). The output of the CFM is printed to the stdout, which means it will show up in your DOS window/UNIX shell/whatever. Easy, huh?

To invoke a CFC, you’ll have to use the CFC HTTP invocation syntax, which is simple enough:

http://(hostname):(port)/(context, if JEE deployment)/(path to CFC)?method=(function to invoke)&(argument list)

Responses from CFC functions are returned in WDDX format, unless the function returns XML, in which case the XML is returned as-is. Oh, and do make sure you tag your function with access=remote so that CF will allow remote calls to it.

For example, to invoke a function called echo, which takes an arguments input1 and input2, on a CFC my.application.callme, on a standalone CF deployment, you could do this:

http://myhost/my/application/callme.cfc?method=echo&input1=yodel&input2=hey

For those who think all this is just too hard (yeah, right!), here’s a simple shell script that will do the job of invoking a CFC - edit it to change your serverroot, then invoke it at will, passing in the CFC, function and argument list (in the form arg1=val1&arg2=val2).

serverroot="http://localhost:8500/"

callto=`echo $1 | sed -e 's/\./\//g'`
cffunction=$2
args=$3

url=$serverroot$callto".cfc?method="$cffunction"&"$args

curl $url

For example, if your save your script as callCF.sh:

callCF.sh my.application.callme echo input1=yodel&input2=hey

Apologies to DOS users - the scripting that I have done, I’ve done on UNIX systems. Even when I’m forced to use Windows on the desktop, I always have Cygwin installed so I have all those great UNIX command line tools handy.

Now I hear you thinking - do we really have to have a server running to make all these invocations? Couldn’t we have a nice lightweight runtime that can be invoked on the fly? In theory, yes, of course we could. In practice, absolutely not - CFMX was built on the assumption that it would run within a JEE application server. Ripping out, and rebuilding, all the plumbing would be a prohibitively expensive task.

Well, there’s at least one wish fulfilled that we can all talk about - if only the others were as easy! I cannot, alas, yet tell you how many more of Sean’s, and others’, wishes will be granted for Scorpio… But rest assured, we’re paying attention. And some of those wishes, well, they’ve already been granted.

Update: Dopefly opines, as Sean does, that a gateway would provide for more secure invocation. I don’t entirely agree with them - this really comes down to where you choose to put your security, in a gateway, or in a CFC. It could easily be embedded in a CFC by checking that the CGI.SERVER_NAME is localhost, or by ensuring that the IP address of the caller, CGI.REMOTE_ADDR, is in a whitelist of IP addresses allowed to invoke the CFC.

If it talks like a duck and walks like a duck, then it must be a duck… and, apparently, it’ll waddle a good deal faster than a turkey, or any other non-duck species.

I’ve been meaning to get to the subject of duck typing and performance ever since I saw Sean give his presentation at CFUnited. Unfortunately, the performance aspect of going duck was what caught most people’s attention; as Sean shouts out duck typing is not about performance.

I finally got round to downloading Sean’s source code and ran some tests of my own. Here’s the low-down - performance is affected only when the argument type or return type is specified as a CFC; if it is anything else, performance is about equal to the duck typed equivalents. If you’re typing arguments and return values to structs, arrays, strings, numbers, whatever, duck typing will not give you any performance gains. Apologies all if I’m repeating what’s already been pointed out; I searched around a bit and couldn’t find anyone else commenting on this particular detail.

CFC type checking is always going to be more expensive than checking any other type, since we need to jump through many of the same hoops as we do during createObject() to actually resolve the path to the CFC template - is the path to type being checked relative or absolute? Is it referenced via a mapping? Is the type being checked a supertype of the type passed in? You get the idea; evaluating these kinds of complex relationships is expensive.

So do duck type if your design calls for it, or if you’re having serious performance problems with code that checks for CFC types. Otherwise, make like a monkey and code!

Update: You can download the code I used for testing here. It’s identical to Sean’s code, with the exception of a new typedNative function in performance.cfm, which is used to test for “native” types, i.e., Struct, Array, whatever.

Bert brought up a rather interesting scenario where he needed to force templates to be reloaded into the template cache, since he was generating bits of CFML code at runtime. I suggested that he might also want to take a look at the evaluate() function, since that would probably allow him to solve his problem in a simpler manner.

As has been noted often enough, evaluate() and iif() are not the most performant of functions, since they do need to dynamically compile the expressions that they’re being asked to process.

Evaluate() simply processes the string passed to it as if it were an expression. Iif() is the CF implementation of the Java ternary operator “?” which is exactly what it calls to do its work; this is faster than using <cfif><cfelse></cfif> to achieve the same effect. Once iif() has figured out which expression to evaluate, it simply delegates control to evaluate() to actually process the expression.

Continue Reading »

I recently had a mail exchange with someone who wanted to get scopes by name, and was using undocumented methods on the PageContext to achieve his purpose. While this will work, there is an easier way…

You can do this using the StructGet function. StructGet attempts to gets a value from a named variable path which may traverse several nested structures, for example “my.dog.food”, where both “my” and “dog” are of type Struct; this may even be applied to scopes - “session.dog.food”. As the documentation notes, this function can be dangerous since it will create a path down to a variable if it doesn’t find it, but used carefully it can be incredibly useful.

In this case, the problem can be solved simply by passing the scope name to StructGet:

<cfset myscopename="server">
<cfset myscope=StructGet(myscopename)>

There you go, an abstract technique for getting scopes by name!

Update: The “someone” is none other than Jared, who has blogged about this too.

After my introduction to caching post, Doug Hughes asked whether it would be possible to wrap CFCs in soft references to create a memory-sensitive cache for use in CFML. I answered that it should, in theory, be possible, but that an implementation would have to take care of a few common design issues that occur when dealing with soft references.

Having said that, I sat down to write my own implementation to fill the dull hours between CFUNITED sessions…

Here’s softcache.cfc, a soft reference cache that you can drop into your applications. I have not tested this extensively; which is not to say that I haven’t tested it at all! If you do find any bugs, leave your comments here, and I’ll see what I can do about them. I’ve tried to comment the code as thoroughly as possible, but if there’s any particular point that is unclear, do leave a comment and I’ll try to explain it further.

Disclaimer: I spend most of my time writing Java code - if the CFML I write could be optimized further, do let me know.

Continue Reading »