AJAST - Cross-domain REST calls using JSON injection

The typical (and original AJAX) approach to calling web services asynchronously from a browser uses the `XMLHTTPRequest` object to request data asynchronously. However, as most of you probably already know, requests made using this object are restricted to the same domain as the script they originate from. This means that in order to request data from services like Google Maps, Flickr, etc. you need to implement a server-side proxy on your domain to use the `XMLHTTPRequest` object. But what if you want to stay on the client side? Enter JSON injection.

JSON injection, or actually script tag injection, is a rather common technique that circumvents the `XMLHTTPRequest` limitation by dynamically injecting script tags into the calling page. A script tag can have any domain as its source, which means that cross-domain calls are possible. The technique is also referred to as JSON callbacks, although it really is not limited to JSON payloads. The technique is also referred to as JSONP, although [the original JSONP](http://ajaxian.com/archives/jsonp-json-with-padding) is a bit more extensive than just callbacks using script injection.

It really is a neat technique without a cool term. JSONP is a term for a superset of JSON injection. A more precise term than JSON injection would be Javascript injection, but that's already used to describe vulnerabilities in web pages where malicious Javascript code is injected through e.g. links from external sites. I hereby propose the term AJAST - Asynchronous Javascript And Script Tags. At least it'll be the name of my implementation. If it doesn't catch on for anything but that, we'll even be a bit more confused than we already are. Now, what do we require to do AJAST?

AJAST requirements

The requirements an AJAST request lays on the server-side are the following:

1. The server must provide its services through HTTP GET requests.
2. The client must be able to supply the name of a callback function that the response will be wrapped in.
3. The server is expected to provide a response on the form callback(payload), where callback is the name of the callback function supplied by the client, and payload is the payload returned by the server. The payload can be XML, JSON, or any other form of data that the Javascript callback function can accept as a single argument.

These requirements are already fulfilled by many REST services, but they are still hard to use in an AJAST fashion due to client side challenges. The two main requirements for an AJAST library are:

1. Complete handling of requests. Nothing more than a URL and a callback should be needed to create a request.
2. Timeouts is a show stopper for AJAST. With script injection, it is difficult to know if a call completes, and from an AJAST usage perspective it is essentially impossible to create a decent solution without knowing if requests complete or not.

Security is another obvious challenge, although in my view it is a challenge for the Internet in general rather than AJAST in particular. Developers creating cross-domain applications should be aware of the security risks involved, and take measures to prevent security breaches accordingly. There is no silver bullet.

Although several examples for implementing an AJAST request are found around the web, I found no fully functional stand-alone implementations. Dan Theurer's article on script requests provides code that can be used to create an implementation, but leaves the timeout problem unsolved. Toolkits such as Dojo also implement variations on the approach using IFrame requests, but they are a lot more hairy, and I really don't want a framework (or parts of it) bloating the web site I am creating just to be able to do AJAST. I want a library that can perform just the task that I want it to perform, and perform it well.

An AJAST library

So, I decided to create my own AJAST library, OX.AJAST, complete with the following features:

• A fully encapsulated mechanism for making AJAST calls. You simply supply a URL, the name of the callback parameter that will be appended to the URL, and a callback function.
• Support for timeouts. Remote requests can of course time out, and time outs need to be handled. Apart from the obvious security challenges involved with using AJAST (note that I'm not saying they're defects, they're challenges for us developers to handle) , this is the hardest challenge for AJAST requests. Without the ability of specifying timeouts, we're essentially in the dark with regards to whether or not a request will complete. OX.AJAST neatly supports timeouts by wrapping the supplied callbacks, putting on a timer, and checking for completion when the timer times out.
• Guarantee that the callback function will be called. Whatever happens, and as a direct consequence of the timeout support, the library guarantees that the callback function will be called. For this reason, the callback function must accept two arguments, the first a boolean indicating if the request succeeded or not, the other a string containing the response from the call. If the first argument is false, the call may have timed out or failed.

Here it is in all its glory:

	// This file contains a simple Javascript broker that encapsulates
	// the AJAST technique, allowing for cross-domain REST
	// (REpresentatoinal State Transfer) calls.
	//
	// Copyright (c) 2008 Håvard Stranden <havard.stranden@gmail.com>
	//
	// Permission is hereby granted, free of charge, to any person
	// obtaining a copy of this software and associated documentation
	// files (the "Software"), to deal in the Software without
	// restriction, including without limitation the rights to use,
	// copy, modify, merge, publish, distribute, sublicense, and/or sell
	// copies of the Software, and to permit persons to whom the
	// Software is furnished to do so, subject to the following
	// conditions:
	//
	// The above copyright notice and this permission notice shall be
	// included in all copies or substantial portions of the Software.
	//
	// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
	// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
	// OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
	// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
	// HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
	// WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
	// OTHER DEALINGS IN THE SOFTWARE.
	if(typeof(OX) === 'undefined') OX = {};
	OX.AJAST =
	{
	Broker : function(url, callbackparameter, optional_decode_json_response, optional_timeout_milliseconds, optional_default_params)
	{
	this.url = url;
	this.cb = callbackparameter;
	this.params = [];
	this.timeout = optional_timeout_milliseconds || 5000; // Timeout in milliseconds
	if(typeof(optional_default_params) !== 'undefined')
	{
	for(p in optional_default_params)
	this.params.push(p + '=' + encodeURIComponent(optional_default_params[p]));
	}
	this.jsonmode = optional_decode_json_response || false;
	},
	__callbacks__ : {},
	__callid__ : 1,
	call: function(url, callbackparameter, callbackfunction, optional_timeout, optional_decode_json_response)
	{
	var callbackid = 'callback' + OX.AJAST.__callid__;
	// Append callback parameter (this also implicitly avoids caching, since the callback id is different for each call)
	url += '&' + encodeURIComponent(callbackparameter) + '=' + encodeURIComponent('OX.AJAST.__callbacks__.' + callbackid);
	// Create script tag for the call
	var tag = OX.AJAST.createScriptTag(url);
	// Get the head of the document
	var head = document.getElementsByTagName('head').item(0);
	// Create a timeout function
	var timedout = function()
	{
	if(OX.AJAST.__callbacks__[callbackid] !== 'undefined') // If the callback still exists...
	{
	// Replace original wrapped callback with a dummy that just deletes itself
	OX.AJAST.__callbacks__[callbackid] = function(){ delete OX.AJAST.__callbacks__[callbackid]; };
	// Signal that the call timed out
	callbackfunction(false);
	// Remove the script tag (timed out)
	head.removeChild(tag);
	}
	};
	// Create timer for the timeout function
	var timer = setTimeout(timedout, optional_timeout || 5000);
	var decode_response = optional_decode_json_response || false;
	// Create the callback function
	OX.AJAST.__callbacks__[callbackid] = function(data)
	{
	// Clear the timeout
	clearTimeout(timer);
	if(typeof(data) === 'undefined')
	callbackfunction(false); // Callback with nothing
	else
	{
	callbackfunction(true, decode_response ? eval(data) : data);
	}
	// Replace original callback with a dummy function
	delete OX.AJAST.__callbacks__[callbackid];
	// Remove the script tag (finished)
	head.removeChild(tag);
	};
	// Inject the call
	head.appendChild(tag);
	},
	createScriptTag: function(url)
	{
	var s = document.createElement('script');
	s.setAttribute('type', 'text/javascript');
	s.setAttribute('id', 'oxajastcall' + OX.AJAST.__callid__++);
	s.setAttribute('src', url);
	return s;
	}
	};
	OX.AJAST.Broker.prototype.call = function(params, callback)
	{
	// Create arguments
	var args = [];
	for(p in params)
	args.push(p + '=' + encodeURIComponent(params[p]));
	for(p in this.params)
	args.push(this.params[p]);
	OX.AJAST.call(this.url + '?' + args.join('&'), this.cb, callback, this.timeout, this.jsonmode);
	};

view raw ox.ajast.js hosted with ❤ by GitHub

Using the AJAST call function

There are two ways of using OX.AJAST. The simplest is to use the call function.

	// Create a function that will be called when the AJAST request completes<br />
	function callCompleted(success, data)
	{
	if(!success)
	alert('Fail');
	else
	alert('Received: ' + data);
	}
	// Call a service
	OX.AJAST.call(
	'http://xampl.com/rest?arg=foo',
	'callback',
	callCompleted);

view raw ajast-sample.js hosted with ❤ by GitHub

The call function will execute the request by appending &callback=wrapper to the URL and injecting a <script> tag with the final URL as the src attribute. This will add a call to the DOM as soon as the data is received, which the browser will execute.

The function called from the injected script tag is a wrapper around the callCompleted function provided to the call function. The wrapper function is created by the call function, and handles timeouts and deletion of the script tag after the callCompleted function completes. As mentioned, by using this wrapper, the AJAST library can guarantee that callCompleted will be called, which significantly eases the handling of asynchronous calls for users of the library.

The function also allows you to specify how long the request will wait for a response before it times out. The default timeout is 5 seconds. Finally, you can pass an argument specifying if you want the response to be automatically decoded from JSON before it is passed to your callback function.

	// Call with a 10 second timeout, decode JSON response
	OX.AJAST.call(
	'http://xampl.com/rest?arg=foo',
	'callback',
	callCompleted,
	10000,
	true);

view raw ajast-sample2.js hosted with ❤ by GitHub

As stated above, all callback functions must be on the form callbackfunction(success, data){}, where success indicates whether or not the asynchronous call succeeded, and data is any data that was received from the call. It is also important to note that data may be undefined, but success will always be true or false.

Using the AJAST broker

The AJAST broker encapsulates a common pattern for REST requests using HTTP GET. Many RESTful services found online typically use some kind of root URL of the form http://xampl.com/rest as the base URL for all their REST services. The query string determines which service is requested, as well as the arguments for the service.

For the services that follow this pattern, the AJAST library provides a Broker class that encapsulates the process of calling the REST services.

The example below shows how the request from the first example can be made using the broker.

	// Create a broker object
	var broker = new OX.AJAST.Broker(
	'http://xampl.com/rest',
	'callback'
	);
	// Perform the same call using the broker
	broker.call({arg: 'foo'}, callCompleted);

view raw ajast-broker-sample.js hosted with ❤ by GitHub

The broker also supports the specification of a timeout limit, automated JSON decoding, and also provides the option of passing a set of default arguments that will be passed with every request, such as a Flickr API key.

	// Create a broker object
	var broker = new OX.AJAST.Broker(
	'http://xampl.com/rest',
	'callback',
	true, // Decode JSON response
	10000, // Timeout in ms
	{APIKey : '123'} // Default parameters
	);

view raw ajast-broker-default-parameters.js hosted with ❤ by GitHub

Now let's do something useful with it.

A real example: Flickr using AJAST

To keep the example as simple as possible, we'll create the functions necessary for a page which fetches the most recent photos from Flickr.

Luckily, Flickr supports REST and JSON callbacks in a lovely manner, so we'll use the broker for our calls.

	function flickrGetRecent()
	{
	// Create a broker
	var broker = new OX.AJAST.Broker(
	'http://api.flickr.com/services/rest/',
	'jsoncallback',
	true,
	10000,
	{api_key: 'YourVeryOwnFlickrApiKey',
	format: 'json'});
	// Perform the call
	broker.call(
	{method: 'flickr.photos.getRecent'},
	recentFetched);
	}

view raw ajast-flickr-sample.js hosted with ❤ by GitHub

We've told the broker to call a function named recentFetched`when the recent photos have been fetched, so let's implement that as well. To keep the example simple, we'll just append the photos to the body of the document.

	function recentFetched(success, rsp)
	{
	// Check for failure
	if(!success || !rsp || rsp.stat != 'ok')
	{
	alert('Call failed');
	return;
	}
	// For each photo...
	for(var i in rsp.photos.photo)
	{
	photo = rsp.photos.photo[i];
	// Create an img element
	var img = document.createElement('img');
	// Set its source to a valid Flickr URL
	img.setAttribute('src',
	'http://farm' + (photo.farm || 1) +
	'.static.flickr.com/' + photo.server +
	'/' + photo.id +
	'_' + photo.secret +
	'_t.jpg');
	// Append the element
	document.body.appendChild(img);
	}
	}

view raw recentFetched.js hosted with ❤ by GitHub

Now we just have to call flickrGetRecent from somewhere in a document, and the most recent photos will be appended to the document.

As you can see, the OX.AJAST library is really easy to use, and enables you to do pure client-side REST service calls across domain boundaries with hardly any effort. I hope you find it useful. Drop a comment if you have problems or suggestions, or if you create improvements to it. Now start using AJAST!

Object-Relational Mappings Considered Harmful

Creating an Object-Relational Mapping (ORM) has become the de facto way of handling persistence in the object-oriented programming paradigm. Almost all systems require some form of persistent state, and relational databases have become the de facto place to put that state. Relational databases are proven, scale well, and organize data in a tabular manner suitable for many of the real world problems that we try to solve, so they are an obvious choice. Choosing them, however, means we have a new problem at our hands, known as the object-relational impedance mismatch.

The problem is that a relational database is not suited for storing an object-oriented model. An object is almost always a non-scalar value, meaning that it won't fit well in a table row. Hence, we have to create a schema suitable for persisting our objects to a set of tables. This schema will be different for each type of object we have, so it's a rather tedious task, but it solves the problem.

As always, we developers tend to dislike repetitive tasks, so we try to simplify and automate the extra work involved with creating an ORM. This has led to various design patterns such as DAO, and in recent years a set of fully automated ORM tools such as Hibernate (Java), NHibernate (.NET), SQLAlchemy (Python), and Propel (PHP) have become very popular. These tools offer a highly transparent solution to the ORM problem, at the addition of a cost that varies heavily with the nature of the ORM problem. Still, their presence moves us closer and closer to a solution - we are creating an abstraction that fully encapsulates the difference between a relational database as our storage for persistent objects, and the objects themselves. Hopefully, we will soon be able to create an ORM that induces a linear or perhaps even constant cost on our solution, regardless of the nature of the problem we are trying to solve.

But wait. The real problem we are trying to solve is how to persist the state of our objects, remember? The database is indeed a store, and hence a candidate solution, but using it introduces an impedance mismatch, which is another problem we need to solve. It really is like trying to fit a square block in a round hole.

Let's stop trying to solve the wrong problem of creating an ORM, and start finding a solution to our initial and real problem of persisting our objects. Recent innovations such as LINQ have taken us a step in the right direction by making persistence an integral part of the language, but we're still a long way from automated persistence for our objects. I am certain, though, that moving focus away from the challenges of an ORM and on to the challenge of persistence in general will take us further.

Let's do that.

(And I'm sorry for reusing Dijkstra's already overused] phrase, but I couldn't resist.)