Category Archives: Server

Modularization in TypeScript

UPDATE 2: The 0.9 release of TypeScript added support for CommonJS-style `module.exports = {}`, allowing you to export a class! This makes exporting/importing very, very much easier, and obviates the need for some of the more convoluted stuff at the end of this post.

UPDATE: Check out the new github project that accompanies this post:typescript-modularizatino-demo

In my last post, I introduced TypeScript, Microsoft’s new offering which layers static typing on top of JavaScript. Today I’d like to go further and discuss modularization and code organization.
TypeScript has the ability to take advantage of a pair of JavaScript modularization standards: CommonJS and Asynchronous Module Definition (AMD). These capabilities allow for projects to be organized in a manner similar to what a “mature,” traditional server-side OO language provides. This is particularly useful for the most likely candidates for TypeScript – large Web Applications.

Although TypeScript’s modularization is powerful, it requires some background knowledge of CommonJS and AMD, and has its pitfalls. I hope in this post to be able to provide some of that background knowledge and provide some helpful pointers in organizing your TypeScript project in a modular fashion. Particular attention is paid to the topic of importing plain JavaScript files in your TypeScript code.

On the State of Modularization in JavaScript

Organizing scripts in JavaScript-based web apps has historically been a thorny issue. With no native, built-in system for modularizing code into packages, applications tended to quickly devolve into Script Tag Hell – dependencies being resolved by remembering to put this script tag before that one, but not before the other! Add to this the difficulty of preventing pollution of the global namespace and you have strong incentives to structure your application in as few files as possible.
Over time, these factors tended to act as a gravitational force, pulling your code closer and closer together, until the entire application collapsed under its own weight into a single black hole of a JavaScript file before all but the most senior of developers cowered in fear.
Okay, so that’s a bit of a dramatization. But the fact is that organizing JavaScript projects by hand was extremely difficult, and often was simply left undone. Fortunately, a pair of specifications was created to address just this problem!

CommonJS and AMD

CommonJS and AMD provide mechanisms for loading JavaScript files as separate modules, giving you the power of organization needed in both server- and client-side JavaScript applications.

The primary differences between CommonJS and AMD are as follows:

  • CommonJS is synchronous in nature; AMD loads files asynchronously
  • CommonJS is primarily used on the server in conjunction with node; AMD is most useful in the browser and its most popular implementation is RequireJS.

If you’re looking for more info, a good overview of the two can be found here (note that you must use the left/right arrow keys to navigate between slides). So, let’s get started with a basic example of what the syntax for each of these looks like.

CommonJS

CommonJS uses three “magic” keywords – require, module, and exports. Here’s what a simple example of a CommonJS Module file might look like:

greeter.js

function sayHello(name) {
   alert("Hello, " + name);
}
exports.sayHello = sayHello;

Here we use the magical exports variable to hang our sayHello function off of. To use this module in another file, we use require.

app.js

var greeter = require("greeter");
greeter.sayHello("Dave");

Simple enough, right? Only methods or variables which are attached to exports are available outside the file. At this time, note that both CommonJS and AMD modules follow the Singleton pattern – when you include a module the first time, the code in that file runs. The second time you include it, you get a reference to the object which was produced the first time.

The last keyword CommonJS uses is the module keyword, which provides some additional capabilities (read about them here), one of which is a module.exports property. This property is initially the same as the exports property and can be used to hang off public methods and variables, but it also allows you to manually define exactly what object gets returned when the current module is required. For instance:

greeter.js

function sayHello(name) {
   alert("Hello, " + name);
}
exports.sayHello = sayHello;
module.exports = "Hello World";

app.js

var greeter = require("greeter");
alert(greeter); //alerts "Hello World"

Here we see that although we set exports.sayHello to a function, we overrode the entire returned module by setting module.exports to the string “Hello World.”

AMD

Let’s now turn to AMD. AMD’s special keywords are define and require (it works differently than the require in CommonJS so keep that in mind). define is used to define new modules, and require is used to load them. Keeping in mind that the “A” stands for “Asynchronous,” we are not surprised that AMD’s syntax involves callback functions.

greeter.js

define([/*dependencies go here*/], function (/*and get mapped to variables here*/) {
   return {
      sayHello: function(name) {
         alert("Hello, " + name);
      }
   }
});

And to load this module, we do so either in another define call, or dynamically with Require.

app.js

//define an "app" module
define(["greeter"], function (greeter) {
   var app = {
      initialize: function() {
         greeter.sayHello("Dave");
      }
   }
   return app;
});

-or-

function initialize() {
//dynamically load "greeter" without defining another module
   require(["greeter"], function (greeter) {
      greeter.sayHello("Dave");
   });
}
initialize();

So, a little bit more verbose here, but the advantage is that we have the power to load modules asynchronously at runtime! Although Require provides build tools which let you compile your code into a single (or multiple) file, you are also free to load modules dynamically, on an as-need basis at runtime. Both of these options are very useful when working on large projects where performance is a concern.
I should mention now that one feature of AMD/RequireJS is the ability to load code which was written with CommonJS in mind. Code like this expects an exports variable, which Require can provide if we write our code like this:

app.js

//define an "app" module
define(["greeter", "exports"], function (greeter, exports) {
   exports.initialize = function() {
      greeter.sayHello("Dave");
   }
});

Here all that we do is specify a dependency of “exports,” and then the resulting exports variable can be used just like you would in CommonJS. The reason I mention this alternative syntax is because this is the way that TypeScript actually compiles when compiling to AMD.

Modularization in TypeScript

Essentially, TypeScript just provides syntactic sugar which sits on top of JavaScript but looks and feels a lot like standard OO fare. This extends to modularization as well. TypeScript allows you to define classes (which really just resolve to prototypes) and modules (namespacing), which can be exported and used in other files. TypeScript then compiles these modules into either CommonJS or AMD format. By default it uses CommonJS, but adding the compiler option –module amd allows you to compile AMD-compliant modules. Please note that in order for browser-based AMD TypeScript code to actually run in a browser, you must manually include RequireJS or another AMD implementation on the page.

I will spend most of the remainder of the article focusing on AMD compilation in the browser, since that is the most common and important use case for most developers, and also happens to be the trickiest.
In Visual Studio 2012, you can set up your project to use AMD by editing your project file’s “BeforeBuild” step to include the –module amd parameter, as described here. Note that if you are using AMD (i.e. in the browser) you must have loaded an AMD loader such as RequireJS. Otherwise, your project may compile, but it will fail to work in the browser.

Classes

Before we get into the details of modularization, let’s talk briefly about classes in TypeScript. Before proceeding, it might be a good time to brush up on your JavaScript prototypes-vs-classes, prototypical inheritance-vs-classical inheritance knowledge. If you’ve had a cursory look at TypeScript, you’ve seen that class in TypeScript works out to be a constructor function/prototype in JavaScript.

pony.ts

class Pony {
   bite() {
      alert("Chomp!");
   }
}

compiles to

pony.js

var Pony = (function () {
   function Pony() { }
   Pony.prototype.bite = function () {
      alert("Chomp!");
   };
   return Pony;
})();

Which is basically how TypeScript classes work – syntactic sugar that ultimately resolves to a plain old constructor function in JavaScript. The benefit of this, of course, is that it is considerably easier for a developer steeped in OO to understand this way of doing things than trying to wrap one’s mind around prototypes and constructor functions right off the bat.

Update:
With the 0.9 release of TypeScript, Microsoft added support for a module.exports – style assignment, just as I hoped they would. You can now export and import classes directly, like so:

    // client.ts 
    class Client { 
        constructor(public name: string, public description: string) { } 
    } 
    export = Client; 

    // app.ts 
    import MyClient = require('./client'); 
    var myClient = new MyClient("Joe Smith", "My #1 client");

This is hugely helpful and supercedes about 80% of the information that follows. I may write another blog post on the subject, but probably not since this is now rather self-explanatory. Please consider all export/import information that follows to be deprecated.

Exporting/Importing

To use our Pony class in another TypeScript file, we simply use the export keyword:

pony.ts

export class Pony {
   bite() {
      alert("Chomp!");
   }
}

Which with CommonJS compilation yields this:

pony.js (CommonJS compilation)

var Pony = (function () {
   function Pony() { }
   Pony.prototype.bite = function () {
      alert("Chomp!");
   };
   return Pony;
})();
exports.Pony = Pony;

And with AMD compilation looks like this:

pony.js (AMD compilation)

define(["require", "exports"], function(require, exports) {
   var Pony = (function () {
      function Pony() { }
      Pony.prototype.bite = function () {
         alert("Chomp!");
      };
   return Pony;
})();
exports.Pony = Pony;
});

To user out Pony class, we first must import that file, like so:

app.ts

import Pony = module("pony");
var pony = new Pony.Pony();
pony.bite();

Implicit Modules, or when TypeScript is unwieldy

Wait, that seems a little bit different than we might have expected… Shouldn’t we be able to just say var pony = new Pony()? As it turns out, no. Let’s take a closer look at the compiled output above. Notice that, as mentioned previously, TypeScript uses exports.* to export functions and variables, rather than just returning the object or function desired. Because of this, our Pony constructor function, rather than itself being the exported object, is actually just a method off of the exported module! I like to call this surprising behavior Implicit Modules (my term, good luck googling it) – when exporting a class in a file, the file essentially becomes its own module which contains the class that you wanted to export.

IMHO, what would have been more convenient would have been for TypeScript’s compilation to yield a module.exports-style assignment, which would have enabled us to import the Pony class directly, rather than just as a variable off of some sort of implicit Pony module or namespace.
While this is largely just a bit of a syntactical inconvenience, it can also make it difficult-to-impossible to port over JavaScript code which relies on module.exports, as someone who registered an issue with Microsoft complained. It remains to be seen whether a future modification to the language specification, or some compiler flag of some sort will make it possible to compile this way. As we will see later, when importing JavaScript libraries, this will cause a little bit of difficulty.
One thing that can be done is to use this deficiency as a feature. Rather than authoring a single class per file, it becomes perhaps more desirable to place a set of related classes in a single file. For example:

animals.ts

class Pony extends Animal {
   bite() {
      alert("I, " + this.name + " have bitten you!");
   }
}
class Animal {
   name:string;
   constructor(name:string) {
      this.name = name;
   }
}
var pony = new Pony("Bob");
pony.bite();

app.ts

import Animals = module("animals");
var pony = new Animals.Pony("Bob");
pony.bite();

Inheritance, or when TypeScript is great

In this example, we see both an arguable weakness (implicit modules) and a great strength of TypeScript – classical inheritance! Being able to elegantly define classes which inherit from each other is invaluable in large-scale application development. I consider this one of the greatest strengths of the language. Of course, ultimately it just resolves to JavaScript, but achieving this capability in pure JavaScript is really ugly and nigh-inscrutable. I prefer scrutable. But back to modularization.
When dealing with TypeScript-only modules, the Import/Export system works more or less like a charm. When you want to start including JavaScript libraries, it gets a bit trickier.

Importing JavaScript

In TypeScript, there are two ways to “include” another source file – reference comments and import declarations. Let’s have a look at each.

Reference comments

Reference comments add a dependency on the source file specified. They are only used for compilation purposes, and can be used to provide IntelliSense for JavaScript files. They do NOT affect the compiled JS output. Here is an example:

/// <reference path="pony.ts"/>
var pony : Pony; //compiler now recognizes Pony class.

This tells the compiler that pony.ts will be available at runtime. It does not actually import the code. This can be used if you are not using AMD or CommonJS and just have files included on the page via script tags.

Import declarations

If we want to load a file via AMD or CommonJS, we need to use an import declaration.

import myModule = module("pony");

This tells the compiler to load pony.ts via AMD or CommonJS. It does affect the output of the compiler.

Importing JavaScript

Let’s face it. Most of the libraries we will want to use are not written in TypeScript – they’re all in vanilla JavaScript. To use them in TypeScript, we’re going to have to do a little bit of porting. Ambient Declarations and Declaration Source Files will be our tools.

Ambient Declarations (declare var)

Ambient Declarations are used to define variables which will be available in JavaScript at runtime, but may not have originated as TypeScript files. This is done with the declare keyword.
As a simple example of how this could be used, let’s say your program is running in the browser, and you want to use the document variable. TypeScript doesn’t know that this variable exists, and if we start throwing document’s around, he’ll throw a nice compiler error. So we have to tell him, like this:

declare var document;

Simple enough, right? Since no typing information is associated with document, TypeScript will infer the any type, and won’t make any assumptions about the contents of document. But what if we want to have some typing information associated with a library we are porting in? Read on.

Declaration source files (*.d.ts)

Declaration source files are files with a special extension of *.d.ts. Inside these files, the declare keyword is implicit on all declarations. The purpose of these files is to provide some typing information for JavaScript libraries. For a simple example, let’s say we have an amazing AMD JavaScript utility library, util.js which we just have to have in our TypeScript project.

define([], function() {
   return {
      sayHello: function(name) {
         alert( "Hello, " + name );
      }
   }
});

If we wanted to write a TypeScript declaration file for it, we would write something like this:

export function sayHello(name:string): void;

Declaration source files stand in for the actual .js files in TypeScript-land. They do not compile to .js files, unlike their plain *.ts peers. One way you can think of it is *.d.ts files act as surrogates for their .js implementations, since plain .js files aren’t allowed in TypeScript-land. They simply describe their JavaScript implementations, and act as their representative. What this means is that now you can import JavaScript! Here’s how we would use our util library:

import util = module("util");
util.sayHello("Dave");

This import statement here uses AMD or CommonJS to load the util.js file, the same as if util.js had been the compiled output of a util.ts file. Our util.d.ts provides the compiler/IDE with the IntelliSense to know that a sayHello method exists. The big takeaway here: If you want to include a JavaScript file, you need to write a *.d.ts file.
Note here that we used an Import Declaration to include our .js file. If we had merely wanted to tell the TypeScript compiler that util would be available at runtime (meaning we already loaded it somewhere else, via script tag or RequireJS), we could have used a Reference Comment in conjunction with an Ambient Declaration. To do that, we would first need to change our declaration source file:

interface Util {
   sayHello(name:string): void;
}

This defines a Util interface which we can then use for compilation purposes.

///<reference path="util.d.ts"/>
declare var util: Util;
util.sayHello("Dave");

It turns out that lots of people in the open source community have been cranking out TypeScript interface definitions for some of the most popular JavaScript libraries, including jQuery, Backbone, Underscore, etc. You can find dozens of these on GitHub in the DefinitelyTyped project. The interface definition for jQuery can be found here.

How to be lazy

What if I don’t want to take the time to re-write my .js library in TypeScript, or carefully craft an exhaustive *.d.ts file? It’s not hard to get around. Back to our util.js example. Let’s say that we had another method, sayGoodbye, which we didn’t want to take the time to define in our util.d.ts file, because of our supreme laziness. Simple. Just define a single export function in util.d.ts (one line isn’t going to hurt you!). Then, when you want to use methods that TypeScript doesn’t know exist, just use an Ambient Declaration, like so:

import util = module("util");
declare var unchained:any;
unchained = util;
unchained.sayGoodbye();

The magic here is the Ambient declaration of the unchained variable, which is of type any. The any type tells TypeScript not to worry about typing – this variable could be anything. Trust us.

Dynamically Importing Existing JavaScript Libraries – The Problem(s)

The trickiest part of all this is getting TypeScript to use AMD or CommonJS to import a JavaScript library or module, rather than just making it compile using a reference comment. There are two tricky components to this.

First, if you are using an interface definition like those found online, you can NOT use that file as your *.d.ts in an import declaration. The interface declaration is only used to provide compiler/Intellisense information to the TypeScript compiler – interfaces are different than classes. In order for you to import your external library like we did in our simple util.js example from earlier, you need a different sort of *.d.ts file – one which uses the export keyword.

Second, recall from earlier how TypeScript has what I like to call Implicit Modules when importing files? You can’t directly import a class – you import the file it is in and then get your class definition off the resulting module object. Well, this causes us some grief when it comes to importing JavaScript modules which don’t follow the exports.* pattern in their AMD implementation, since most of the time when you import a “class” (constructor function) in AMD, you expect the imported object to be the constructor function itself.

Static Solution – Exporting to global namespace

The first, and simplest way to use a JS library in TypeScript is to simply make a Reference Comment to the *.d.ts interface definition, make an Ambient Declaration for the variable, and then do the actually loading in your RequireJS configuration, exporting the library into the global namespace. This method is acceptable if you are talking about common core libraries such as jQuery or Backbone, but since it relies on the global namespace it is NOT a recommended solution generally. Here’s what this looks like, using jQuery as an example.
In our index.html, we start our application up by loading Require and pointing it at appConfig.js, a plain JavaScript file which sets up RequireJS and starts our application.

index.html

<script type="text/javascript" src="libs/require.js" data-main="appConfig.js"/>

appConfig.js

require.config({
paths: {
   'jquery': 'libs/jquery-1.8.3.min'
},
shim: {
   'jquery': {
      exports: '$'
   }
}
});
require( ['app', 'jquery'], function(App) {
   App.start();
});

app.ts

///<reference path="jquery.d.ts"/>
declare var $:JQueryStatic; //JQueryStatic is defined in jquery.d.ts
export class App {
   start() {
      $( "#content" ).html( "<h1>Hello World</h1>" );
   }
}

The key here is that jQuery is loaded in a non-AMD, globally scoped fashion. In the shim-jquery-exports-$ section in the RequireJS configuration, you can see that the exports keyword tells Require that when “jquery” is loaded, the result is “exported” into the global variable $. Then in our TypeScript file, we simply add a Reference Comment for our *d.ts interface definition and then make an Ambient Declaration saying that the variable $ will be available at runtime and is of the type JQueryStatic.

This is a great method for application-wide libraries like jQuery, but as I mentioned before, it is NOT advisable to use this willy-nilly due to its reliance on the global namespace as well as its load-everything-up-front approach, which may not be desirable in some larger applications. Also note that anytime you want to include a new JS library, you must change your application-level configuration, and this cannot be done (not easily at least) in TypeScript.

Dynamic Solution

So how do we use TypeScript to dynamically import JavaScript libraries? To get around the first problem mentioned above, what I like to do is to define two separate *.d.ts files: one containing the interface definition you probably pulled off the web, and another which exports a single variable of the type defined in the interface file. Let’s use jQuery as our example again. The jquery.d.ts definition defines a JQueryStatic interface. Lets rename our interface definition file to jquery-int.d.ts, and create a new jquery.d.ts that looks like this:

jquery.d.ts

///<reference path="jquery-int.d.ts"/>
export var $:JQueryStatic;

This will allow TypeScript to compile if we import jQuery like below.

app.ts

import JQuery = module( "libs/jquery" );
var $:JQueryStatic = JQuery.$;
export class App {
   start() {
      $( "#content" ).html( "<h1>Hello World</h1>" );
   }
}

Now we are able to compile in TypeScript. However, let’s say that we have a fairly standard AMD-compliant loader file for jquery, which might look something like this:

jquery.js

define( ["libs/jquery-1.8.3.min"], function() {
   return $;
});

Using an AMD “loader” file like this is a common way of modularizing non-AMD JavaScript libraries. The problem here though is although our jquery.js loader returns $ when imported, in TypeScript our import statement expects an object that has a property of $. This is the second problem I mentioned earlier. My workaround for this is to change my AMD loader file to make it use exports.* just like TypeScript does.

define(["exports", "libs/jquery-1.8.3.min"], function (exports) {
   exports.$ = $;
});

Now when we import our jquery.d.ts in TypeScript, we will have the results we expected: a nice exported module with a $ property that happens to conform to our JQueryStatic definition. After weeks of scouring the web, this is the best method that I have come up with for dynamically importing JavaScript libraries in TypeScript. Let’s review the steps:

Dynamic Strategy Summary

  1. Snag an interface definition (*.d.ts file) off the web, or create one yourself. Remember that you can always be lazy and fall back on the any type.
  2. Rename this interface definition file example-int.d.ts.
  3. Create a new example.d.ts file which exports a single variable of the type defined in your interface definition
  4. Create a “loader”-style file example.js and use exports.* to export the desired library.
  5. Where desired, simply import the example module and find your desired library as a property off of the imported module.

And that’s it! It is a bit involved and definitely more work that I would have liked. I am hoping that the maintainers of the TypeScript language someday soon add the ability to use module.exports like I mentioned earlier, but until then this sort of workaround seems to be the order of the day.

Denouement

To summarize our findings, let’s turn now to an elegantly crafted outline:

  1. TypeScript provides handy-dandy static typing and IntelliSense on top of core JavaScript
  2. Modularization in core JavaScript is crappy to non-existent
  3. AMD and CommonJS are great and allow you to cleanly organize code
  4. TypeScript can do AMD or CommonJS
    1. –module flag let’s you switch between the two
    2. TypeScript inheritance is great (tangential but true)
    3. CommonJS module.exports is not allowed.
    4. TypeScript-to-TypeScript is clean and awesome (with the exception of limitations from c.)
    5. JavaScript-to-TypeScript
      1. Relies on Reference Comments, Import Declarations, Ambient Declarations, and Declaration Source Files
      2. Is fairly simple when using global namespace
      3. Is more involved when loading dynamically
        1. Problems
          1. interface definitions are not the same as exports.
          2. Cannot directly import because of Implicit Modules
        2. Solution: see “Strategy Summary” above

I spent a lot of time trying to figure out how to dynamically import JavaScript libraries, so I hope you find my strategy useful. Other similar sorts of strategies I looked at can be found here and here. The first one is a bit odd and I did not like that it requires (no pun intended) you to use the ugly AMD/Require syntax in TypeScript, and the second forces you to modify the interface definitions you find online or write your own class-based definitions.

I like my strategy because, although there is some overhead in writing AMD loader-files which use exports.*, you can leverage the host of online interface definitions while maintaining TypeScript’s clean and elegant syntax. Please recall though that this technique is not necessary if the library in question is loaded in the global namespace!

If you’ve found an even better way of importing JavaScript, I’d love to hear about it. I would also be totally stoked if you’d follow me on Twitter – @brettjonesdev

This article first appeared on Keyhole Software’s blog. Stay tuned for more of the latest in tech for small shops to the Enterprise!

UPDATE: I have created a simple example of the two main techniques (static versus dynamic imports) I describe here. Check it out on github: typescript-modularizatino-demo

Intro to TypeScript Language and Tooling

TypeScript, Microsoft’s new open source JavaScript derivative, brings static typing along with a number of conventional OOP features to the Wild West of JavaScript. Much like CoffeeScript, this syntactical cousin of Ruby and Python compiles to plain old JavaScript. However, unlike CoffeeScript, TypeScript is in fact a superset of the JavaScript language. What this means is that you can actually write vanilla JavaScript in TypeScript (which is cool).

Language Basics

Static Typing – Hello World

Here’s a simple example of TypeScript’s static typing:

function greet(name:string, times:number, el:HTMLElement) {
    var message: string = "";
    for (var i = 0; i < times; i++) {
        message += "Hello, " + name;
    }
    el.innerHTML = message;
}
greet("Bob", 3, document.getElementById('content'));

If we try to pass our greet function parameters of the incorrect type, TypeScript won’t compile:

greet("Bob", "not a number", document.getElementById('content'));

Object Orientation

In my honest opinion, the greatest strength of TypeScript is its introduction of classic OO constructs like classes and inheritance. Granted, these sorts of things can be done in JavaScript’s prototypical system (see David Pitt’s blog post on the subject), but it tends to be more verbose, a bit confusing and far from elegant. In fact, it is so unwieldy that some popular libraries such as MooTools and Backbone provide their own OO abstractions to shield the common developer from the gritty realities of prototypical inheritance.

TypeScript provides the kind of OO goodies that developers over the past twenty years have come to expect. This can be very useful in helping ease developers coming from a conventional OO background into the wild and wonderful world of JavaScript and prototypical inheritance. Look at the following example of a simple class:

class Animal {
    name: string;
    constructor(name: string) {
        this.name = name;
    }
    sayHello() {
        alert("Hello, my name is " + this.name);
    }
}

This compiles to the following JavaScript:

var Animal = (function () {
    function Animal(name) {
        this.name = name;
    }
    Animal.prototype.sayHello = function () {
        alert("Hello, my name is " + this.name);
    };
    return Animal;
})();

Here we can see that although TypeScript appears to follow a more traditional OOP paradigm, under the covers all it really does is add some syntactic sugar to JavaScript’s basic prototypical inheritance model.

Inheritance

Let’s look at a simple example of one of the most useful of all OO paradigms (and my personal favorite) – Inheritance.

class Animal {
    name: string;
    constructor(name: string) {
        this.name = name;
    }
    sayHello() {
        alert("Hello, my name is " + this.name);
    }
}

class Pony extends Animal {
    sayHello() {
        super.sayHello();
        alert("and I am a pony!");
    }
}

var pony: Pony = new Pony("George");
pony.sayHello();

Pretty cool! I won’t show you what this compiles to here (it’s ugly), but if you’re curious, copy this into Microsoft’s online TypeScript Playground to get a better feel for how TypeScript compilation works.

Plain JavaScript in TypeScript

Although TypeScript allows you to use static typing, at any point in development you are free to fall back to writing vanilla JavaScript. For instance, our original greet method example from earlier could have been written in plain JS:

function greet(name, times, el) {
    var message = "";
    for (var i = 0; i < times; i++) {
        message += "Hello, " + name;
    }
    el.innerHTML = message;
}

Modularization and multi-file

TypeScript provides built-in support for CommonJS and AMD modules. It is quite simple to import and export TypeScript files, just like you would in a server-side language. Importing JavaScript libraries is a bit trickier, but can still be done. I will cover more on this in a later blog post dedicated to the topic, so stay tuned!

Tooling

TypeScript was created by Microsoft, so as you’d expect, the first major IDE to have support for it is Visual Studio 2012. They have a nice plugin that integrates the TypeScript compiler into the IDE. Note that this download includes the TypeScript compiler, which can be run from the command line, as well as the VS plugin (VS 2012 must already be installed when the plugin is installed in order for the IDE to include TypeScript support).

As far as support goes with other IDEs, the current landscape is a bit sparse. WebStorm (my personal favorite web IDE, created by the people who brought you IntelliJ) has support coming in version 6, currently only available in the Early Access Program. There does not seem to be a solid Eclipse plugin yet. There appear to be some offerings available for Sublime Text, emacs and vim.

At the moment, the clear frontrunner – as you might expect – is Visual Studio. Let’s look at what it has to offer.

Visual Studio 2012

You can create a new TypeScript project by selecting “HTML Application with TypeScript.”

This creates a basic project that includes a default.html that loads app.js, the compiled result of app.ts, the main TypeScript file.

Running in Browser

Building our solution uses the TypeScript compiler to generate an app.js output file. To run, we can select what browser we want to use. Like any level-headed developer, I choose Chrome.

Selecting this option opens up a new tab in Chrome running our app:

Debugging

We have two options to debug in Chrome developer tools: debug the compiled JavaScript output, or use source maps to debug our TypeScript files. The first option is simple enough – we just open our developer tools, open up app.js and start setting breakpoints.

Source maps are a bit different. If you’re not familiar with the concept, you can read about it here. Basically, source maps map the compiled, possibly minified code that the browser sees onto the actual source code you wrote, with all its syntactic niceties. This, of course, makes debugging much easier. In TypeScript, adding the -sourcemap switch on compilation generates sourcemap information in the compiled output. Visual Studio adds this option by default, so we don’t have to worry about it here.

To use source maps in Chrome, you must enable the source maps option:

This gives us the option to load app.ts and view our TypeScript code:

These debugging capabilities make real TypeScript development much more practical.

IntelliSense

One of the biggest benefits that comes with using TypeScript is the ability to use powerful IntelliSense. With plain JavaScript’s dynamic typing, it is quite difficult for IDEs to offer safe, clean and reliable IntelliSense capabilities (WebStorm does it the best, in my opinion). With TypeScript, you can rely on such useful operations as Renaming a method or class. This is something very hit-or-miss in pure JS, and will be a great boon when developing large-scale applications.

Takeaway/First Impressions

What to make of TypeScript? The reaction to TypeScript’s release has ranged from dismissive to enthusiastic. Naturally, opponents of static typing dismiss it as an unholy pollution of JavaScript’s type-agnostic purity, whereas fans of compiled languages have great praise for it. Those who are critical of Object Orientation generally will not be impressed, while adherents will be excited to finally have OO available to them in the browser.

My take: it depends on you and your use case.

If you are working on a simple, lightweight web application with no more than a handful of developers, the overhead of compilation and learning new language features will probably not be worth the cost in flexibility and speed of development. You won’t like it.

If, on the other hand, you are working on a large-scale Enterprise application with multiple teams and dozens of developers, TypeScript could very well be the tool that makes a JavaScript-based application feasible. There is a certain degree of safety and stability to be found in static typing, and OO certainly helps prevent repetitive coding tasks. If your developers are all JavaScript gurus with years of experience on the client, you probably don’t need TypeScript. If you have a gaggle of Java/C# guys, TypeScript could be huge for you.

So should you invest in TypeScript? Again, it depends on your team and your project.

I’ll add a final caveat – TypeScript is very new right now. It could be a big risk for an organization to take a leap with TypeScript this early in the game. It certainly helps that a giant like Microsoft is behind it, but it remains to be seen if TypeScript will be a long-lived platform. I would certainly advise caution at this point.

In my next post, I will be examining TypeScript’s modularization techniques and how existing JavaScript libraries can be leveraged, so stay tuned!

UPDATE
Part two on TypeScript modularization is now available here. Enjoy!

Brett Jones, asktheteam@keyholesoftware.com

I would also be honored if you would follow me on Twitter – @brettjonesdev.

This article first appeared on Keyhole Software’s blog. Stay tuned for more of the latest in tech for small shops to the Enterprise!

Node Application Server with CouchDB

Much has been made lately of “NoSQL” non-relational databases. A few weeks back, Lou at Keyhole Software wrote a post introducing CouchDB, a free Apache Foundation document datastore. Lou’s application was hosted entirely from CouchDB. I recently wrote a web application with a more traditional approach, with a server using CouchDB as its backend. Traditional in that sense, but non-traditional in another: the backend language of choice is JavaScript.

That’s right, today’s post will introduce you to nodeJS, the server-side JavaScript engine. node allows you to do just about anything that you could do server-side in languages like Java, C#, Ruby or Python, but in JavaScript! This has a couple of advantages. First of all, with more and more web application development happening in the browser with MVC-like frameworks such as Backbone (more on that later), it is now easy for front-end devs to seamlessly transition to back-end development. No switching gears mentally when changing from JavaScript to Java, plus you get to use JSON/JS objects server- and client-side. JSON FTW! Second, like the browser-based JavaScript we all know and love, node is asynchronous in nature. Server threads can use callbacks instead of having a thread hanging while waiting for a DB I/O to complete. What that means in practical terms is node is fast.

The application

Today’s post will showcase a simple backend for a web application, written with the help of the Node library Express. The front-end of this particular application is written with Backbone, but that’s a story for another day. This application architecture, with an MVC-powered JavaScript framework powering a single-page application in the browser, node on the server, and a NoSQL database serving as datastore represent the cutting edge in modern web application development. It is also very easy to understand and quite a fun stack to work with!

The application I will be taking you through – Freelection – can be found running online here. The source code can be found on gitHub. It is a simple tool my elementary school teacher wife asked me to build to let kids at her school vote in the presidential election. It allows users to create an election, add candidates, vote in this election and then view the results, in aggregate and broken down by Polling Station (this is just an extra dimension which was needed to break down results by classroom).

node basics

To get started with node, you must first install node on your machine. Note: for Windows users, this can be a bit of a pain. Follow the instructions here to get set up.

The libraries/dependencies for a node application can be specified in a package.json file. The bundled command line utility, npm is used to install the dependencies in this package. For Java jockeys, this is similar to a Maven pom, only 3 times as intuitive and 12 times more beautiful. Here’s what mine looks like:

package.json

{
  "name": "freelection",
  "version": "0.0.1",
  "private": true,
	"engines": {
	"node": "0.8.12",
	"npm": "1.1.49"
  },
  "scripts": {
    "start": "node app"
  },
  "dependencies": {
    "express": "3.0.0rc5",
	"cradle":"0.6.4",
	"underscore":"1.4.2",
	"emailjs":"0.3.2",
	"handlebars":"1.0.7"
  }
}

This defines the application, “freelection” and indicates the version of node and npm to be used, as well as the start script to run (node app on the CL will run our app.js file as a node application) and dependencies. Navigating to our root directory, containing package.json and app.js and running the command npm install will install the desired dependencies into a node_modules folder.

Application core

Once npm has installed the required libraries, we can start our app up by simply running node app. This runs our app.js file, listed below, as a node application.

app.js

var express = require('express'),
	routes = require('./routes'),
	election = require('./routes/election'),
	candidates = require('./routes/candidates' ),
	vote = require('./routes/vote' ),
	http = require('http'),
	path = require('path'),
	cradle = require('cradle');

var app = express();

exports.app = app;

app.configure(function() {
	app.set('port', process.env.PORT || 8080);
	app.set('views', __dirname + '/views');
	app.set('view engine', 'jshtml');
	app.use(express.favicon());
	app.use(express.logger('dev'));
	app.use(express.bodyParser());
	app.use(express.methodOverride());
	app.use(app.router);
	app.use(express.static(path.join(__dirname, 'public')));
});

app.configure('development', function() {
	app.use(express.errorHandler());
});

app.get('/', function( req, res ) {
	res.sendfile( "public/index.html" );
});

app.post('/election', election.postElection);
app.get('/election/:electionId', election.getElectionInfo);

app.get('/candidates', candidates.getCandidates);
app.get('/candidate/:candidateId', candidates.getCandidateInfo);
app.put( '/candidate/:candidateId', candidates.updateCandidate );
app.delete( '/candidate/:candidateId', candidates.deleteCandidate );
app.post( '/candidate', candidates.addCandidate );

app.post( '/castVote', vote.castVote );
app.get( '/results/:id', vote.getResults );

http.createServer(app).listen(app.get('port'), function() {
	console.log("Express server listening on port " + app.get('port'));
});

Let’s break this deceptively simple file down. At the top, we see require() statements. These are the equivalent of imports, and load other JavaScript files for use in app.js. ‘./routes/election’ for instance loads the file ./routes/election.js as the variable election. Likewise, var express = require(‘express’) loads the Express module as defined in package.json.

Managing scope with exports

In node, functions and variables are only accessible from within the same file. Essentially, like most reasonable OOP languages, you have private access unless otherwise specified. In node, that “otherwise” comes in the form of the magical exports object. For example:

email.js

var emailJS = require("emailjs");

/* server is not accessible to other files */
var server = emailJS.server.connect({
	user: process.env.SMTP_USER,
	password: process.env.SMTP_PASS,
	host:    "smtp.gmail.com",
	ssl:     true
});

/* this makes the sendEmail function available outside of email.js */
exports.sendEmail = function( to, subject, body ) {
	server.send( {
		text:    body,
		from:    "freelectionapp@gmail.com",
		to:      to,
		subject: subject,
		attachment: [ {data:"<html>" + body + "</html>", alternative:true} ]
	}, function(err, message) { console.log(err || message); });
};

elsewhere.js

var email = require("./utility/email")
email.sendEmail( "congress@us.gov", "Budget", "Please balance the budget! Signed, everyone" );

Simple enough? If this is confusing you can read a bit more here.

Configuring and running express

So, back to our main app.js file.

var app = express();

exports.app = app;

app.configure(function() {
	app.set('port', process.env.PORT || 8080);
	app.set('views', __dirname + '/views');
	app.set('view engine', 'jshtml');
	app.use(express.favicon());
	app.use(express.logger('dev'));
	app.use(express.bodyParser());
	app.use(express.methodOverride());
	app.use(app.router);
	app.use(express.static(path.join(__dirname, 'public')));
});

Here we call express() to initialize our express app. We follow this up by using exports to make app accessible to the outside, and configuring our app. There are a couple more “magic” variables here worth noting – __dirname is the path to the directory in which your node app is running, and process.env.PORT is used to pull in the environment variable %PORT%.

Skipping to the bottom of the file now, we start up the app we just created:

http.createServer(app).listen(app.get('port'), function() {
	console.log("Express server listening on port " + app.get('port'));
});

app.get(‘port’) pulls the port variable we set in our configuration above. Once the HTTP server is created, the callback method we provided is fired, logging to the console what port our express app is running on. To start our app from the command line, we simply run node app from the command line, and are greeted with this message:

Express server listening on port 8080

Now we’re up and running! Let’s take a look at how we set up controller paths with express.

Express controller paths/service endpoints

In express, we define HTTP controller paths based off of the four HTTP verbs: GET, POST, PUT, and DELETE. For example:

app.get('/', function( req, res ) {
	res.sendfile( "public/index.html" );
});

In this case, a GET request for http://localhost:8080/ will get handled by the function specified above. The request gets marshalled to the req JavaScript object, and the response can be manipulated as res. In this case, we simply call the sendfile() method on the res object, specifying the path to our main index.html file. This index.html file contains our single page web application, and is the only controller path in this application which represents a full page reload. The rest of the controller paths/service endpoints specified are accessed by the client via AJAX.

Service endpoints can be used in conjunction with exports to defer service handling to another file/module:

app.js

election = require('./routes/election')

app.get('/election/:electionId', election.getElectionInfo);

election.js

exports.getElectionInfo = function( req, res ) {
	var electionId = req.param('electionId');
	console.log( "getElectionInfo: " + electionId );
	db.get( electionId, function( err, doc ) {
		if ( err) {
			console.log( err );
			res.send(500, "Unable to retrieve election data");
		} else {
			console.log( doc );
			res.json( doc );
		}
	});
};

Here you can see how our service asynchronously responds with either an error or a JSON document. Which brings us to our first CouchDB query.

CouchDB

For this project, a CouchDB instance has been set up for local development, and for my production deployment on Heroku, an environment variable is set which points to my Cloudant installation of CouchDB. (As an aside, Heroku makes deployment of node web apps an absolute cinch. Very simple model of worker “gyros” and it’s built on Amazon EC2. Worth looking into!) To connect to it, I use cradle, a CouchDB driver for node. Cradle is quite simple to use and serves as a simple interface between node and CouchDB’s RESTful architecture.

I define a node file called dao.js which instantiates and exports an instance of my cradle object, db.

dao.js

var cradle = require('cradle');

if ( process.env.CLOUDANT_URL ) {
	var db = new(cradle.Connection)(process.env.CLOUDANT_URL, 80).database('elections');
} else {
	db = new(cradle.Connection)('http://127.0.0.1', 5984).database('elections');
}

db.exists(function (err, exists) {
    if (err) {
      console.log('error', err);
    } else if (exists) {
      console.log('db elections exists');
    } else {
      console.log('database elections does not exist. Creating...');
      db.create();
      console.log('database created');
    }

    db.save('_design/candidate', {
        views: {
          byId: {
            map: function (doc) {
            	if (doc.type === 'candidate') {
            		emit(doc._id, doc);
        		}
        	}
          },
          byElectionId: {
        	  map: function(doc) {
        		  if ( doc.type === 'candidate' ) {
        			  emit( doc.electionId, doc );
        		  }
        	  }
          }
        }
      });

    db.save('_design/vote', {
        views: {
          byElection: {
        	  map: function( doc ) {
        		  if ( doc.type === 'vote' ) {
        			  emit( doc.electionId, doc );
        		  }
        	  }
          }
        }
      });

  });

exports.db = db;

In this file, we check to see whether our ‘elections’ database exists, and if it does not, we create it. We then save a number of CouchDB views, which if you recall from Lou’s post, allow us to query our database based off of different keys and values.

We use a single database to store our data. Every document is keyed off of _id. To differentiate the different kinds of documents which are stored (election, candidate, vote, etc.), by convention we include a type variable on each document which indicates what kind of document it is. For instance, an election document looks like this:

{
   "_id": "1a00a48331732c4436d51d770777f94f",
   "_rev": "1-12146938649f35ee37b0d72b541897a2",
   "type": "election",
   "name": "Cambridge Elementary Presidential Election",
   "email": "bjones@keyholesoftware.com",
   "description": "This is an example election"
}

Whereas a candidate record may look like this:

{
   "_id": "f311b1dbca3624ef21959b2204fa4e40",
   "_rev": "1-4d7bd4605957125729b82ed3cd7d86bd",
   "type": "candidate",
   "electionId": "1a00a48331732c4436d51d770777f94f",
   "name": "Barack Obama",
   "party": "Democrat",
   "description": ""
 }

Keep in mind that there is no such thing as a schema in CouchDB, so our application itself is responsible for keeping data type consistent! Here we see that candidates are linked to election documents by the field electionId. This is one way to accomplish the equivalent of foreign keys in a RDBMS.

Updates/inserts

Updating CouchDB is fairly straightforward with cradle. For instance, this is how a new vote record is created:

db.save({
	type: 'vote',
	electionId: electionId,
	candidateId: candidateId,
	candidate: candidate,
	station: station
}, function( err, doc ) {
	console.log( err, doc );
	if ( err ) {
		res.send(500, err );
	} else {
		res.send( {success: true} );
	}
});

Note that if db.save() had been passed in the _id of an existing document, that document would have been updated, rather than a new record created.

CouchDB views and Map/Reduce

Let’s take a closer look at our database’s design documents and their views. Recall that in CouchDB, views are grouped into different “design documents,” and can be queried against.

Views are defined in terms of a map and (optional) reduce function. The map function is called for every single document in the datastore. When large quantities of data are involved, this is actually efficient across multiple machines because CouchDB subdivides the records into subsets, which are processed in parallel. The results from map are then passed on to the reduce function, which can then whittle down those potentially huge result sets into the subset desired. In our application, there are no reduce functions, only map. Map/Reduce is a topic worthy of its own blog post, so for more information I recommend reading more here. Let’s look at the views in our candidates design document:

dao.js

db.save('_design/candidate', {
        views: {
          byId: {
            map: function (doc) {
            	if (doc.type === 'candidate') {
            		emit(doc._id, doc);
        		}
        	}
          },
          byElectionId: {
        	  map: function(doc) {
        		  if ( doc.type === 'candidate' ) {
        			  emit( doc.electionId, doc );
        		  }
        	  }
          }
        }
      });

This is cradle’s way of defining a design document called candidate. Candidate can be queried one of two ways, directly by _id, or in aggregate, based off of each candidate’s electionId.

The byId view is simple enough to understand. It simply filters documents based off of doc.type. emit() is a special method in CouchDB which passes the results along to reduce or back to the query. The first parameter it takes is the id by which the view can be queried. The second is the document which is returned. Querying the byId view without specifying an _id will return all candidates across all elections.

byElectionId is used to get a list of candidate documents based off of their electionId. Note that unlike in byId, the key for this view is the document’s electionId. Here is how this view gets queried:

candidates.js

exports.getCandidates = function(req, res) {
	var electionId = req.param('id');
	console.log( "get candidates for election: " + electionId);
	db.view( 'candidate/byElectionId' , { key: electionId }, function(err, doc ) {
		if ( err ) {
			console.log(err);
			res.send( 500, err );
		} else {
			var candidates = _.pluck(doc, "value");
			console.log( "Candidates: ", candidates );
			res.json( candidates );
		}
	});
};

Note that the doc response array from the cradle query contains metadata along with each of the actual documents returned. _.pluck is used to grab the value attribute from each of the objects in the doc array. value is the actual document object for each record. The results from this service call look like this:

[
 {
   "_id": "1a00a48331732c4436d51d77077b4463",
   "_rev": "1-669993a8d4339e32c309cfe129e22e86",
   "type": "candidate",
   "electionId": "1a00a48331732c4436d51d770777f94f",
   "name": "Mitt Romney",
   "party": "Republican",
   "description": ""
 },
 {
   "_id": "f311b1dbca3624ef21959b2204fa4e40",
   "_rev": "1-4d7bd4605957125729b82ed3cd7d86bd",
   "type": "candidate",
   "electionId": "1a00a48331732c4436d51d770777f94f",
   "name": "Barack Obama",
   "party": "Democrat",
   "description": ""
 }
]

Conclusion

CouchDB and node are well-suited for rapid development of simple, scalable applications. CouchDB is a non-relational database, but if you are OK with the tradeoffs of going the NoSQL route with your application, you can do much worse than the node/CouchDB pairing. There is a certain beauty to having all three layers – client, server and datastore – written in JavaScript. No need to marshal JSON to rigid server-side Objects or deal with the paradigm-mismatch that is Object-Relational mappings. If you’re looking to build fast, scalable applications in record times (this entire project took less than a week while only working part-time), you would do well to consider the node approach.

If you have any questions, feel free to shoot me an email. I hope you will also consider following me on twitter – @brettjonesdev

References

Although not all of these libraries were covered in this article, the complete tech stack for this application is listed below and is worth taking a look at. These are all cutting-edge tools worth learning about. I.e. all the cool kids are using them!

Node – Server-side JavaScript
Express – node framework for REST service calls
CouchDB – NoSQL distributed database
cradle – CouchDB driver for node
RequireJS – Asynchronously load JS files as separate modules – Implementation of AMD spec
Backbone – Client-side MVC* framework**
jQuery
Bootstrap – Amazing CSS framework and widget toolkit worth looking into
Handlebars – Templating engine
Backbone.validation – Backbone plugin for form/model validation
Underscore – Utility library on which Backbone is built

* Not really MVC
** Not really a framework

This article first appeared on Keyhole Software’s blog. Stay tuned for more of the latest in tech for small shops to the Enterprise!

Annotation for Spring Data Source

If you’ve ever written a Spring application that uses multiple datasources, you know that things can get a bit messy trying to figure out what datasource is used where. In this post, I will share simple Annotation-based solution I’ve used in production applications to manage data sources in DAOs declaratively.

To start with, let’s assume we have a Spring application which has three data sources, oracle, mysql, and warehouse.

To start, let’s create an implementation of Spring’s AbstractRoutingDataSource.  We’ll call it simply, Database.

Here’s what it looks like:

public class Database extends AbstractRoutingDataSource
{
   private final static String DATA_SOURCE_KEY = "currentDataSource";

   public enum Source
   {
      oracle, mySql, warehouse;  //names must match names in springapp-servlet.xml
   }

   public static void setDataSource( Source dataSource )
   {
      ThreadStorage.getInstance().put( DATA_SOURCE_KEY, dataSource );
   }

   public static Source getDataSource()
   {
      return (Source) ThreadStorage.getInstance().get( DATA_SOURCE_KEY );
   }

   @Override
   protected Object determineCurrentLookupKey()
   {
      return getDataSource().name();
   }
}

Our springapp-servlet.xml would look like this:

   <bean class="com.springapp.Database" id="dataSource">
      <property name="targetDataSources">
         <map key-type="java.lang.String">
         <!--These keys must match the names in Database.Source enum-->
            <entry key="oracle" value-ref="oracle"/>
            <entry key="mySql" value-ref="mySql"/>
            <entry key="warehouse" value-ref="warehouse"/>
         </map>
      </property>
      <property name="defaultTargetDataSource" ref="oracle"/>
   </bean>

As you can see, our implementation of AbstractRoutingDataSource works by storing the name the Source enum in ThreadStorage. That way, once it is set, any other section of code within the thread will have access to that key, which will be used by Spring to select the proper Data Source.

Now let’s look at how this can be used. Let’s say we have a DAO which provides access to User data:

@Repository
public class UserDaoImpl implements UserDao
{
   @Autowired
   private SimpleJdbcTemplate simpleJdbcTemplate;

   public String getUserName( String userId ) 
   {
      String sql = "select user_name_first_last from users where user_id = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId);
      return simpleJdbcTemplate.queryForObject( sql, String.class, source );
   }

   public List<Purchase> getUserPurchaseHistory( String userId )
   {
      String sql = "select * from member_purchase_history where userId = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId );
      return simpleJdbcTemplate.query( sql, new PurchaseRowMapper(), source );
   }
}

Now let’s say that getUserName() pulls data from an oracle database, but getUserPurchaseHistory() uses a warehouse connection. We could specify that like so:

   public String getUserName( String userId )
   {
      Database.setDatabaseSource( Database.Source.oracle );
      String sql = "select user_name_first_last from users where user_id = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId);
      return simpleJdbcTemplate.queryForObject( sql, String.class, source );
   }

   public List<Purchase> getUserPurchaseHistory( String userId )
   {
      Database.setDatabaseSource( Database.Source.warehouse );
      String sql = "select * from member_purchase_history where userId = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId );
      return simpleJdbcTemplate.query( sql, new PurchaseRowMapper(), source );
   }

This works fine – each method sets its Data Source and uses it appropriately. Now let’s suppose we have an Email Service which uses our UserDao:

@Service
public class EmailServiceImpl implements EmailService
{
   @Autowired
   SimpleJdbcTemplate simpleJdbcTemplate;
   
   @Autowired 
   UserDao userDao;

   public void sendEmail( String userId, String emailTypeCode, String emailAddress )
   {
      Database.setDatabaseSource( Database.Source.mySql );
      
      String userName = userDao.getUserName( userId );
            
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue(  "code", emailTypeCode );
      String emailBody = simpleJdbcTemplate.queryForObject( "select email_body from emailTypes where email_code = :code", String.class, source );
      
      sendEmail( userName, emailBody, emailAddress );
   }

Do you see the problem? Since the data source is stored in the Thread, when we call our userDao.getUserName(), it has the unintended consequence of changing our data source for all subsequent connections! We could fix this by saving off the Previous Data Source and setting it back at the end of each of our methods in UserDao, like this:

   public String getUserName( String userId )
   {
      Database2.Source previousDataSource = Database2.getDatabaseSource();
      Database.setDatabaseSource( Database.Source.oracle );
      String sql = "select user_name_first_last from users where user_id = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId);
      return simpleJdbcTemplate.queryForObject( sql, String.class, source );
      Database2.setDatabaseSource( previousDataSource );
   }

This works, but it a little verbose and also adds several lines which a Developer must Remember (not my strong suit) to write each time he writes a DB I/O method. We can do better.

First, let’s add a little bit of functionality to our Database class:

public class Database extends AbstractRoutingDataSource
{
   private final static String DATA_SOURCE_KEY = "currentDataSource";
   private final static String DATA_PREVIOUS_SOURCE_KEY = "previousDataSource";

   public enum Source
   {
      oracle, mySql, warehouse;  //names must match names in springapp-servlet.xml
   }

   public static void setPreviousDataSource( Source dataSource )
   {
      ThreadStorage.getInstance().put( DATA_PREVIOUS_SOURCE_KEY, dataSource );
   }

   public static Source getPreviousDataSource()
   {
      return (Source) ThreadStorage.getInstance().get( DATA_PREVIOUS_SOURCE_KEY );
   }

   public static void setDataSource( Source dataSource )
   {
      ThreadStorage.getInstance().put( DATA_SOURCE_KEY, dataSource );
   }

   public static Source getDataSource()
   {
      return (Source) ThreadStorage.getInstance().get( DATA_SOURCE_KEY );
   }

   @Override
   protected Object determineCurrentLookupKey()
   {
      return getDataSource().name();
   }
}

Now comes the fun part. We create an Annotation called DataSource:

public @interface DataSource 
{
   Database.Source value();
}

Now, let’s create a Spring AOP / AspectJ Aspect to advise methods which have a @DataSource annotation:

@Component
@Aspect
public class DataSourceAOP
{
   @Before("execution(* *.*(..)) && @annotation(dataSource)")
   public void setDatabaseSourceForMethods( DataSource dataSource )
   {
      setDataSource( dataSource );
   }

   @Before("execution(* *.*(..)) && @within(dataSource)")
   public void setDatabaseSourceForMethodsWithinClass( JoinPoint joinPoint, DataSource dataSource )
   {
      //If the method has its own method-level DataSource Annotation set, then do NOT set it to the class-level default
      if( !hasMethodLevelAnnotation( joinPoint ) )
      {
         setDataSource( dataSource );
      }
   }

   private boolean hasMethodLevelAnnotation( JoinPoint joinPoint )
   {
      boolean result = false;
      Signature signature = joinPoint.getStaticPart().getSignature();
      if ( signature instanceof MethodSignature )
      {
         Method method = ((MethodSignature)signature).getMethod(); //this will pull the interface method
         Object target = joinPoint.getTarget();
         Method implementingMethod = null;
         try
         {
            //this gets the implementing method, which is where any overriding DataSource annotations should be.
            implementingMethod = target.getClass().getMethod( method.getName(), method.getParameterTypes() );  
         }
         catch( Exception e )
         {
            e.printStackTrace( System.out );
         }

         if ( implementingMethod != null && implementingMethod.isAnnotationPresent( DataSource.class ) )
         {
            result = true;
         }
      }
      return result;
   }

   @After("execution(* *.*(..)) && @within(dataSource)")
   public void resetDatabaseSourceForMethodsWithinClass( JoinPoint joinPoint,DataSource dataSource )
   {
      if ( !hasMethodLevelAnnotation( joinPoint ) )
      {
         Database.setDatabaseSource( Database.getPreviousDatabaseSource() );
      }
   }

   @After("execution(* *.*(..)) && @annotation(dataSource)")
   public void resetDatabaseSourceForMethods( DataSource dataSource )
   {
      Database.setDatabaseSource( Database.getPreviousDatabaseSource() );
   }
   
   private void setDataSource( DataSource dataSource )
   {
      Database.setPreviousDatabaseSource( Database.getDatabaseSource() );
      Database.setDatabaseSource( dataSource.value() );
   }
}

There is the magic. This code will allow you to specify a Data Source for a method declaratively, like so:

@DataSource( Database.Source.mySql )
public void doSomething()
{
   //do stuff
}

Since our Aspect fires before each method call and sets the previous data source, then fires at the end of each call and resets the data source to what it was previously, you can call methods that use one data source inside of methods that use a different one, without any unexpected consequences. Our example from before becomes this:

   @DataSource( Database.Source.mySql )
   public void sendEmail( String userId, String emailTypeCode, String emailAddress )
   {
      Database.setDatabaseSource( Database.Source.mySql );
      
      String userName = userDao.getUserName( userId );
            
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue(  "code", emailTypeCode );
      String emailBody = simpleJdbcTemplate.queryForObject( "select email_body from emailTypes where email_code = :code", String.class, source );
      
      sendEmail( userName, emailBody, emailAddress );
   }

Then our UserDao looks like this:

   @DataSource( Database.Source.oracle )
   public String getUserName( String userId )
   {
      String sql = "select user_name_first_last from users where user_id = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId);
      return simpleJdbcTemplate.queryForObject( sql, String.class, source );
   }

   @DataSource( Database.Source.warehouse )
   public List<Purchase> getUserPurchaseHistory( String userId )
   {
      String sql = "select * from member_purchase_history where userId = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId );
      return simpleJdbcTemplate.query( sql, new PurchaseRowMapper(), source );
   }

Much better! For just one final touch, let’s think about DAOs which almost exclusively deal with the same data source. Do we have to specify @DataSource( Database.Source.mySql ) for each method? No! DataSourceAOP also lets you use our DataSource annotation at the class level, like so:

@Repository
@DataSource( Database.Source.mySql )
public class PurchaseDaoImpl implements PurchaseDao
{
   public void purchaseSomething( String userId, String productId )
   {
   }
}

Thanks to our handy-dandy DataSource annotation and DataSourceAOP, when purchaseSomething() gets called, his data source will already by set to mySql. If, however you specify a Data Source on a method, the method-level DataSource will take precedence. In other words, you can do this:

@Repository
@DataSource( Database.Source.oracle)
public class UserDaoImpl implements UserDao
{
   @Autowired
   private SimpleJdbcTemplate simpleJdbcTemplate;

   public String getUserName( String userId ) /*This method will get mySql as its data source!*/
   {
      String sql = "select user_name_first_last from users where user_id = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId);
      return simpleJdbcTemplate.queryForObject( sql, String.class, source );
   }

   @DataSource( Database.Source.mySql )
   public List<Purchase> getUserPurchaseHistory( String userId )
   {
      String sql = "select * from member_purchase_history where userId = :userId";
      MapSqlParameterSource source = new MapSqlParameterSource();
      source.addValue( "userId", userId );
      return simpleJdbcTemplate.query( sql, new PurchaseRowMapper(), source );
   }
}

Before we go, I want to make a few disclaimers. Since our solution is powered by AOP, it has some of the inherent weaknesses and pitfalls associated with AOP. Most notably, if you call one of our advised methods internally (i.e. from the same class), our advice will not be executed and data source will not be set.

In other words, this works:

public class SomeOtherClass
{
   @Annotation
   UserDao userDao;
   
   public void printUserName( String userId )
   {
      System.out.println( userDao.getUserName( userId );
   }
}

but this doesn’t:

@Repository
@DataSource( Database.Source.oracle)
public class UserDaoImpl implements UserDao
{
   /* stuff from above */

   public void printUserName( String userId )
   {
      System.out.println( getUserName( userId ); /*getUserName()'s DataSource doesn't get set*/
   }

I recently found an *awesome* workaround that can let you get around the AOP-doesn’t-work-on-internal-calls issue, which I will soon be implementing in my code.

If you have any questions or suggestions, please leave a comment!