Tag Archives: Node

Introducing Marionette-Require-Boilerplate for Your Single Page Application

Marionette-Require-Boilerplate (MRB) is a simple boilerplate I created to help get Single Page Applications off the ground with a small starter project. It incorporates a number of great tools and best practices to save time in the early stages of a project.

MRB integrates such cutting edge libraries as Backbone, Marionette, Require, Grunt, Jasmine, Bootstrap and jQuery Mobile, among others. These libraries work together to help you start your project off with a rock-solid foundation, with a powerful module system, composite architecture, simple build process, and unit tests. Marionette-Require-Boilerplate is part of the BoilerplateMVC suite, which includes standouts like Backbone-Require-Boilerplate, the inspiration and basis for MRB. Stay tuned for more from the BoilerplateMVC team!

The GitHub repo has more extensive documentation, but in this post I’d like to introduce some of the underlying technologies and explain what makes MRB a great place to start for your SPA.

Marionette

Marionette is a framework that sits on top of Backbone and provides a composite architecture and a set of tools to help you solve some of the most common problems in Backbone applications, including nested views and layouts, memory management and cleanup of zombie views, and event-driven architecture. Backbone is a fantastic library and provides many useful pieces, but its great strength — lightweight flexibility — is also its great weakness. It is not a framework and leaves you to figure out some basic functions, like how best to create and remove views, or how to nest views within each other.

If you are not confident in your ability to competently architect a Backbone JS application, or you’re just looking to save time, then Marionette is the tool for you. Read more about what it has to offer in the Marionette chapter of Addy Osmani’s Backbone Fundamentals e-book.

Require

RequireJS is the most popular implementation of the Asynchronous Module Definition (AMD) spec. It provides a powerful, flexible and consistent way to organize your application into separate modules with clearly defined dependencies. It allows you to load these modules as separate files asynchronously in your code, or use the r.js optimizer to load all of your modules in a single, minified JS file for production builds.

Note: Marionette has a concept of Module that is very different from the RequireJS “Module” concept. Do not confuse the two. In Require, a module is simply a unit of code that is run once and returns a function or object of some kind for use in other modules. In Marionette, a Module is essentially a section of your application that can be started or stopped, serving as a coarse grain method of organizing your code. It would be possible to define a Marionette Module in a Require Module if you like, but your head might explode.

Bootstrap and jQuery Mobile

Twitter Bootstrap is a leading front-end UI framework for building rich web applications. It includes a large set of CSS classes and jQuery widgets that you can add to your markup to transform it into beautiful, sleek and customizable UI elements. It is an ubiquitous framework — popular because it helps developers who lack certain design skills to easily crank out clean, elegant UIs with minimal effort and non-invasive code and CSS. jQuery Mobile is a leading UI framework for mobile applications that builds on jQuery UI and brings a number of mobile-friendly widgets and controls to the table.

Mobile vs Desktop

MRB uses a scheme (borrowed from Backbone-Require-Boilerplate) that detects the user’s browser and loads either Bootstrap or jQuery Mobile accordingly. It also loads either the Desktop Marionette Controller or the Mobile Marionette Controller. In Marionette, a Controller handles a Router’s routes, and is used to instantiate Models and Views and show them in appropriate Regions or Layouts.

In MRB, the Mobile/Desktop Controllers load Views which work with either jQuery Mobile or Bootstrap (for instance, MobileController loads a MobileHeaderView, while DesktopController loads a DesktopHeaderView). This is our strategy for differentiating behavior and UI elements for mobile or desktop. This is just my strategy, and it may or may not work well for you, so take it or leave it. If you don’t want to worry about the mobile/desktop dichotomy up front, you might consider checking out Marionette-Require-Boilerplate-Lite, which eschews this division and skips jQuery Mobile in favor of a purely Bootstrap approach. Whichever you prefer.

Grunt

Grunt is a first-rate build tool built on Node which allows you to run any number of compatible plugins. In MRB, we use Grunt to execute our RequireJS Optimization, minify our JavaScript and CSS, and run JSHint to detect bugs in our code.

Jasmine

MRB uses Jasmine to execute and manage unit tests. Jasmine’s syntax is very readable and makes it an attractive choice for unit tests.

Handlebars

Handlebars is our templating engine of choice. It offers a clean, friendly syntax that discourages excessive logic in your views, and is a popular choice these days over EJS-style templating options like Underscore’s _.template(). In MRB, we use the require-handlebars-plugin to automatically load our templates as AMD modules, pre-compiled (read: big performance boost) and with custom helper methods and i18n support.

Node HTTP Server

MRB runs a simple Node HttpServer on port 8001. This is not necessary, but since Grunt and the RequireJS optimizer both run on Node, it is a reasonable assumption that the user will have it installed. If you don’t want to run MRB on Node, you can just as easily point a simple Apache web server at the public/ directory in the project. A web server is required; using the plain file system won’t work because of RequireJS’s dynamic loading of resources.

Best Tools plus Best Practices

With Marionette-Require-Boilerplate, I have attempted to bring together a collection of great tools and best practices to help web applications get started quickly and with a firm footing. I hope you find it as useful as I have. I’d love to hear your feedback, so follow me on GitHub or hit me up on Twitter!

Node Application Server with CouchDB

Much has been made lately of “NoSQL” non-relational databases. A few weeks back, Lou at Keyhole Software wrote a post introducing CouchDB, a free Apache Foundation document datastore. Lou’s application was hosted entirely from CouchDB. I recently wrote a web application with a more traditional approach, with a server using CouchDB as its backend. Traditional in that sense, but non-traditional in another: the backend language of choice is JavaScript.

That’s right, today’s post will introduce you to nodeJS, the server-side JavaScript engine. node allows you to do just about anything that you could do server-side in languages like Java, C#, Ruby or Python, but in JavaScript! This has a couple of advantages. First of all, with more and more web application development happening in the browser with MVC-like frameworks such as Backbone (more on that later), it is now easy for front-end devs to seamlessly transition to back-end development. No switching gears mentally when changing from JavaScript to Java, plus you get to use JSON/JS objects server- and client-side. JSON FTW! Second, like the browser-based JavaScript we all know and love, node is asynchronous in nature. Server threads can use callbacks instead of having a thread hanging while waiting for a DB I/O to complete. What that means in practical terms is node is fast.

The application

Today’s post will showcase a simple backend for a web application, written with the help of the Node library Express. The front-end of this particular application is written with Backbone, but that’s a story for another day. This application architecture, with an MVC-powered JavaScript framework powering a single-page application in the browser, node on the server, and a NoSQL database serving as datastore represent the cutting edge in modern web application development. It is also very easy to understand and quite a fun stack to work with!

The application I will be taking you through – Freelection – can be found running online here. The source code can be found on gitHub. It is a simple tool my elementary school teacher wife asked me to build to let kids at her school vote in the presidential election. It allows users to create an election, add candidates, vote in this election and then view the results, in aggregate and broken down by Polling Station (this is just an extra dimension which was needed to break down results by classroom).

node basics

To get started with node, you must first install node on your machine. Note: for Windows users, this can be a bit of a pain. Follow the instructions here to get set up.

The libraries/dependencies for a node application can be specified in a package.json file. The bundled command line utility, npm is used to install the dependencies in this package. For Java jockeys, this is similar to a Maven pom, only 3 times as intuitive and 12 times more beautiful. Here’s what mine looks like:

package.json

{
  "name": "freelection",
  "version": "0.0.1",
  "private": true,
	"engines": {
	"node": "0.8.12",
	"npm": "1.1.49"
  },
  "scripts": {
    "start": "node app"
  },
  "dependencies": {
    "express": "3.0.0rc5",
	"cradle":"0.6.4",
	"underscore":"1.4.2",
	"emailjs":"0.3.2",
	"handlebars":"1.0.7"
  }
}

This defines the application, “freelection” and indicates the version of node and npm to be used, as well as the start script to run (node app on the CL will run our app.js file as a node application) and dependencies. Navigating to our root directory, containing package.json and app.js and running the command npm install will install the desired dependencies into a node_modules folder.

Application core

Once npm has installed the required libraries, we can start our app up by simply running node app. This runs our app.js file, listed below, as a node application.

app.js

var express = require('express'),
	routes = require('./routes'),
	election = require('./routes/election'),
	candidates = require('./routes/candidates' ),
	vote = require('./routes/vote' ),
	http = require('http'),
	path = require('path'),
	cradle = require('cradle');

var app = express();

exports.app = app;

app.configure(function() {
	app.set('port', process.env.PORT || 8080);
	app.set('views', __dirname + '/views');
	app.set('view engine', 'jshtml');
	app.use(express.favicon());
	app.use(express.logger('dev'));
	app.use(express.bodyParser());
	app.use(express.methodOverride());
	app.use(app.router);
	app.use(express.static(path.join(__dirname, 'public')));
});

app.configure('development', function() {
	app.use(express.errorHandler());
});

app.get('/', function( req, res ) {
	res.sendfile( "public/index.html" );
});

app.post('/election', election.postElection);
app.get('/election/:electionId', election.getElectionInfo);

app.get('/candidates', candidates.getCandidates);
app.get('/candidate/:candidateId', candidates.getCandidateInfo);
app.put( '/candidate/:candidateId', candidates.updateCandidate );
app.delete( '/candidate/:candidateId', candidates.deleteCandidate );
app.post( '/candidate', candidates.addCandidate );

app.post( '/castVote', vote.castVote );
app.get( '/results/:id', vote.getResults );

http.createServer(app).listen(app.get('port'), function() {
	console.log("Express server listening on port " + app.get('port'));
});

Let’s break this deceptively simple file down. At the top, we see require() statements. These are the equivalent of imports, and load other JavaScript files for use in app.js. ‘./routes/election’ for instance loads the file ./routes/election.js as the variable election. Likewise, var express = require(‘express’) loads the Express module as defined in package.json.

Managing scope with exports

In node, functions and variables are only accessible from within the same file. Essentially, like most reasonable OOP languages, you have private access unless otherwise specified. In node, that “otherwise” comes in the form of the magical exports object. For example:

email.js

var emailJS = require("emailjs");

/* server is not accessible to other files */
var server = emailJS.server.connect({
	user: process.env.SMTP_USER,
	password: process.env.SMTP_PASS,
	host:    "smtp.gmail.com",
	ssl:     true
});

/* this makes the sendEmail function available outside of email.js */
exports.sendEmail = function( to, subject, body ) {
	server.send( {
		text:    body,
		from:    "freelectionapp@gmail.com",
		to:      to,
		subject: subject,
		attachment: [ {data:"<html>" + body + "</html>", alternative:true} ]
	}, function(err, message) { console.log(err || message); });
};

elsewhere.js

var email = require("./utility/email")
email.sendEmail( "congress@us.gov", "Budget", "Please balance the budget! Signed, everyone" );

Simple enough? If this is confusing you can read a bit more here.

Configuring and running express

So, back to our main app.js file.

var app = express();

exports.app = app;

app.configure(function() {
	app.set('port', process.env.PORT || 8080);
	app.set('views', __dirname + '/views');
	app.set('view engine', 'jshtml');
	app.use(express.favicon());
	app.use(express.logger('dev'));
	app.use(express.bodyParser());
	app.use(express.methodOverride());
	app.use(app.router);
	app.use(express.static(path.join(__dirname, 'public')));
});

Here we call express() to initialize our express app. We follow this up by using exports to make app accessible to the outside, and configuring our app. There are a couple more “magic” variables here worth noting – __dirname is the path to the directory in which your node app is running, and process.env.PORT is used to pull in the environment variable %PORT%.

Skipping to the bottom of the file now, we start up the app we just created:

http.createServer(app).listen(app.get('port'), function() {
	console.log("Express server listening on port " + app.get('port'));
});

app.get(‘port’) pulls the port variable we set in our configuration above. Once the HTTP server is created, the callback method we provided is fired, logging to the console what port our express app is running on. To start our app from the command line, we simply run node app from the command line, and are greeted with this message:

Express server listening on port 8080

Now we’re up and running! Let’s take a look at how we set up controller paths with express.

Express controller paths/service endpoints

In express, we define HTTP controller paths based off of the four HTTP verbs: GET, POST, PUT, and DELETE. For example:

app.get('/', function( req, res ) {
	res.sendfile( "public/index.html" );
});

In this case, a GET request for http://localhost:8080/ will get handled by the function specified above. The request gets marshalled to the req JavaScript object, and the response can be manipulated as res. In this case, we simply call the sendfile() method on the res object, specifying the path to our main index.html file. This index.html file contains our single page web application, and is the only controller path in this application which represents a full page reload. The rest of the controller paths/service endpoints specified are accessed by the client via AJAX.

Service endpoints can be used in conjunction with exports to defer service handling to another file/module:

app.js

election = require('./routes/election')

app.get('/election/:electionId', election.getElectionInfo);

election.js

exports.getElectionInfo = function( req, res ) {
	var electionId = req.param('electionId');
	console.log( "getElectionInfo: " + electionId );
	db.get( electionId, function( err, doc ) {
		if ( err) {
			console.log( err );
			res.send(500, "Unable to retrieve election data");
		} else {
			console.log( doc );
			res.json( doc );
		}
	});
};

Here you can see how our service asynchronously responds with either an error or a JSON document. Which brings us to our first CouchDB query.

CouchDB

For this project, a CouchDB instance has been set up for local development, and for my production deployment on Heroku, an environment variable is set which points to my Cloudant installation of CouchDB. (As an aside, Heroku makes deployment of node web apps an absolute cinch. Very simple model of worker “gyros” and it’s built on Amazon EC2. Worth looking into!) To connect to it, I use cradle, a CouchDB driver for node. Cradle is quite simple to use and serves as a simple interface between node and CouchDB’s RESTful architecture.

I define a node file called dao.js which instantiates and exports an instance of my cradle object, db.

dao.js

var cradle = require('cradle');

if ( process.env.CLOUDANT_URL ) {
	var db = new(cradle.Connection)(process.env.CLOUDANT_URL, 80).database('elections');
} else {
	db = new(cradle.Connection)('http://127.0.0.1', 5984).database('elections');
}

db.exists(function (err, exists) {
    if (err) {
      console.log('error', err);
    } else if (exists) {
      console.log('db elections exists');
    } else {
      console.log('database elections does not exist. Creating...');
      db.create();
      console.log('database created');
    }

    db.save('_design/candidate', {
        views: {
          byId: {
            map: function (doc) {
            	if (doc.type === 'candidate') {
            		emit(doc._id, doc);
        		}
        	}
          },
          byElectionId: {
        	  map: function(doc) {
        		  if ( doc.type === 'candidate' ) {
        			  emit( doc.electionId, doc );
        		  }
        	  }
          }
        }
      });

    db.save('_design/vote', {
        views: {
          byElection: {
        	  map: function( doc ) {
        		  if ( doc.type === 'vote' ) {
        			  emit( doc.electionId, doc );
        		  }
        	  }
          }
        }
      });

  });

exports.db = db;

In this file, we check to see whether our ‘elections’ database exists, and if it does not, we create it. We then save a number of CouchDB views, which if you recall from Lou’s post, allow us to query our database based off of different keys and values.

We use a single database to store our data. Every document is keyed off of _id. To differentiate the different kinds of documents which are stored (election, candidate, vote, etc.), by convention we include a type variable on each document which indicates what kind of document it is. For instance, an election document looks like this:

{
   "_id": "1a00a48331732c4436d51d770777f94f",
   "_rev": "1-12146938649f35ee37b0d72b541897a2",
   "type": "election",
   "name": "Cambridge Elementary Presidential Election",
   "email": "bjones@keyholesoftware.com",
   "description": "This is an example election"
}

Whereas a candidate record may look like this:

{
   "_id": "f311b1dbca3624ef21959b2204fa4e40",
   "_rev": "1-4d7bd4605957125729b82ed3cd7d86bd",
   "type": "candidate",
   "electionId": "1a00a48331732c4436d51d770777f94f",
   "name": "Barack Obama",
   "party": "Democrat",
   "description": ""
 }

Keep in mind that there is no such thing as a schema in CouchDB, so our application itself is responsible for keeping data type consistent! Here we see that candidates are linked to election documents by the field electionId. This is one way to accomplish the equivalent of foreign keys in a RDBMS.

Updates/inserts

Updating CouchDB is fairly straightforward with cradle. For instance, this is how a new vote record is created:

db.save({
	type: 'vote',
	electionId: electionId,
	candidateId: candidateId,
	candidate: candidate,
	station: station
}, function( err, doc ) {
	console.log( err, doc );
	if ( err ) {
		res.send(500, err );
	} else {
		res.send( {success: true} );
	}
});

Note that if db.save() had been passed in the _id of an existing document, that document would have been updated, rather than a new record created.

CouchDB views and Map/Reduce

Let’s take a closer look at our database’s design documents and their views. Recall that in CouchDB, views are grouped into different “design documents,” and can be queried against.

Views are defined in terms of a map and (optional) reduce function. The map function is called for every single document in the datastore. When large quantities of data are involved, this is actually efficient across multiple machines because CouchDB subdivides the records into subsets, which are processed in parallel. The results from map are then passed on to the reduce function, which can then whittle down those potentially huge result sets into the subset desired. In our application, there are no reduce functions, only map. Map/Reduce is a topic worthy of its own blog post, so for more information I recommend reading more here. Let’s look at the views in our candidates design document:

dao.js

db.save('_design/candidate', {
        views: {
          byId: {
            map: function (doc) {
            	if (doc.type === 'candidate') {
            		emit(doc._id, doc);
        		}
        	}
          },
          byElectionId: {
        	  map: function(doc) {
        		  if ( doc.type === 'candidate' ) {
        			  emit( doc.electionId, doc );
        		  }
        	  }
          }
        }
      });

This is cradle’s way of defining a design document called candidate. Candidate can be queried one of two ways, directly by _id, or in aggregate, based off of each candidate’s electionId.

The byId view is simple enough to understand. It simply filters documents based off of doc.type. emit() is a special method in CouchDB which passes the results along to reduce or back to the query. The first parameter it takes is the id by which the view can be queried. The second is the document which is returned. Querying the byId view without specifying an _id will return all candidates across all elections.

byElectionId is used to get a list of candidate documents based off of their electionId. Note that unlike in byId, the key for this view is the document’s electionId. Here is how this view gets queried:

candidates.js

exports.getCandidates = function(req, res) {
	var electionId = req.param('id');
	console.log( "get candidates for election: " + electionId);
	db.view( 'candidate/byElectionId' , { key: electionId }, function(err, doc ) {
		if ( err ) {
			console.log(err);
			res.send( 500, err );
		} else {
			var candidates = _.pluck(doc, "value");
			console.log( "Candidates: ", candidates );
			res.json( candidates );
		}
	});
};

Note that the doc response array from the cradle query contains metadata along with each of the actual documents returned. _.pluck is used to grab the value attribute from each of the objects in the doc array. value is the actual document object for each record. The results from this service call look like this:

[
 {
   "_id": "1a00a48331732c4436d51d77077b4463",
   "_rev": "1-669993a8d4339e32c309cfe129e22e86",
   "type": "candidate",
   "electionId": "1a00a48331732c4436d51d770777f94f",
   "name": "Mitt Romney",
   "party": "Republican",
   "description": ""
 },
 {
   "_id": "f311b1dbca3624ef21959b2204fa4e40",
   "_rev": "1-4d7bd4605957125729b82ed3cd7d86bd",
   "type": "candidate",
   "electionId": "1a00a48331732c4436d51d770777f94f",
   "name": "Barack Obama",
   "party": "Democrat",
   "description": ""
 }
]

Conclusion

CouchDB and node are well-suited for rapid development of simple, scalable applications. CouchDB is a non-relational database, but if you are OK with the tradeoffs of going the NoSQL route with your application, you can do much worse than the node/CouchDB pairing. There is a certain beauty to having all three layers – client, server and datastore – written in JavaScript. No need to marshal JSON to rigid server-side Objects or deal with the paradigm-mismatch that is Object-Relational mappings. If you’re looking to build fast, scalable applications in record times (this entire project took less than a week while only working part-time), you would do well to consider the node approach.

If you have any questions, feel free to shoot me an email. I hope you will also consider following me on twitter – @brettjonesdev

References

Although not all of these libraries were covered in this article, the complete tech stack for this application is listed below and is worth taking a look at. These are all cutting-edge tools worth learning about. I.e. all the cool kids are using them!

Node – Server-side JavaScript
Express – node framework for REST service calls
CouchDB – NoSQL distributed database
cradle – CouchDB driver for node
RequireJS – Asynchronously load JS files as separate modules – Implementation of AMD spec
Backbone – Client-side MVC* framework**
jQuery
Bootstrap – Amazing CSS framework and widget toolkit worth looking into
Handlebars – Templating engine
Backbone.validation – Backbone plugin for form/model validation
Underscore – Utility library on which Backbone is built

* Not really MVC
** Not really a framework

This article first appeared on Keyhole Software’s blog. Stay tuned for more of the latest in tech for small shops to the Enterprise!