When using ‘use strict;’ in your scripts you’ll find that you are no longer allowed to overwrite native methods like FileReader() so how do you test that these methods are being called with the appropriate parameters? Lets start with a typical function call involving FileReader() and then modify it to make it easier to test.

function importFile(file) {
  var reader = new FileReader();
  reader.onload = function(e) {
    processData(e.target.result);
  };
  reader.readAsText(file);
}

Pre ‘use strict’; days you could simply stub out the global FileReader() but since that’s no longer an option we need to get a little creative with our code structure. First thing we’re going to do is create a FileReader instance generator function.

function importFile(file) {
  var reader = generateFileReader();
  reader.onload = function(e) {
    processData(e.target.result);
  };
  reader.readAsText(file);
}

function generateFileReader() {
  return new FileReader();
}

Then we’ll move the onload callback to a named function.

function importFile(file) {
  var reader = generateFileReader();
  reader.onload = _readerOnloadHandler;
  reader.readAsText(file);
}

function generateFileReader() {
  return new FileReader();
}

function _readerOnloadHandler(e) {
  processData(e.target.result);
}

Now you can test the importFile function and its parts by stubbing out the generateFileReader function to return a basic reader stub and not have to worry about the native method. In the following example I’m using two simple stubbing methods to generate stub functions and methods.

it('parses files', function() {
    // Set up the stubs.
    var processStub = stubMethod('processData');
    // The second parameter of the stubMethod is what generateFileReader
    // will return when it's called.
    var reader = stubMethod('generateFileReader', {
      onload: null,
      readAsText: stubFunction();
    });
    // Call the public method.
    importFile('/path/to/file');
    // Make assertions
    assert.equal(reader.calledOnce(), true);
    assert.equal(reader.readAsText.calledOnce(), true);
    assert.equal(reader.readAsText.lastArguments()[0], '/path/to/file');
    // Call the callback.
    reader.onload({ target: { result: 'file data' }});
    // Make assertions
    assert.equal(processStub.calledOnce(), true);
    assert.equal(processStub.lastArguments()[0], 'file data');
  });

Splitting up the code in this way makes unit testing possible because you are essentially wrapping the native function call in a function which are you are able to stub out. Happy testing!

It’s common when moving from one version of your application to another that you will want to maintain all of the SEO cred you have built up while simultaneously moving to a new url syntax. To do this people usually reach for mod_rewrite with apache or nginx for which there is quite a bit of documentation on this topic. Unfortunately the same can’t be said for rewriting and 301 redirecting when using HAProxy.

I have a rather common use case. I plan on moving this blog from Tumblr to Ghost using the Ghost Juju charm and the HAProxy charm to handle load balancing, reverse proxy and rewriting and redirecting the old Tumblr style urls to the Ghost url format.

Using mod_rewrite you would likely write something similar to the following to handle rewriting the url to the new syntax and redirecting with a 301 response code:

RewriteEngine On  
RewriteRule ^/post/\d+/(.+)/? http://example.com/$1  [R=301,L]

HAProxy however doesn’t have a single rule for rewrite and redirect instead we have to combine reqrep, to rewrite the url, and redirect, to handle the actual redirection.

Assume the following front and backend configurations:

frontend haproxy-0-80
    bind 0.0.0.0:80
    default_backend haproxy_service

backend haproxy_service
    balance leastconn
    cookie SRVNAME insert
    server ghost-0-2368 10.0.3.220:2368 maxconn 100 cookie S0 check

In order to rewrite the url we first need to add the rewrite into the frontend:

frontend haproxy-0-80
    bind 0.0.0.0:80
    default_backend haproxy_service
    reqrep ^([^\ :]*)\ /post/\d+/(.+)/?     \1\ /\2

This will rewrite the old Tumblr style url format to the new Ghost style url format and pass that url off to the Ghost webserver. If you’re ok with the user still seeing and using the old url style then you can stop here. Both the real Ghost url format and the old Tumblr style url format will work. If however you want to tell the users and any search engines that the old url is no longer valid and to use a new one instead we need to add the redirect rule:

frontend haproxy-0-80
    bind 0.0.0.0:80
    default_backend haproxy_service
    reqrep ^([^\ :]*)\ /post/\d+/(.+)/?     \1\ /\2
    redirect prefix / code 301

The HAProxy redirect syntax requires us to specify what kind of redirect we want to occur. The options are ‘location’, ‘prefix’, and ‘scheme’ none of these truly fit redirecting an old to new url. Fortunately we can trick HAProxy into doing just what we want by telling it we want to redirect to change the prefix of the url and passing / as the url to prefix along with the code we want to send, 301.

We aren’t quite done. If we leave this as is it will redirect every rule, including the Ghost url formatted ones which will put it into a redirect loop. In order to fix this we need to create an access control list to only redirect the old urls:

frontend haproxy-0-80
    bind 0.0.0.0:80
    default_backend haproxy_service
    acl old_url path_beg /post
    reqrep ^([^\ :]*)\ /post/\d+/(.+)/?     \1\ /\2
    redirect prefix / code 301 if old_url

To the frontend we added a new acl rule called “old_url” which returns true if the path begins with /post. We then add the conditional ‘if old_url” to the redirect rule and we’re done. After restarting the HAProxy service you’ll be able to use the old url structure and be 301 redirected to the new Ghost syntax urls which also remain functional.

In trying to resolve this issue I spent many hours reading the HAProxy documentation, reading blog posts and testing. I Even created a serverfault question which I have now updated with the solution so I hope that this post will save others a bunch of time. As always if you have any questions or comments please comment below or mention me on Twitter @fromanegg.

Juju is brilliant. Ok I am a little biased being that I work at Canonical on the Juju project, but every week I’m more and more impressed with how awesome Juju is and how easy it makes developing software and working in the cloud. I tweet and post a bunch about Juju but today I was asked to explain what Juju is to someone like they are 5.

Juju is often described as apt-get for the cloud, but what does someone who isn’t familiar with the Ubuntu ecosystem know about apt-get? I think I’ll need to go even more abstract…

Let’s say that you had built the most awesome Lego race car body (kids still play with Lego right?) but you didn’t know how to make the wheels or make it move with one of those Mindstorm engines. So now you have to go and play around for a long time to learn how to make a wheel and how to hook up engines. But this is going to take a long time and your mom is going to call you for supper soon, there has to be someone who is an expert wheel maker and Mindstorm engine builder right? Wouldn’t it be awesome if they could build wheels and engines you could use in your race car so you can finish it before supper?

Well that is what Juju does. It allows people who have expertise in a specific field to build packages that you can connect to your own projects without needing to be an expert in that field. So how does this help you write software faster in the cloud? Well I think that’s best explained with another, more grown up, example.

Recently, I wrote a Juju Charm for the Ghost blogging platform so that I can move this blog off of Tumblr and onto something a little more customize-able. The problem? I needed a front end server which was capable of load balancing the webservers when the load picks up and I don’t have the time to learn all about the various options and the best way to install and configure them. So I went to what’s known as the Juju Charm Browser and picked the haproxy Charm and added it into my environment. With multiple web servers I could no longer rely on Ghost’s built in SQLite implementation so I needed to hook up to an external MySQL database. Back to the Charm Browser I went and grabbed the MySQL Charm.

So now I have a load-balanced horizontally scale-able Ghost blog (coming soon). You can have one too, it’s incredibly easy too. For you to get your very own horizontally scale-able load-balanced Ghost blog all you have to do is execute these commands:

juju deploy ghost
juju deploy haproxy
juju deploy mysql
juju add-relation ghost haproxy
juju add-relation ghost mysql

Lets pretend for a moment that haproxy isn’t cutting it any longer and you instead want to use apache2:

juju destroy-service haproxy
juju deploy apache2
juju add-relation ghost apache2

Maybe your blog is super popular and you need another 5 webservers:

juju add-unit ghost -n 5

That’s it. You have now taken advantage of many peoples domain expertise to develop a cloud environment for your own blog.

So what if you wanted to use MySQL or any of these other Charms for a different application? That’s the best part these charms are written using the best practices for the particular charm but have easy to interface with hooks. To enable the Ghost charm to communicate with the haproxy charm all I had to write was:

#!/usr/bin/node
var exec = require('child_process').exec;
var port, address;

function storePort(err, returnedPort) {
  port = returnedPort;
  exec('unit-get --format=json private-address', storeAddress);
}

function storeAddress(err, returnedAddress) {
  address = returnedAddress
  exec('relation-set port=' + port + ' hostname=' + address);
}

exec('config-get --format=json port', storePort);

Juju charms can be written in anything that can be executed. The Ghost charm was written in JavaScript, the MySQL one in bash. Others use Python, Puppet, Chef, Ansible, even Docker containers can be orchestrated using a Juju Charm.

Want to run your own wiki:

juju deploy mediawiki
juju deploy mysql
juju deploy haproxy
juju add-relation mediawiki mysql
juju add-relation mediawiki haproxy

How about a MongoDB cluster, Hadoop cluster, Django app, a video transcoding cluster, and more including your own applications. All easily deployable and scalable across public clouds like EC2, HP Cloud, Joyent, Your private OpenStack cloud, and even your very own local machine. That’s right, the above commands all work to deploy identical set ups to all of these targets and more.

This just scratched the surface of the power of Juju but I hope that this glimpse has made you interested enough to go do some exploring of your own. You can find the documentation to get started with Juju here. And as always if you have any questions or comments you can comment below, find me on Twitter @fromanegg on G+ +Jeff Pihach or hop into #juju on irc.freenode.net and ask away.

Last week we released a new version of the Juju GUI which brings with it one major UI change plus a huge refactoring of the application state system.

If you have any questions about Juju or the Juju GUI, you can read the official documentation for Juju and join us on freenode.net in #juju and #juju-gui

Since Facebook released Flux there has been a lot of chatter about the unidirectional data flow architecture and how it helps large scale applications be easier to reason about and develop. With the Juju GUI nearing 70,000 lines of code in the core application we were running into an issue where it was becoming difficult to maintain the correct state throughout the many rendered UI components which are being consistently changed from the user interactions and changes coming in over the websocket from the users Juju environment.

In an effort to remedy this we determined that the only way to solve our current issues and prevent new ones going forward is to develop a unidirectional data flow architecture vs our MVC event style system which was currently in place. The execution flow we decided on was as follows:

  1. When a user visits a url or a delta comes in over the websocket it is parsed and split into disparate sections of state for each component that’s involved.
  2. That state is then saved into the state system.
  3. When the state system changes it diffs from it’s previous state and passes the diff off to the dispatcher.
  4. The dispatcher scans through the diff and passes off the various state components to their registered handlers.
  5. Those handlers then pass the updated data into the UI components.
  6. The UI components are then responsible for updating their DOM representation.
  7. If the user makes a change to the UI, that UI component requests a change from the state system and the cycle repeats. You’ll notice that this is strikingly similar to the Flux architecture:

It’s great to see cases of the multiple discovery hypothesis in action. It’s a sign that you’re on the right track for solving the bigger picture problems. It’s also nice to see someone formalizing this architecture for client side applications in the hopes that others will be able to skip these scalability problems. While our implementation differs from Flux, the architecture is nearly identical. I highly recommend this architecture to anyone writing a large complex application of any kind, client or server side, as it dramatically reduces the complexity of the applications execution.

Do you work on a large application? Do you think this architecture could help simplify your app? Let me know in the comments below, @fromanegg, or +Jeff Pihach Thanks for reading!