technology from back to front

Archive for the ‘Javascript’ Category

Automating pre-deployment sanity checks with Grunt

Grunt is a great tool for building, running and deploying ‘Single Page Apps’. I have a single grunt command to build and deploy to S3 for production, but recently I added some extra functionality to make deployment safer and even easier:

  • Abort if you are not on master branch
  • Abort if there are any uncommitted local changes
  • Abort if not up to date with the origin repo
  • Create a file revision.txt containing the deployed git revision hash, so we can GET it from the server and be sure of which revision is live
  • Automatically create a tag with the date and time.

I found a few existing pieces to implement some of these, but not all of them, and I ended up with a set of custom Grunt tasks, which I present here in the hope that they are useful to others. They could perhaps be packaged up into a Grunt plugin.

With no further ado, here is the stripped down Gruntfile, just showing the parts relevant to this post, though the deploy-prod task definition leaves in the other task names for context in the overall flow.

module.exports = function(grunt) {

  // Load all grunt tasks matching the `grunt-*` pattern
  require('load-grunt-tasks')(grunt);

  grunt.initConfig({
    // Lots of other Grunty things
    // ...

    // Executing the 'gitinfo' command populates grunt.config.gitinfo with useful git information
    // (see https://github.com/damkraw/grunt-gitinfo for details) plus results of our custom git commands.
    gitinfo: {
      commands: {
        'status': ['status', '--porcelain'],
        'origin-SHA': ['rev-parse', '--verify', 'origin']
      }
    },

    gittag: {
      prod: {
        options: {
          tag: 'prod-<%= grunt.template.today("ddmmyy-HHMM") %>'
        }
      }
    },

    shell: {
      gitfetch: {
        command: 'git fetch'
      },
      saverevision: {
        // Save the current git revision to a file that we can GET from the server, so we can
        // be sure exactly which version is live.
        command: 'echo <%= gitinfo.local.branch.current.SHA %> > revision.txt',
        options: {
          execOptions: {
            cwd: 'dist'
          }
        }
      }
    },
  });

  grunt.registerTask('check-branch', 'Check we are on required git branch', function(requiredBranch) {
    grunt.task.requires('gitinfo');

    if (arguments.length === 0) {
      requiredBranch = 'master';
    }

    var currentBranch = grunt.config('gitinfo.local.branch.current.name');

    if (currentBranch !== requiredBranch) {
      grunt.log.error('Current branch is ' + currentBranch + ' - need to be on ' + requiredBranch);
      return false;
    }
  });

  grunt.registerTask('check-no-local-changes', 'Check there are no uncommitted changes', function() {
    grunt.task.requires('gitinfo');

    var status = grunt.config('gitinfo.status');

    if (status != '') {
      grunt.log.error('There are uncommitted local modifications.');
      return false;
    }
  });

  grunt.registerTask('check-up-to-date', 'Check code is up to date with remote repo', function() {
    grunt.task.requires('gitinfo');
    grunt.task.requires('shell:gitfetch');

    var localSha = grunt.config('gitinfo.local.branch.current.SHA');
    var originSha = grunt.config('gitinfo.origin-SHA');

    if (localSha != originSha) {
      grunt.log.error('There are changes in the origin repo that you don\'t have.');
      return false;
    }
  });

  // Some of these tasks are of course ommitted above, to keep the code sample focussed.
  grunt.registerTask('deploy-prod', ['build','prod-deploy-checks','gittag:prod','aws_s3:prod']);

  grunt.registerTask('prod-deploy-checks', ['gitinfo','check-branch:master','check-no-local-changes','shell:gitfetch','check-up-to-date']);
};

We rely on a few node modules:

  • grunt-git which provides canned tasks for performing a few common git activities. We use it for tagging here.
  • grunt-gitinfo which sets up a config hash with handy data from git, and allows adding custom items easily. This helps us to query the current state of things.
  • grunt-gitshell which lets us run arbitrary command line tasks. We use it to git fetch (not supported by grunt-git, though we could probably have abused gitinfo to do it) and to save the current revision to file. I hope that the command I use for that is cross-platform, even to Windows, but it’s only tested on Mac so far.

Hence I ended up with the following added to package.json:

    "grunt-git": "~0.2.14",
    "grunt-gitinfo": "~0.1.6",
    "grunt-shell": "~0.7.0"
by
Sam Carr
on
15/08/14

Super-simple JavaScript inheritance

JavaScript uses prototype-based inheritance, which can prove a bit of a puzzler for those of us used to class-based object orientation. At first glance it seems like it’s basically the same and as if it can be used in very nearly the same way. If you pretend that those prototype objects are in fact classes and ignore the nuances you can get surprisingly far and code up a decent-sized heap of working code. However you will eventually be bitten and have to read up on what’s really happening at the nuts and bolts level. This is guaranteed to happen just when you’re under pressure, trying to get a critical feature working. It’s a horrible feeling, the dawning realisation that the subtle bug you can’t grok is because things don’t work the way you thought at a very basic level and that your heap of code is founded on quicksand. This happened to … a friend of mine.

This post isn’t going to attempt to explain the depths and subtleties of JavaScript’s prototype model. Plenty of others have been there before. In fact we will embrace our class-based stubbornness and attempt to get it working the way we really wanted. Plenty of others have done this too, but there are a few issues with most of their solutions:

  • They are too simplistic and don’t cover all the cases required, like having an arbitrarily deep hierarchy that can call up the constructor chain neatly
  • They are too complicated, having developed into powerful libraries with many features
  • The perennial problem: I didn’t write them, so am not in control and able to understand exactly what’s going on and adapt to exactly my needs – no more, no less.*

I present the result below, wrapped up for require.js. There is really very little code indeed – just two functions: inherit and superConstructor.

// Because class is a keyword in JS, consumers should inject this as clazz.
define(function() {

  return {
    // In the unlikely event that you need to explicitly call a superclass implementation
    // of a method, because a method with the same name exists in the current class:
    //  foo.parent.bar.call(this, x, y);
    inherit: function(child, parent) {
      child.prototype = Object.create(parent.prototype);
      child.prototype.constructor = child;
      child.prototype.parent = parent.prototype;
    },

    // The superclass constructor should generally be called from child's constructor
    // otherwise it won't run and fields defined there will be missing:
    //   superConstructor(this);
    superConstructor: function(self) {
      // The constructor that we call here may in turn wish to call superConstructor() to
      // call its own parent's constructor (but with the same 'self') so we must take
      // special measures to allow this, as self will be the same object with each recursion.
      var constructor = (self.nextParent) ? self.nextParent.constructor : self.parent.constructor;
      self.nextParent = self.parent.parent;
      constructor.call(self);
      self.nextParent = undefined;
    }
  }

});

The contents of inherit are much as you’ll find in many a blog post, though there’s a surprising amount of subtle variation out there!

More interesting is superConstructor, which employes a somewhat offensive tactic to allow calls all the way up the constructor chain. What makes this difficult is that ‘this’ must remain the actual object being constructed throughout those nested calls, so we need to manually provide the context to know what the next constructor up the chain is.

Having done this and saved the code above into clazz.js, we can write code with inheritance as follows (working example as a jsfiddle).

// A Dog can bark.
function Dog() {
    console.log('Constructed a dog');
}
Dog.prototype.bark = function() { return 'Woof' };

// A Yorkie is a Dog that barks a lot!
clazz.inherit(Yorkie, Dog);
function Yorkie() {
    var self = this;
    clazz.superConstructor(this);
}
Yorkie.prototype.bark = function() {
    var noise = this.parent.bark.call(this);
    return noise + noise + noise;
};

// Create dogs and see what noises they make.
console.log(new Dog().bark());
console.log(new Yorkie().bark());

To be fair, my super-simple inheritance library is extremely restricted in its abilities, for instance not handling constructor parameters. But that’s because I didn’t need them, and any extra features should be easy to add. Most of all it was a valuable learning experience.

* Actually I love an off-the-shelf library as much as the next chap (or chappess) – but if you don’t feel comfortable with the libraries on offer and the problem seems nicely tractable and a worthwhile learning experience then why not go for it. You can always change your mind.

by
Sam Carr
on
16/06/14

CSS Transitions can’t animate display change

I’d like to demonstrate a fairly simple CSS issue that caught me out, and the straightforward solution. Put simply CSS Transitions do not work if there is a change in the display property as part of the same change that fires the transition, but you can workaround this by separating out the display change.

If you’re not already aware, CSS Transitions are a cute way of animating transitions on your web page. Simply add a transition property in your CSS stating which property of the element should be animated when it changes, and over what period of time.

.animatedWidth {
    transition: width 2s;
}

In the example above, whenever the width of the element is changed (e.g. programmatically from JavaScript) it will animate that change over 2 seconds, complete with ease-in and ease-out by default.

I’ve created a jsfiddle with a more convoluted example that demonstrates the display problem, so you can inspect the HTML, CSS and JS, and run it in the browser. The example has three coloured bars (though the second two start off invisible) and an Animate button. Click the button and you’ll see that the ordinary transition animates the width of the bar as expected, but where the coloured bar is being made visible at the same time it just winks into existence in its end state with no animation. The third bar appears and then animates correctly, because our JS separately shows it then triggers the animation. It uses a timeout with zero delay to achieve this, effectively giving the rendering engine its chance to handle the display change before then triggering the animation.

button.on('click', function() {
    // To get the animation working we need to change the
    // display property first (via jQuery toggle()) and then
    // trigger the CSS transition with a zero-delay timeout.
    bar3.toggle();
    window.setTimeout(function() {
        bar3.toggleClass('animate');
    }, 0);
});

In my real world situation where I first stumbled across this effect, the item being animated started offscreen (and invisible) and slid into place, with the problem only evident on Chrome for some still unknown reason. The change of display property was but one of many things going on via incidental CSS so it took some sleuthing to figure out that it was responsible for the problem. Coming at it from that baffling angle for the first time, the problem and its solution were not nearly so obvious as presented above!

by
Sam Carr
on
27/05/14

A simple Knockout page router

Knockout.js is a pleasantly simple approach to data-binding ViewModels into your HTML. Like many JavaScript libraries it sticks to a core mission with a few simple concepts, which makes it quite approachable. Its simple template support means that you don’t need to write much code to get a top-level page router going in your single page app (SPA) and that’s exactly what I have done.

Knockout-routing

It uses hash-based routing, so URLs must be of the form http://foo.com/index.html#myPage. This approach means that even a statically hosted site with just the one real URL (index.html in this example) and zero server-side dynamicism can be a SPA with multiple virtual pages. All requests will ultimately come to index.html and then the router takes over and shows the right actual page based on the hash in the URL. Back and forward buttons work, as does page refresh, bookmarking, emailing links etc.

The code is on GitHub, with a decent README explaining the features and the key files to look at, so I won’t repeat that here. The code is also well-commented, with the intention that you can (and should) read it to see how it works. You can clone it, then simply double click src/index.html to open it in your browser and see its capabilities demonstrated. Nice and easy.

The router itself is just a 61 line JavaScript file, which would be very easy to extend with further features that you might need. The rest of the code on GitHub shows how to use it by example, and demonstrates all of its features.

Any feedback is very much appreciated. I imagine there are other similar routers out there, but this one is mine and making it (and using it in anger) taught me a lot and provided a nice, tight result which I can easily add to as required.

by
Sam Carr
on
30/04/14

Getting back into front-end web development

I’ve been working on a small SPA (Single Page Application) – just HTML, CSS and JavaScript statically served and doing its thing entirely in the browser. I learned a great deal throughout the project, but here are some of the things that strike me as most valuable.

Get a good workflow going

I used Grunt to setup a nice build system that is mostly a joy during development. It took a while to evolve my Gruntfile, but now when I edit a file, the results are immediately refreshed in the browser (I don’t even have to hit cmd-R). I can deploy to S3 test, staging and live sites with a single command that takes about 3 seconds. My SASS files are compiled down to minified CSS, my JS is minified with source maps etc.

The hardest part of using Grunt is figuring out how to configure it and its many contrib plugins. I could have done with a definitive reference or perhaps I could have used Yeoman to give me an out of the box solution. However I recognised that I was always going to have to figure out the guts of Grunt so I think I really was better off bespoking it from the start. I’m glad I did as now I have a tight setup that does precisely what I want and that I understand completely.

Now it seems there is a new kid on the scene, Gulp – nicely introduced in this tutorial blog post. I will definitely be looking closely at that for my next project, with the piping approach looking like the key step beyond Grunt, along with nicer syntax. I’d also look at Browserify, for a nicer way to piece together the JS bits.

Learn JavaScript properly

To the uninitiated, JavaScript is fairly surprising in many subtle ways, and though I can grok the prototype-based inheritance fairly easily, the scoping rules caught me out repeatedly. This was especially the case as I tried to create JQuery plugins with private methods and state. Eventually a simple old article by grand-daddy of JavaScript writing Douglas Crockford gave me the vital clues I was missing.

Really I should just read his book, and I would recommend that anyone else doesn’t just attempt to learn JavaScript as they go, but takes some time to pro-actively figure out the core concepts – it will pay off in short order.

jQuery is non-negotiable

And the award for most indispensable library goes to: jQuery. Seriously, it should be baked into the browsers or the ECMAScript standard. The nicest thing about it is I can pretty much just guess at the API and be right most of the time, though reading the docs usually reveals new conveniences that go further than I even imagined.

Browser quirks can be a living nightmare

JavaScript itself is fairly reliable, especially with judicious use of mature libraries like jQuery that paper over the cross-browser DOM cracks. CSS in complicated scenarios is where it all seems to go wrong however.

It’s amazing how broken/different some browsers are. Here are just a few highlights, though every day brought tens of new oddities and associated workarounds.

  • Mobile Safari on iOS 7 reports the viewport height inconsistently (depending on how you access it) leading to bad layout and horrible JavaScript workarounds.
  • Use of -webkit-overflow-scrolling:touch causes the hardware accelerated renderer to kick in, resulting in various flickers, flashes and flitches with content not rendering.
  • IE 10 on Windows 8 shows back/forward overlays at the left/right of the screen when your mouse moves near them, obscuring links in those locations.
  • Chrome running on Retina Macs suffers from strange graphical glitches when running CSS Animations, but is fine with CSS Transitions. However other browsers/platforms really need CSS Animations to get smooth, hardware accelerated movement. In my case it was necessary to implement both approaches and select using browser detection.
by
Sam Carr
on
06/03/14

Grunt uglify file specs

I struggled a bit finding relevant examples of Gruntfile configuration for Uglify, so having solved a few specific problems myself, here’s what I came up with.

This is just a snippet from the whole Gruntfile of course, and contains half-decent comments already, though I’ll provide some extra explanations below to point out the most interesting bits.

// Variables used internally within this config.
conf: {
  app: 'app',
  dist: 'dist',
  // Just our own custom scripts.
  script_files: ['scripts/*.js'],
  // All scripts that should be minified into final result.
  // Ordering is important as it determines order in the minified output and hence load order at runtime.
  // We don't include jquery (though we could) as it's better to get it from Google where possible.
  minify_js_files: [
      'scripts/vendor/modernizr/modernizr.custom.js',
      '<%= conf.script_files %>',
      'scripts/polyfills/**/*.js']
},

uglify: {
  options: {
    banner: '/*! <%= pkg.name %> <%= grunt.template.today("yyyy-mm-dd") %> */\n',
    sourceMap: '<%= conf.dist %>/scripts/source.map.js',
    sourceMapRoot: '/scripts',
    sourceMappingURL: '/scripts/source.map.js',
    sourceMapPrefix: 2
  },
  // For dev, effectively just concatenate all the JS into one file but perform no real minification.
  // This means that the HTML is the same for dev and prod (it just loads the single .js file) but
  // debugging in the browser works properly in the dev environment. It should work even when fully
  // minified, given the source maps, but practice shows that it doesn't.
  dev: {
    options: {
      report: false,
      mangle: false,
      compress: false,
      beautify: true
    },
    files: [{
      expand: true,
      cwd: '<%= conf.app %>',
      src: '<%= conf.minify_js_files %>',
      dest: '<%= conf.dist %>/scripts/main.min.js',
      // Because we want all individual sources to go into a single dest file, we need to use this
      // rename function to ensure all srcs get the same dest, otherwise each would get a separate
      // dest created by concatenting the src path onto dest path.
      rename: function(dest, src) { return dest; }
    }]
  },
  prod: {
    options: {
      banner: '/*! <%= pkg.name %> <%= grunt.template.today("yyyy-mm-dd") %> */\n',
      report: 'min',
      mangle: true,
      compress: true
    },
    files: '<%= uglify.dev.files %>'
  }
},

Use of a rename function for configuring file srcs and dests

I was really struggling to come up with src/dest configuration for Uglify that pushed all of my source files into a single minified dest file. To be fair, this is trivially easy in the common case, as you can simply use files: { ‘dest_path’: ['file_1', 'file2'] }.

However I have my list of source files in <%= conf.minify_js_files %> and the paths therein do not include the root app/ directory, because this works out for the best in various other Grunt tasks (not shown) where I use cwd in the files block to supply that root dir. Unfortunately, without my custom rename function, a separate dest is calculated for each src file, by concatenating the src path with the dest, so instead of one minified JS file we get lots of individual files sprayed over all sorts of unintended locations. The trivial rename function I’ve used overrides those calculated dest locations to our originally intended single dest. Where different srcs have the same dest, the grunt-contrib-uglify plugin has the good sense to assume you want to merge their output. And hence we get the result we want. To be clear, this is only complicated because I want to use cwd in the file config rather than using the simpler approach.

Re-using files blocks in multiple places

You can share common options amongst multiple targets by putting them at the top level of the task and overriding/extending as required in the specific targets. However you can’t do this when specifying files. In my case I want to specify the same files for both dev and prod Uglify targets, so I specify them in full for dev then use Grunt’s templating facility to refer to them from prod with files: ‘<%= uglify.dev.files %>’.

Theoretically I could have put the definition in the conf block at the top, but it’s specific to Uglify and only used there so I prefer it to be local to the Uglify task. It seems obvious now that I can refer back to it like this, but at the time I struggled to see how to achieve it. I think I had a blind spot for the generic nature of the templating mechanism, having only used it in a rigid way for config previously, and still being very new to Gruntfiles.

Uglify may break JS debugging

I found that my minified JS files could not be successfully debugged in any browsers. I could see the original un-minified code thanks to the source maps and I could set breakpoints, but they usually wouldn’t trigger, or if I did break (e.g. with JS ‘debugger’ command in my code) it was impossible to get variable values. All very frustrating.

Whilst I’m developing I use Grunt’s watch task to serve my files and to auto-process modifications on the fly, so in that mode I turn off all the actual minification features of Uglify and effectively just concatenate the files together into one. Because it’s still the same single file with the same name as in production, I can use identical static HTML to include my JS script. The source maps are still there and allow me to see the individual files in the browser debugger.

by
Sam Carr
on
08/01/14

Enhancing peer review through GitHub

You love GitHub. Of course you do. You love peer review. You especially love sending a pull request back asking for nits to be picked. So when your submitter claims to have addressed your concerns, how do you check? You could walk the commits. You could diff the entire pull request against master. If only you could diff the HEAD of the pull request against the original state of the pull request, letting you check just the new set of commits…

With github-differ you can!

Simply add this tiny extension[1] to your Chrome, and it will decorate each commit in GitHub’s Commits tab. Pick any two commits, and the extension will redirect you to a page showing the comparison of those two commits! Job done!

[1]The JavaScript involved is so small that it should be trivial to port this to FireFox’s GreaseMonkey framework.

by
Frank Shearar
on
27/03/13

Live updates to Meteor from Postgres

[Extra form validation angular-ui]: http://angular-ui.github.com/
[AngularJS]: http://angularjs.org/
[Meteor]: http://meteor.com/
[Node.js]: http://nodejs.org/
[Postgres]: http://www.postgresql.org/
[TimeTracker]: http://www.lshift.net/timetracker.html
[Meteorite]: https://github.com/oortcloud/meteorite
[Meteor Postgres]: http://stackoverflow.com/questions/10802191/using-meteor-with-postgresql
[Node.js pg]: http://gurjeet-tech.blogspot.co.uk/2012/11/install-pg-nodejs-module-node-postgres.html
[Postgres Node.js LISTEN NOTIFY]: http://bjorngylling.com/2011-04-13/postgres-listen-notify-with-node-js.html
[pg Standalone Client]: http://lheurt.blogspot.co.uk/2011/11/listen-to-postgresql-inserts-with.html
[Meteor Fibers]: http://stackoverflow.com/questions/10192938/meteor-code-must-always-run-within-a-fiber-when-calling-collection-insert-on-s
[Fiber]: http://en.wikipedia.org/wiki/Fiber_(computer_science)
[Node Fibers]: https://github.com/laverdet/node-fibers
[Node Postgres]: https://github.com/brianc/node-postgres
[Meteor-AngularJS]: https://github.com/lvbreda/Meteor_angularjs
[Meteor Angular Leaderboard demo]: https://github.com/bevanhunt/meteor-angular-leaderboard
[Meteor Principles]: http://docs.meteor.com/#sevenprinciples
[NOTIFY]: http://www.postgresql.org/docs/9.0/static/sql-notify.html
[LISTEN]: http://www.postgresql.org/docs/9.0/static/sql-listen.html
[js2coffee]: http://js2coffee.org/

I’ve been playing around with [Meteor][] recently for an internal LShift project in which I wanted the browser to have a read-only live view onto some timetracking data from [TimeTracker][] as it changes. When a developer records time spent on a particular task, a row is inserted into a [Postgres][] database. Simples.

One of Meteor’s selling points is its transparent client-server data synchronisation through its powerful `Collections` API, which is backed by MongoDB on the server side. This poses a potential challenge for us: how can we leverage the reactive `Collection`s of Meteor yet feed it with Postgres data?

Well, I did manage to wrestle my fork of Bevan Hunt’s (a Meteor contributor!) excellent [Meteor Angular Leaderboard demo][] into live page update as database insertions occur on Postgres. You could extend this to deal with database updates and deletions as well, an exercise left for the willing.

Read more…

by
hok
on
25/02/13

Search

Categories

You are currently browsing the archives for the Javascript category.

Feeds

Archives

2000-14 LShift Ltd, 1st Floor, Hoxton Point, 6 Rufus Street, London, N1 6PE, UK+44 (0)20 7729 7060   Contact us