unit-tests karma-runner/jasmine profiling

Question

For unit-tests we use grunt/karma-runner/jasmine/phantom.js. Because we try to cover any new functionality, a number of unit-tests grows up rapidly. And, unfortunately, time

Answer 1

Finding slow tests

• reportSlowerThan (dots or progress reporter will report all the tests that are slower than given number in ms),

• you can jump to the browser and profile your tests in the same way you would do it with an app (eg. flamecharts in Chrome are awesome).

Making the tests faster

• parallelisation:

  We are working on this. You will be able to parallelize the tests across multiple browsers. Of course, these browsers don't have to run on the same machine so you will be able to use multiple machines too.

  I don't think running multiple instances of Karma itself would help that much, as it is written in a very "non-blocking" way and I haven't seen it to be the bottleneck. We should however optimize the CPU/memory usage. A lot of improvements in this area is coming in v0.12.

• running only a subset of tests:

  You should totally use iit/ddescribe (Jasmine). Any reason why this does not work for you? Btw. Mocha's implementation of it.only() and describe.only() has issues, so I recommend Jasmine (or fixing Mocha).

  There is also a prototype of Google Closure plugin which only loads the files you really need (say you iit one test, it will only load/parse/evaluate the files you really need for that iit). This improves start time on huge projects significantly - we should do similar tricks for other module loaders (RequireJS, CommonJS).

• write faster code:

  You probably know this already... ;-)

  DOM is slow. A good separation of logic/DOM will allow you to test more stuff without actually touching DOM. If you use Angular, you might check out the testing directives example.

  Using Dependency Injection pattern helps a lot as you basically minimize the environment you need to prepare (and also tear down) for each test. Here's an example.

  In general, lower lower tests are faster. I think it's good to cover each functionality/issue with lowest possible test.

• eliminate memory leaks:

  If your tests leak memory, the browser will get slower and slower and so the test number 1000 will be pretty slow even though it's well written. Make sure each test frees up all the references. You can use http://localhost:9876/debug.html and profile the memory. Check the memory before executing (after Jasmine executed all the describe() blocks and collected the tests) and then after executing the tests - it should be the same.

  Finding some memory leaks is harder than others, but it can significantly improve the speed - I've seen stuff like from ~5mins to under a minute just by eliminating memory leaks.

Answer 2

1a) About profiling: you can use karma reporters, add to your karma-config this strings:

    reporters: [
        'progress',
        'junit',
    ],
    junitReporter: {
        outputFile: basePath + 'test_out/unit.xml',
        suite: 'unit'
    },

it will produce nice xml like this:

<testcase name="BuilderController should have a tileSelectorId" time="0.015" classname="unit Chrome 31.0.1650 (Windows 7).tileBuilder testing"/>
<testcase name="BuilderController by default should sort by title" time="0" classname="unit Chrome 31.0.1650 (Windows 7).tileBuilder testing"/>

And you can sort and choose which slowest test should be optimized

1b) Also you can use reportSlowerThan from karma config

     reportSlowerThan: 100,

Karma will inform you about tests slower than 100ms

2) I don't know how to start more karma-runner processes/threads.

3) because grunt:watch provide you changed filename.js, you can start corresponding filename.spec.js, so you have to write special grunt-task

and use ddescribe and iit to start only specified describe/it.

Answer 3

If you are looking for a way to know which specs are slow, I wrote a blog post (http://jipiboily.com/how-to-know-jasmine-specs-are-slow/) about it, but short version is use this reporter I made:

// This works under Jasmine 2.3
var slowSpecsReporter = {
  specStarted: function(result) {
    this.specStartTime = Date.now()
  },
  specDone: function(result) {
    var seconds = (Date.now() - this.specStartTime) / 1000
    if (seconds > 0.5) {
      console.log('WARNING - This spec took ' + seconds + ' seconds: "' + result.fullName + '"')
    }
  },
}
jasmine.getEnv().addReporter(slowSpecsReporter);