Azure functions Unit testing Frameworks

One of the things which makes the software engineering hard is creating a bug free software. No customer likes the bug ridden software and it has ripple effect on efforts & costs. The software paradigm has shifted from the waterfall methodology of creating software to a more agile & iterative way to doing things and that means the approach of testing has also changed.

This is the Mike Cohn’s test pyramid given by him in his book “Succeeding with Agile”

While working with one of the customers, the technical solution required to use a lot of Azure functions(server-less) on Node JS. Majority of the customer development team was new to the Node JS stack and as the node js community is wide and vast, their requirement was to pick the node libraries/packages which were simple to learn with good documentation and can be used for upcoming projects as well. To make sure the quality of Azure functions code is good, we had to pick a unit testing framework to work with. The official documentation for unit testing Azure functions is on Jest but the community talks highly of Mocha. The development team thought to do the comparison between the available technologies and this post is about our learning during the process.

There are various test runners available on the market and are heavily used as well. To not to compare all the libraries available on the market, we decided to compare 2 front runners for node js testing (Jest and Mocha) and all the necessary libraries which work in conjunction with the test runner.

Mocha is oldest of the three and is quite stable. Currently the community push looks to be behind Jest and can be seen by the number of stars and forks.

Productivity is directly relative to the developer experience and thus the experience provided by the framework is important. When working on the tests we have found there are few differences between the two.

Sometimes we do want our test to fail often and try to find out the issues in the cycle as soon as we can. When a test fails both of the frameworks provides a different style of output and details.

To take an example, we created a typo in the code to see what would be the expected details we can get for us to debug.

With Jest, the output is little different where it shows the line number along with the a pointer to show the error.

Expected and Received:

If the expected result is different than the received result, the test case fails and both test runners provides an output result with expected vs received.

In Case of Mocha, the expected vs actual results are color coded with +/- signs. Expected result is in green and actual is in red.

In case of Jest, the output is also color coded and the differential results are also highlighted.

Lets take an example of something more than a string comparison, say you are expecting a JSON response from an external API.

Mocha output for object comparison:

In Jest, the object comparison came out like this:

Mocha gives you the flexibility to use any assertion library and there are many libraries present to consume with Mocha. Chai is the most popular one and we also used Chai for our comparison.

Jest comes with an inbuilt assertion and mocking library. One thing which we have seen is that you can write describe(), tests() or it(), Jest supports all conventions. It makes it easy for developers when coming from different test framework to continue with the same conventions.

For mocking, like assertion library any mocking library can be used with Mocha. One of the popular choice is Sinon.

Jest has its only inbuilt mocking framework as well which supports mocking by creating a mock function or manual mock to override a dependency.

Both Sinon and Jest has worked for us and we were able to mock the implementations for external API calls and Azure-SDK calls. Please see the references to see the workable solution using both.

Here is the cheat sheet for Jest and Sinon:

Mocha doesn’t support parallel test running while Jest does but you can use other npm modules in-conjunction with Mocha to run test in parallel.

We have looked around the internet to gather some performance metrics for both Jest and Mocha, we have found few companies have migrated from Mocha to Jest and gained some performance improvements.

Each of the framework has its own pros and cons and it also boils down to your team engineering past experience and current appetite to chose a new framework & library.

There are few cases studies where companies have migrated from one platform to others and have shared their experiences. You might want to take a look and understand their pain points and learning.

Before deciding on one framework, you can play around with them in your environment and In case you are wondering how to start, here is the starter kit for unit testing the Azure functions using Mocha or Jest.

References:

Related posts:

One of the major limitations when it comes to starting a project is cost. One of the biggest challenges that awaits a project manager is determining exactly when to set a budget baseline. If you are…

Financial inclusion refers that individuals and businesses have access to financial products and services that meet their needs, delivered in a sustainable way. The gains from these are not only for…