Service to Service communication with Hemera and NATS

How to increase scalability, performance and maintainability of your software

Patrick Heneise

Patrick Heneise
February 13, 2018
4 min read

Hemera 5 has just been released and we’d like to give you an overview of why message queues are great for your Node.js application and how you can apply them easily in your code to optimise performance and maintainability.

Projects get bigger all the time, features are added along the way and you might end up with a big application (monolith). The application becomes difficult to understand and modify, difficult to implement new features and over time, code quality decreases and development speed slows down. To avoid this issue from the start, you can structure your application by Domain (Domain Driven Design), using separate services (needn’t necessarily be *micro *services) for separate logical clusters or business purposes. For example: Products, Users and the Shopping Cart. Each service has different needs and requirements: you might have a database for millions of products and highly volatile eCommerce transactions. It would not make much sense to use the same database and system resources for such different purposes and you can save a lot of money by optimising usage. Service-based architecture also helps to exchange parts in your technology stack to evolve with your requirements.

Coming from a single application, you would probably import/require products.js and use something like products.list() to retrieve a list of products, maybe even enhanced with pagination and other metadata. products.js is maintained by another developer and you trust it to work as you expect it. There are several ways to do this in a service-based architecture. We decided to use NATS as message queue system. It’s very simple and incredibly fast. There’s an official NATS docker image that you can pull in docker-compose.

There are alternatives like HTTP, RabbitMQ etc. which all have their pros and cons which we won’t discuss in this article.

NATS Remote Functions with Hemera

Hemera is a Node.js toolkit for NATS and simplifies messaging in distributed systems. The pattern we use is called RPC — Remote Procedure Call (named Request & Reply in Hemera). Instead of the applications internal products.list() example above, the call is wrapped in a hemera function and executed by a different service. The behaviour is exactly the same, you can use async/await, Promises or Callbacks, only the function call changes a little bit:

hemera.act({topic: “users”, cmd: “list”})

To register the function in the distributed system, you add the topic and command to the messaging system:

hemera.add({topic: “users”, cmd: “list”}, function() { … })

“act” calls for a function, “add” registers a function on the queue.

On each service, you add available functions: users.list(), users.get(id), product.update({}) etc. From the API (or anywhere else) you call (act) that function with the required parameters.

The full command to register a function on the queue with hemera requires a topic, a command and the function. Further attributes are optional:

Congratulations, you wrote your service-to-service Hello World App. Let’s move on.

Joi

A neat addition to mention at this point is Joi Schema Validation. As the name says, a defined schema is used for input validation. The function above doesn’t accept “Hello” as value for a but instead throws an error. There’s no need to re-validate or check the input parameters, for example if (!req.a) {...}, if (!req.b) {...} , which is great. We’re working on a way to automatically extract the Joi validation of a Hemera function into a JSDocs for better documentation.

Improved Hemera handlers

The first step when writing something other than “Hello World” is to use a named function instead of the inline function above. To keep definition and implementation together, we export both from a single file:

In the main file (ie. index.js), those are added to the queue:

hemera.add(getProduct.params, getProduct.handler)

This pattern has worked well for us, the functions are all registered in one place. And the definition, validation rules and implementation are in a single file, which makes it easy to focus on the function. There’s no need to switch through multiple files.

Demo Time

To demonstrate how all this works together, we have prepared a repository with all the necessary files here: hemera-nats-demo. Every service runs in a docker container, all services connect to the central NATS system, register their functions and can then be called by the API.

sequence diagram of a distributed function callsequence diagram of a distributed function call

Summary

NATS and hemera have helped us simplify a monolithic system into many different, small services. We subsequently increased our productivity, as developers are more confident while working on smaller services. Decisions and changes result in less technical impact and more user value.

Special thanks to the team behind NATS and Dustin Deus, the lead author of hemera.

We will write about unit and integration testing of distributed services in the next article. Follow us to get notified about the publication.

See Also