Scaling your Meteor application with the cluster package
So you just released your brand new Meteor application to the public which is broadly adopted but also slowed down due to the high demand?
In this blog post we’ll take a look how you can scale your Meteor application with the help of the cluster package which was developed by the Meteorhacks guys.
Scaling a web application
Here are some things we might do to scale our web application / reduce the load on our web application:
- Extend the CPU / Memory of the server (Not recommended with Meteor due to single threaded Node.js background)
- Extend the database server with additional resources (this is interesting for Meteor applications because the data driven architecture relies heavily on the database)
- Add a CDN to cache static resources to reduce the load on the server
- Add additional servers and load balance the traffic to those servers
- …
As we’ve seen there are several things we can do. In this blog post we’ll focus on the addition of servers and traffic routing to those servers.
The cluster package
Before we dive into an example let’s take a closer look at the beauty of the cluster package.
Without the custer package we have to setup a load balancer at the server level. Meaning that we have to deal with e.g. nginx and the correct configuration to use load balancers in combination with application servers.
With the help of the cluster package we just need to add the package to our application, provide some servers and set the correct settings for the cluster package. The cluster package will then take care of traffic routing and cluster management. This is a huge improvement.
Let’s take a look at an example to get a better understanding how this works.
Scaling a Meteor application
Note: We just take a grasp look at the functionality of the cluster package here. The described setup will only route traffic to different application servers, but use one server as a single entry point which routes the traffic to those servers.
This is not recommended when you want a production ready setup.
For additional / production setups you might take a look at the packages documentation.
The first thing you have to do is to add the cluster package to your Meteor application.
Just run:
meteor add meteorhacks:cluster<br />to add the cluster package to your application.
After that you have the cluster functionality available in your app.
Run your app with the following variables:
CLUSTER_DISCOVERY_URL=mongodb://mongo-url CLUSTER_SERVICE=web CLUSTER_ENDPOINT_URL=http://ipaddresss:port meteor<br />Let’s take a closer look at the variables:
The CLUSTER_DISCOVERY_URL is the URL to a MongoDB database. The cluster package uses the MongoDB database to communicate between the Meteor applications. This database can be the Meteor application database (where all your other collections are stored) or a different one.
The CLUSTER_SERVICE variables stores a name for your application which is used internally by the package (just name it “web” and forget about it for now).
The CLUSTER_ENDPOINT_URL is the URL to your application.
That’s nearly everything we have to do. Just spin up additional applications and the cluster package will take care of traffic routing to those applications.
Here’s an example how two Meteor applications can be setup:
# The first application:<br />
CLUSTER_DISCOVERY_URL=mongodb://clustersettings:12345/dbname CLUSTER_SERVICE=web1 MONGO_URL=mongodb://maincollections:12345/dbname CLUSTER_ENDPOINT_URL=http://localhost:3000 meteor --settings settings.json --port 3000</p>
<p># The second application:<br />
CLUSTER_DISCOVERY_URL=mongodb://clustersettings:12345/dbname CLUSTER_SERVICE=web2 MONGO_URL=mongodb://maincollections:12345/dbname CLUSTER_ENDPOINT_URL=http://localhost:3000 meteor --settings settings.json --port 4000<br />Nice!
Other setups
As noted above: This setup is just an introduction to the cluster package and not recommended for production usage.
If you’re interested in more complex setups and want to dive deeper into the usage of the cluster package you should check out this blog post by Meteorhacks.
Additional resources
You want to know more about Meteor and scalability? Here are some additional resources you should definitely check out:
