Capacity planning

Disk space

Overview

Three contributors to Zato’s demands on disk space are

Logs

A single server running on INFO level will need at most several megabytes of disk space daily to store its logs assuming there are no errors or warnings to store.

Running in DEBUG mode will make it log about 5 times more information regarding Zato itself - in addition to the internal data logged any input/output accepted and produced on a given server will also be logged and these should also be taken into account.

A capacity plan should also consider how much information the deployed services produce - this will depend on the exact nature of a given solution and should be a matter of consultation between architects, admins and programmers.

SQL ODB

A fresh environment consisting of 1 cluster and 2 servers with 300 services each will take up at most 100 MB of disk space.

There are several columns that will impact the growth the most:

Column Max. size Notes
deployment_package.payload 5 MB

Each package that should be hot-deployed is stored in this column. There are as many rows in the table as there have been packages to hot-deploy.

For instance, 50 packages 3 MB each have been hot-deployed so far -> 150 MB are needed + the overhead of the database.

service.wsdl 5 MB

Each service can have a WSDL document attached.

For instance, 300 services have WSDL, 4 MB each -> 1.2 GB is needed + the overhead of the database.

job.extra 0.5 MB

Any extra data a scheduler’s job should be invoked with.

For instance, 100 jobs defined 0.2 MB of extra data each -> 20 MB are needed + the overhead of the database.

deployed_service.source 0.5 MB

Source code of each of the services deployed on each server.

For instance, 300 services 0.3 MB each are deployed on 2 servers -> 180 MB are needed + the overhead of the database.

The rest of the columns are simple integers and conservative varchars.

No logs are stored in the SQL ODB, no historical data, nothing really heavy, only configuration of various clusters re-using the same ODB.

Redis broker

Redis can be by far the biggest factor in discussing how much disk space Zato needs however it’s not possible to estimate it in a general way, each particular Zato environment needs to considered alone.

Statistics will always generate a lot of keys and you should use tools such as redis-sampler to get an understanding of how much they will take up.

Note that old and uneeded statistics can always need to be cleaned up.

CPU

Zato servers can use as many CPUs as you can offer - there are no hard limits or minimal requirements. The more CPU you will assign to systems Zato is running on the faster Zato itself will perform, provided main.gunicorn_workers is set to an optimal value.

Keep in mind though that adding more CPU will rarely make the whole environment run faster, for instance, if Zato is limited by poor response times from backend servers, throwing in more CPU won’t achieve anything.

Additionally, Zato servers use modern, asynchronous libraries and they can withstand hundreds and thousands of connections on just a few CPU so it’s recommended that you start with a smaller number of processors and add more only when they’re needed.

Assuming the load-balancer has its own dedicated system, that system should have one fast CPU only, more CPUs aren’t needed and indeed, won’t be used.

RAM

You should allow for at least 50 MB of RAM for each worker process, however the upper limit will depend on how big the traffic is.

The load-balancer is very light on RAM and you should start with 256 MB and assign more if it’s needed.

As with disk space - Redis can take a lot of RAM but how much it can be depends on the exact usage patterns and can’t be predicted reliably - it’s recommended that during development and tests an estimation of how much RAM the given Zato environment needs be obtained.