Using environment variables

There are several ways in which Zato components can receive and use environment variables. A basic use case is to export them in ~/.bashrc of the "zato" user or to provide them to a starting container's environment. Then, they can be used by your integration services if you access them through os.environ, which is how environment variables are accessed in Python.

Independently of the above, it is possible for environment variables to dynamically alter configuration of your components. It is also possible for environment variables to be provided in a dedicated file instead of exporting them through ~/.bashrc or Docker.

Dynamic environment variables

Each Zato component has one or more configuration files that can be used to fine-tune its operations in a specific environment. For instance, servers have files such as server.conf or simple-io.conf and your scheduler has a file called scheduler.conf.

It is a common need to modify only selected parts of these files, i.e. the majority of it stays as it is but just a few parts, such as TCP ports, should be changed.

One way to do it is custom scripting. For instance, your CI/CD pipeline could use sed, or a similar tool, to replace the parts that need to be updated. This works, but an alternative to it, built into Zato, is the usage of dynamic environment variables that will take precedence over the values from config files.

Let's have a look at a part of a typical server.conf file. This specific example is responsible for the host and port that a Zato server listens on and, in this case, we would like to change it to listen on "127.0.0.1:17030".

[main]
gunicorn_bind=0.0.0.0:17010

You can achieve it by exporting an environment variable in this format before a server is started:

Zato_Config_Server_server__conf_main_gunicorn_bind="127.0.0.1:17030"

Let's break down the name of this variable:

Zato_Config_Server - Indicates that we are overriding a variable of a server configuration file.
server__conf - Name of the file that contains the variable that we are overriding. Note that the dot needs to replaced with double underscores because Bash environment variables cannot have dots, dashes, and similar punctuation, in their names.
main - Corresponds to the "[main]" section in the configuration file.
gunicorn_bind - Name of the key whose value we are setting.
127.0.0.1:17030 - The new value for the key.

Note that the file will not be changed on disk yet the server will recognize the value. That is why these variables are called dynamic - they can point to any key in any .ini configuration file without actually changing the file itself.

Here are a few more examples of overriding the default configuration from .ini files:

Zato_Config_Server_sso__conf_password_expiry="30"
Zato_Config_Server_server__conf_odb_extra="echo=True"
Zato_Config_Server_simple__io__conf_secret_prefix="my_prefix1_, my_prefix2_"
Zato_Config_Scheduler_scheduler__conf_bind_port=31597
Zato_Config_Scheduler_scheduler__conf_server_server_port=17099

You can also insert additional keys into configuration sections in the same way. This is useful with the parts of configuration that do not use static keys.

For instance, in the scheduler.conf file of each scheduler, there is a section called [command_pause] which lets you control what API users can issue the "pause" command to the scheduler. By default, this section is empty, but you can add a new key to it, called "client1", without modifying the file, using an environment variable such as this one:

Zato_Config_Scheduler_scheduler__conf_command_pause_client1="/api/8znc8ytrbm5/pause"

Remember that the variables need to be exported before a given component starts. How to do it exactly will depend on how you deploy Zato - it may mean adding them to the ~/.bashrc file of the "zato" user or, if you use Docker Quickstart, it may mean that they need to be provided to a starting container, or you can provide them in a special file, as described below.

Reading environment variables from a file

When you start a Zato component, you can provide an option called "--env-file" from the command line. This option needs to point to a file with a list of environment variables that the component should make use of.

In this way, you do not need to export the variables through ~/.bashrc or Docker. Instead, you can prepare a dedicated file with environment variables, which in certain situations may be more convenient than having to export them.

For instance, to start a server and make it import environment variables from a file:

zato start ~/env/qs-1/server1 --env-file=/opt/zato/env.ini

Such a file with environment variables is in the format as below:

[env]
key1=value1
key2=value2
key3=value3

That is, there is a section called "[env]" and a list of variables and their values follows, with each key/value pair in its own line. There are no limits as to how many keys and values you can place in the file. The file's name needs to be "env.ini" and it can be placed in any directory as long as this specific name is used.

Note that you can still combine this option with Docker. For instance, you can generate the file on your host based on the configuration that is available on host and then map it to the container and start a server with "--env-file" pointing to the file mapped from the host.

The "--env-file" is also available in the enmasse command. Thanks to this, you can reuse the same file with variables when provisioning an environment with enmasse and when Zato components are started.

Hot-deploying environment variables

The contents of the env.ini file, described in the previous section, is re-read by a server each time the file is modified. This lets you hot-deploy environment variables that your services make use of without restarting the servers.

Note, too, that if your configuration files depend on environment variables, the config files have to be redeployed after the env.ini file is hot-deployed.

For instance, let's say you have a config file called "my.ini" with a key of "MyPassword" which points to an environment variable called "Corp_My_Password" and the environment variable is defined in env.ini.

If you modify the password in the env.ini file, and you want for the contents of my.ini to be redeployed, you need to modify my.ini too. You do not need to change its contents, it will suffice if you use the "touch" command, e.g. "touch /path/to/my.ini" will redeploy my.ini with the new password from env.ini.