Configuration, enmasse and backups

Configuration, enmasse and backup get used as if they were one thing, and treating them as one leads to environments that cannot be recreated and backups that hold nothing worth keeping. They are three answers to three different questions, and this page walks through one integration to show where each of the three does its work.

Three questions

Which values do my services read right now? That is configuration - the files in config/user-conf, including config tables, read through self.config. These values change while an environment runs - a partner renames its codes, a threshold moves, a new source comes online - and none of those changes means the environment itself changed.

What does my environment consist of? That is enmasse - the REST channels, outgoing connections, security definitions and everything else the dashboard manages, exported to YAML and imported anywhere. The YAML is the environment's definition, which is why it belongs in git and goes through review - a change to it is a change to what the environment is.

What did this installation hold that cannot be rebuilt? That is a backup - and the answer is shorter than most people expect, because everything above is already in git and rebuilds from there.

One integration, three views

Take an API that receives subscriber status updates from roaming partners. It is a REST channel with an API key, a service behind it, and a config table that translates each partner's codes.

What the environment is - the channel and its security - is enmasse YAML:

security:
  - name: Partner Status API Key
    is_active: True
    type: apikey

channel_rest:
  - name: Partner Status
    service: api.process-status
    url_path: /api/partner/status
    security: Partner Status API Key

What the service reads - which partners exist and what their codes mean - is a config table:

# config/user-conf/partners.ini

[NORDIC_MOBILE]
ACTIVE = ACTIVE
BARRED = SUSPENDED
CLOSED = DEACTIVATED

[ALPINE_TELECOM]
IN_SERVICE  = ACTIVE
NON_PAYMENT = SUSPENDED
CANCELLED   = DEACTIVATED

Both files live in the project repository, next to the service's code. A new environment - a test server, a replacement for one that was lost, a second region - gets all three the same way: deploy the project, import the enmasse file, and the config files arrive with the mounted project directory. Nothing about this is a restore operation, and no copy of the old installation was needed.

What a backup of that installation would add is the part git does not have - the audit log records of every request the channel handled, and any SQL database running outside the container. Those cannot be rebuilt from the project, which is exactly why they are what a backup is for.

Keeping configuration is not backing it up

The question "how do I back up my configuration" usually means "how do I make sure a new environment gets my configuration", and the answer to that is not a backup - it is the project in git. The enmasse YAML and the config files are ordinary text files under version control, with history, review and the ability to recreate the environment on any machine, which is more than any backup of a running installation gives you.

The reverse holds too. A backup of a container's filesystem does not give you a reviewable definition of the environment - it gives you one frozen copy of one installation, with no history and no way to apply it partially. The backup page draws the full line: the project in git covers the definition, database backups cover the records, and the container itself needs nothing.

Where a change belongs

When a value changes - a partner adds a code, a cutoff moves - that is a config table edit, made on the Config tables screen or in the repository, live everywhere without a restart.

When the environment changes - a new channel, a new connection, a credential rotated - that is an enmasse change, made in the dashboard and exported, or written in the YAML directly, and imported where it should apply.

When neither changed and you still need yesterday's audit records, that is what the backup was for.

Learn more