Managing server objects en masse

Overview

In addition to the GUI, CLI and API, you can use the enmasse command line tool to manage server objects as a group. This lets you store each object configuration in YAML or JSON files, in a repository where they can be versioned or backed up and to use enmasse to perform roll-outs from command line without the use of GUI or other methods.

Enmasse handles backup and restore of server objects, including channels, scheduler jobs, security definitions, etc. It specifically does not manage user services; service management is documented in a separate chapter.

Tasks you can use enmasse for

  • Exporting server objects from a cluster as text in YAML or JSON format
  • Preparing cluster configuration and merging it with a contents of an already existing cluster
  • Importing objects from an export file into clusters, optionally replacing objects that already exist

Usage scenarios

  • You create a development cluster and quickly prototype a solution using the GUI. For tests and production, however, you prefer to store the config in a git repository. Hence you use enmasse to export all server objects from a dev cluster into JSON. The JSON is next imported to tests and later on, on production.
  • You have an already existing cluster and want to add or replace server objects. You prepare a JSON config file and import it to a cluster.
  • You have a team of programmers each responsible for a part of a solution and you want to limit the number of conflicts in a central config repository. You split the local config file into multiple smaller ones, each of which can be included by enmasse to form a whole that will be imported as a single unit.

Workflow

  • Export objects from an already existing cluster to a file
  • Customise the file, optionally splitting it into smaller units that can be included later on.
  • Combine all the includes, optionally merging ODB objects from a cluster of choice, into an export file. This is a file that can be imported to clusters.
  • Import objects from an export file, optionally replacing objects that already exist in a cluster.

Notes

  • Path to a server is always required and serves a double role.

    If you are exporting data it must be path to a running server you are exporting data from.

    If you are importing data, the path must point to a running server you are importing data into.

  • For exports, –input means an input file out of which an export file should be created.

    For imports, –input is an export file out of which definitions should be imported on a cluster.

  • For imports, the order enmasse works in is guaranteed to be:

    • Existing definitions are updated
    • Existing objects other than definitions are updated
    • New definitions are created
    • New objects other than definitions are created

    In this way, objects that depend on definitions can be sure the definitions that they need already exist in the ODB.

  • When creating an export file, enmasse tries to report as many warnings and errors in one message as possible. When importing a file, it does the contrary, and will stop as soon as it encounters anything it can not deal with to give you a chance to have a look at an erroneous situation as early as it can be done.

What arguments to use depending on a use-case

Use case Arguments to use
Merge a file containing includes into a single file –export-local
Export ODB objects to a file –export-odb
Merge objects from a file and ODB into a new file –export-local –export-odb
Ignore that a resulting file has some definitions missing –ignore-missing-defs
Import from a file, quitting early if any object already exists –import
Import from a file, replacing objects that already exist –import –replace-odb-objects

Usage samples

  • Creating an export file from a running server

    $ zato enmasse /opt/server1/ --export-odb
    ODB objects read
    Data exported to /opt/zato/export.yml
    $
    
  • Importing an export file on a cluster - any existing objects will be updated in place

    $ zato enmasse /server1/ --input export.yml --import --replace-odb-objects
    Updated object 'myjmswmq2' (def_jms_wmq)
    Updated object 'myjmswmq3' (def_jms_wmq)
    Updated object 'channel-amqp1' (channel_amqp)
    Updated object 'channel-amqp2' (channel_amqp)
    Updated object 'ftp1' (outconn_ftp)
    Updated object 'ftp2' (outconn_ftp)
    Created object 'channel-myhttp3' (http_soap)
    Created object 'myhttp1' (http_soap)
    Created object 'myhttp2' (http_soap)
    $
    
  • Importing an export file on a cluster - because “–replace-odb-objects” is not set, any updates to already existing objects will be rejected

    $ zato enmasse /server1/ --input export.yml --import
    2 warnings and 0 errors found:
    +-----------------+--------------------------------------------------------------+
    |       Key       |                            Value                             |
    +=================+==============================================================+
    | warn0001/W01    | {'name': 'channel-amqp1', 'service': 'my-service',           |
    | already exists  | 'is_active': False, 'queue': 'MYQUEUE',                      |
    | in ODB          | 'consumer_tag_prefix': 'myconsumer', 'def': 'myamqp1'}       |
    |                 | already exists in ODB {'def_name': 'myamqp1', 'name':        |
    |                 | 'channel-amqp1', 'service_name': 'zzz.my-service',           |
    |                 | 'is_active': False, 'data_format': None, 'queue':            |
    |                 | 'MYQUEUE', 'def_id': 3, 'consumer_tag_prefix':               |
    |                 | 'myconsumer', 'id': 1} (channel_amqp)                        |
    +-----------------+--------------------------------------------------------------+
    | warn0002/W01    | {'name': 'channel-amqp2', 'service': 'my-service',           |
    | already exists  | 'is_active': False, 'queue': 'MYQUEUE',                      |
    | in ODB          | 'consumer_tag_prefix': 'myconsumer', 'def': 'myamqp1'}       |
    |                 | already exists in ODB {'def_name': 'myamqp1', 'name':        |
    |                 | 'channel-amqp2', 'service_name': 'my-service',               |
    |                 | 'is_active': False, 'data_format': None, 'queue':            |
    |                 | 'MYQUEUE', 'def_id': 3, 'consumer_tag_prefix':               |
    |                 | 'myconsumer', 'id': 2} (channel_amqp)                        |
    +-----------------+--------------------------------------------------------------+
    $