Receiving files in a service

What each file transfer item contains and what happens when your service refuses a file.

A schedule invokes the service you named once for each file that is ready. The file has already been read by the time your service runs, so there is nothing to download and no connection to obtain - the contents are in the request.

What the service receives

The item arrives as the raw request:

from zato.server.service import Service

class ProcessInvoice(Service):

    def handle(self):

        item = self.request.raw_request

        self.logger.info('%s from %s, %s bytes', item.file_name, item.conn_name, item.size)

Everything the item contains:

Attribute	Notes
`conn_type`	Which kind of connection the file came from
`conn_name`	The name of the connection
`schedule_name`	The name of the schedule that picked the file up
`directory`	The directory the file was found in
`file_name`	The name of the file on its own, without any directories
`full_path`	The directory and the name together, which is what to log and what to quote in an error
`size`	The size in bytes, as the directory listing reported it
`last_modified`	The modification time, as an ISO-8601 string
`data`	The contents, as bytes

data is bytes, always. Text files need decoding, and the encoding is something you know and the file does not say:

from zato.server.service import Service

class ProcessOrders(Service):

    def handle(self):

        item = self.request.raw_request
        text = item.data.decode('utf8')

        for line in text.splitlines():
            self.logger.info(line)

One file, one invocation

Each file is a separate invocation with its own correlation ID, so a directory holding fifty files produces fifty invocations, each visible on its own in the logs and in the dashboard. Nothing batches them together and nothing hands your service a list.

The order of files within one run is the order the remote server listed them in, which is not something to depend on. If files have to be processed in a particular order, that order has to come out of their names or their contents.

Refusing a file

If your service raises, the file is refused. It is left exactly where it was, it is not moved and it is not deleted, and the next run offers it again. Nothing is ever lost because a service failed.

This is what to do with a file you cannot process - validate it, raise on what is wrong, and let it sit there until somebody fixes the file or the code:

from zato.server.service import Service

class ProcessInvoice(Service):

    def handle(self):

        item = self.request.raw_request
        text = item.data.decode('utf8')

        if not text.startswith('INVOICE'):
            raise Exception(f'Not an invoice file: `{item.full_path}`')

        self.logger.info('Taking %s', item.full_path)

A file that keeps being refused keeps coming back, run after run, so a permanently bad file is a permanent error in the logs until it is dealt with. If that is not what you want for a given feed, catch the problem in your own service, put the file's contents somewhere for later inspection, and return normally so the file is moved out of the way.

One refused file has no effect on the others. The run continues to the next file and the healthy ones go through as usual.

Passing the contents on

From here the file is data like any other, and the rest of the platform is available in the usual way - invoke another service, write to a database, publish to a topic, call a REST endpoint:

from zato.server.service import Service

class ProcessInvoice(Service):

    def handle(self):

        item = self.request.raw_request

        # Hand the contents to whichever service knows what to do with them
        self.invoke('billing.import-invoice', {
            'source': item.full_path,
            'received': item.last_modified,
            'data': item.data.decode('utf8'),
        })