Python Microsoft Fabric - Sending reports on a schedule

Query Fabric once a month, build a CSV and deliver it by SFTP and email.

On the first of each month, one scheduled service queries a lakehouse table for last month's totals, builds a CSV from them and delivers it to an SFTP server and by email.

The table in Fabric

The query reads last month's rows of the invoices table:

The service

The query groups last month's rows, the result becomes CSV, and the file goes out twice - to the SFTP server through an SFTP connection and as an attachment through an SMTP connection, both created in the Dashboard beforehand.

# -*- coding: utf-8 -*-

# stdlib
import csv
import io
from datetime import date, timedelta

# Zato
from zato.common import SMTPMessage
from zato.server.service import Service

class MonthlyInvoiceSummary(Service):

    name = 'reports.monthly-invoice-summary'

    def handle(self):

        workspace_id = '11111111-2222-3333-4444-555555555555'
        lakehouse_id = '66666666-7777-8888-9999-000000000000'

        conn = self.microsoft.fabric['My Fabric']

        # Last month - from its first day up to, not including, the first day of this month.
        today = date.today()
        month_start = today.replace(day=1)
        previous_month_end = month_start - timedelta(days=1)
        previous_month_start = previous_month_end.replace(day=1)
        month_label = previous_month_start.strftime('%Y-%m')

        range_start = previous_month_start.isoformat()
        range_end = month_start.isoformat()

        # One row per insurer - what was invoiced and what has been paid.
        sql = f'''
        select
            insurer,
            count(*) as invoices,
            round(sum(amount), 2) as invoiced,
            round(sum(case when status = 'Paid' then amount else 0 end), 2) as paid
        from invoices
        where sent_at >= '{range_start}'
          and sent_at < '{range_end}'
        group by insurer
        order by insurer
        '''

        rows = conn.query(workspace_id, lakehouse_id, sql)

        # Turn the rows into CSV ..
        buffer = io.StringIO()
        writer = csv.DictWriter(buffer, fieldnames=['insurer', 'invoices', 'invoiced', 'paid'])
        writer.writeheader()
        writer.writerows(rows)

        text = buffer.getvalue()
        file_name = f'invoice-summary-{month_label}.csv'

        # .. put it where the finance system picks files up ..
        remote_path = f'/reports/{file_name}'

        sftp = self.sftp['Finance SFTP']
        sftp.write(text, remote_path)

        # .. and email it.
        msg = SMTPMessage()
        msg.subject = f'Invoice summary {month_label}'
        msg.to = 'finance-reports@example.com'
        msg.from_ = 'zato@example.com'
        msg.body = f'The invoice summary for {month_label} is attached.'
        msg.attach(file_name, text)

        smtp = self.email.smtp.get('Reports Email')
        smtp.conn.send(msg)

        self.response.payload = {'file': file_name, 'rows': len(rows)}

Scheduling it

In the Dashboard, go to Scheduler → Config, click "Create a new job", pick the cron-style type with 0 6 1 * * - six in the morning on the first day of each month - and reports.monthly-invoice-summary as the service. The scheduler page shows the form and how to create the same job from code.

The file

Run for March, the CSV has one line per group of the query:

insurer,invoices,invoiced,paid
Cascade Health Plan,2,2895.50,2275.00
Evergreen Assurance,1,415.75,0.00
Pacific Mutual,1,1840.00,1840.00

See also

PageWhat it covers
Sending and receiving filesThe same CSV written into the lakehouse for the Fabric team instead
Building an API on top of Fabric dataThe same query, answered on demand over REST
Queriesquery and Spark sessions in detail

Learn more