Fabric notebooks and pipelines

Run a notebook and read what it wrote, run a pipeline and refresh a report, and let Fabric read your own systems.

This page will show you how to work with Microsoft Fabric notebooks and pipelines in Python:

  • Running a notebook and reading its results
  • Running a pipeline and refreshing a report
  • Letting a notebook read your own systems

In Fabric, running a notebook, running a pipeline and refreshing a report's data are all jobs, and a job takes minutes, so working with one has two parts:

  • conn.run_job starts the job and returns right away with its ID
  • conn.wait_for_job takes that ID and waits until the job is done, raising an exception if it failed or took too long

This way, a service can start a job and continue with other work while Fabric runs it, or call the two one after another to simply wait for the result, which is what the services on this page do.

Connection and IDs

The app registration, the Fabric connection in the Zato Dashboard, and the workspace and lakehouse IDs are the same as in the tutorial, so set them up as described there first.

The jobs also need the ID of the item they run, which is in the browser's address bar when the item is open in Fabric:

ItemIn the address bar afterUsed on this page
Notebook/synapsenotebooks/ba7b1e23-b680-4fe7-a982-2a979b125604
Pipeline/pipelines/bf4868aa-05c8-4213-ab4d-8dcbe3189767
A report's semantic model/datasets/e040a801-73bd-4b01-b036-d490c7122941

A report reads its data from a semantic model, which in the workspace list is the item with the same name as the report but a different icon.

Running a notebook and reading its results

In this example, a notebook in Fabric works out which of tomorrow's appointments need a reminder and writes them to the reminder_candidates table, and each morning a scheduled service needs those rows to send the reminders out.

The service below will:

  • Start the notebook with conn.run_job
  • Wait until it is done with conn.wait_for_job
  • Read the table the notebook wrote with conn.query
# -*- coding: utf-8 -*-

# Zato
from zato.server.service import Service

class SendReminders(Service):

    name = 'fabric.send-reminders'

    def handle(self):

        # The IDs from the tutorial and from the table above ..
        workspace_id = '262ddd3d-3d0d-4495-b80f-6dddda1e23a1'
        lakehouse_id = '73811955-064d-481a-a7e9-f9c563124f9e'
        notebook_id = 'ba7b1e23-b680-4fe7-a982-2a979b125604'

        # .. get a Fabric connection ..
        conn = self.microsoft.fabric['Zato Fabric']

        # .. start the notebook ..
        job_id = conn.run_job(workspace_id, notebook_id, 'RunNotebook')

        # .. wait until it is done, a notebook takes a few minutes ..
        conn.wait_for_job(workspace_id, notebook_id, job_id)

        # .. read the table it wrote ..
        sql = """
        select appointment_id, location, starts_at
        from reminder_candidates
        order by starts_at
        """
        rows = conn.query(workspace_id, lakehouse_id, sql)

        # .. in a real service, this is where each reminder would be sent ..
        for row in rows:
            appointment_id = row['appointment_id']
            location = row['location']
            self.logger.info(f'Reminder -> {appointment_id} at {location}')

        # .. and tell our caller how many there were.
        self.response.payload = {'reminders': len(rows)}

After invoking the service you'll see:

{"reminders": 9}

Running a pipeline and refreshing a report

In this example, once the nightly load has put the day's rows into the lakehouse, a pipeline in Fabric reshapes them into the occupancy table that a Power BI report reads, and then the report's data has to be refreshed, because a report shows what it loaded last, not what the table holds now.

The service below will:

  • Start the pipeline and wait for it
  • Start the refresh of the report's semantic model and wait for it too
  • Return when both are done, so a caller knows the report is current
# -*- coding: utf-8 -*-

# Zato
from zato.server.service import Service

class RefreshOccupancyReport(Service):

    name = 'fabric.refresh-occupancy-report'

    def handle(self):

        # The IDs from the tutorial and from the table above ..
        workspace_id = '262ddd3d-3d0d-4495-b80f-6dddda1e23a1'
        pipeline_id = 'bf4868aa-05c8-4213-ab4d-8dcbe3189767'
        dataset_id = 'e040a801-73bd-4b01-b036-d490c7122941'

        # .. get a Fabric connection ..
        conn = self.microsoft.fabric['Zato Fabric']

        # .. run the pipeline and wait for it ..
        job_id = conn.run_job(workspace_id, pipeline_id, 'Pipeline')
        pipeline = conn.wait_for_job(workspace_id, pipeline_id, job_id)

        # .. refresh the report's data and wait for that too ..
        job_id = conn.run_job(workspace_id, dataset_id, 'DefaultSemanticModelRefresh')
        refresh = conn.wait_for_job(workspace_id, dataset_id, job_id)

        # .. and tell our caller when each of them ended.
        self.response.payload = {
            'pipeline_ended': pipeline['endTimeUtc'],
            'refresh_ended': refresh['endTimeUtc'],
        }

After invoking the service you'll see:

{"pipeline_ended": "2026-09-02T02:34:11Z", "refresh_ended": "2026-09-02T02:36:40Z"}

If the pipeline fails, conn.wait_for_job raises an exception with Fabric's reason and the refresh does not start, so the report keeps showing last night's data rather than a half-loaded one.

Letting a notebook read your own systems

In this example, the appointments live in a database in your own network and a notebook in Fabric needs them, so a service reads the database and a REST channel publishes the service over HTTPS, where the notebook reads it like any other API.

The service below will:

  • Take a since time from its caller, so that the notebook reads what changed rather than everything
  • Read the appointments from the local database through a SQL connection
  • Return them, and the channel turns them into JSON
# -*- coding: utf-8 -*-

# Zato
from zato.server.service import Service

class GetAppointments(Service):

    name = 'fabric.get-appointments'
    input = 'since'

    def handle(self):

        # The query to run ..
        sql = """
        select appointment_id, location, starts_at, status
        from appointments
        where updated_at >= :since
        order by starts_at
        """

        # .. the value for it, from our input ..
        params = {
            'since': self.request.input.since,
        }

        # .. get a connection to the local database ..
        conn = self.out.sql['Appointments DB']

        # .. run the query ..
        rows = conn.execute(sql, params)

        # .. and return the rows to our caller.
        self.response.payload = rows

The channel is created under Connections → REST → Channels in the Dashboard, with the service above, the URL path /api/appointments and an API key that the notebook will send.

In the notebook, one cell reads the channel and turns the rows into a DataFrame:

# In a Microsoft Fabric notebook
import requests

url = 'https://api.example.com/api/appointments'
headers = {'X-API-Key': '<the key>'}
params = {'since': '2026-08-31T00:00:00Z'}

response = requests.get(url, headers=headers, params=params)
appointments = response.json()

frame = spark.createDataFrame(appointments)
display(frame)

See also

PageWhat it covers
Your first Fabric integrationThe app registration, the connection and where the IDs come from
Fabric API - Jobsconn.run_job, conn.wait_for_job, conn.get_job and conn.cancel_job, job types and statuses
Writing data to FabricThe nightly load the pipeline runs after

Learn more

Development documentation Everything about writing services, in one place
Requests and responses What a service receives, what it returns and how to shape both
Integration examples Ready-made code for the systems you are likely to connect to
IDE and debugging Write services in the Dashboard or in your own editor
Data models Declare inputs and outputs and have them validated for you
In-depth API tutorial The full platform tutorial, from installation to production patterns