Python Microsoft Fabric - Loading data into tables
Append new rows every night, replace small tables whole and load files that are already in the lakehouse.
This page loads data into lakehouse tables in three ways - appending the day's new rows to a table that grows, replacing a small table whole with the current list, and turning a file that is already in the lakehouse into a table.
Two kinds of tables
A lakehouse is where Fabric keeps tables and files together, and the tables in it fall into two kinds. Some record what happened, e.g. orders, calls or transactions, and they only ever grow, so new rows are appended to them. Others describe things, e.g. sites, products or customers, and they are small, so the whole table is replaced with the current list whenever it changes.
Appending the day's rows
The service below runs once a night. It keeps the time it stopped at in the cache, takes everything recorded since then, appends the rows and moves the marker forward.
Before it runs, the admissions table holds three rows:
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class AppendAdmissions(Service):
def handle(self):
workspace_id = '11111111-2222-3333-4444-555555555555'
lakehouse_id = '66666666-7777-8888-9999-000000000000'
conn = self.microsoft.fabric['My Fabric']
# Where the previous run stopped ..
since = self.cache.get('admissions.loaded_until')
# .. the rows recorded since then, from the system that owns them ..
rows = [
{
'admission_id': 'ADM-1041',
'location': 'Riverside',
'admitted_at': '2026-03-11T07:15:00Z',
'discharged_at': '',
'status': 'Admitted',
},
{
'admission_id': 'ADM-1042',
'location': 'Riverside',
'admitted_at': '2026-03-11T09:40:00Z',
'discharged_at': '',
'status': 'Admitted',
},
]
# .. add them to what the table already holds ..
conn.write_table(workspace_id, lakehouse_id, 'admissions', rows, mode='Append')
# .. and remember the last timestamp for the next run.
last_row = rows[-1]
self.cache.set('admissions.loaded_until', last_row['admitted_at'])
self.response.payload = {'appended': len(rows), 'since': since}
After the run, the two new rows are at the bottom of the table:
write_table takes any number of rows. It writes them as CSV files to the lakehouse's Files section, 100,000 rows per file, then asks the lakehouse to load all of the files into the table in one operation and returns when that operation is done.
Replacing a small table
The system that owns the list answers with its own field names. The service maps them to the table's columns and replaces the table whole - no mode needed, replacing is the default.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class ReplaceLocations(Service):
def handle(self):
workspace_id = '11111111-2222-3333-4444-555555555555'
lakehouse_id = '66666666-7777-8888-9999-000000000000'
conn = self.microsoft.fabric['My Fabric']
# The reply from the system that owns the list, in its own field names.
items = [
{'id': 'LOC-01', 'label': 'Riverside', 'town': 'Portland', 'region': 'OR', 'bed_count': 120},
{'id': 'LOC-02', 'label': 'Oak Hill', 'town': 'Salem', 'region': 'OR', 'bed_count': 80},
{'id': 'LOC-03', 'label': 'Maple Grove', 'town': 'Eugene', 'region': 'OR', 'bed_count': 60},
]
# Rename the fields to the table's columns ..
rows = []
for item in items:
row = {
'location_id': item['id'],
'name': item['label'],
'city': item['town'],
'state': item['region'],
'beds': item['bed_count'],
}
rows.append(row)
# .. and replace the table with the current list.
conn.write_table(workspace_id, lakehouse_id, 'locations', rows)
self.response.payload = {'locations': len(rows)}
Loading a file that is already there
Sometimes the data is already in the lakehouse - another team dropped a CSV in Files/incoming - and it only needs to become a table. load_table starts that load and returns the address where its progress can be checked, wait_for_operation waits there until the load is done.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class LoadAdmissionsFile(Service):
def handle(self):
workspace_id = '11111111-2222-3333-4444-555555555555'
lakehouse_id = '66666666-7777-8888-9999-000000000000'
conn = self.microsoft.fabric['My Fabric']
# Start loading the file into the table ..
location = conn.load_table(
workspace_id,
lakehouse_id,
'admissions',
'Files/incoming/admissions-2026-03-11.csv',
mode='Append',
)
# .. and wait until Fabric reports that it is done.
operation = conn.wait_for_operation(location)
self.response.payload = {'status': operation['status']}
The path is relative to the lakehouse. The file is read as CSV with a header row by default, file_format='Parquet' reads Parquet instead, and a whole folder of files loads in one call with path_type='Folder' and recursive=True.
Each load shows up in the lakehouse's list of recent runs, which is where to look when a load takes longer than expected:
See also
| Page | What it covers |
|---|---|
| Keeping lookup tables up to date | One nightly job that replaces several small tables |
| Sending and receiving files | Files dropped for you, files you write for the Fabric team |
| Tables | write_table, load_table, list_tables and wait_for_operation in detail |