API Reference¶

A practical tour of what the library gives you: the two clients, authentication, the query patterns that make it ergonomic, and the power-user features (proxy, custom SSL, timeouts, concurrent batching, CSV export, throttling).

The two clients¶

	`ClientContext`	`GraphClient`
Target API	SharePoint REST API v1	Microsoft Graph API
Use for	SharePoint lists, files, folders, search, site admin, permissions	Outlook, OneDrive, Teams, OneNote, Planner, Users, Groups

from office365.graph_client import GraphClient
from office365.sharepoint.client_context import ClientContext

graph = GraphClient(tenant="contoso.onmicrosoft.com").with_client_secret("client_id", "client_secret")

ctx = ClientContext("https://contoso.sharepoint.com/sites/team").with_client_secret(
    "contoso.onmicrosoft.com", "client_id", "client_secret"
)

Authentication¶

Every modern flow is built in — client secret, certificate, username/password, device code, and interactive (plus on-prem NTLM and cookies for SharePoint):

Flow	`GraphClient`	`ClientContext`
Client secret	`with_client_secret(client_id, secret)`	`with_client_secret(tenant, client_id, secret)`
Certificate	`with_certificate(client_id, thumbprint, key)`	`with_client_certificate(tenant, client_id, thumbprint, cert_path)`
Username / password	`with_username_and_password(client_id, user, password)`	`with_username_and_password(tenant, client_id, user, password)`
Device code	`with_device_flow(client_id)`	`with_device_flow(tenant, client_id, scopes)`
Interactive	`with_token_interactive(client_id)`	`with_interactive(tenant, client_id, scopes)`
On-prem (NTLM)	—	`with_user_credentials(username, password)`
Cookies / custom	`with_token_callback(...)`	`with_cookies(...)`, `with_access_token(...)`

See the auth examples for full, runnable flows.

Query patterns¶

Filtering¶

OData $filter works identically on both clients:

guests = client.users.filter("userType eq 'Guest'").get().execute_query()
files = ctx.web.lists.get_by_title("Documents").items.filter("FSObjType eq 0").get().execute_query()

For complex SharePoint queries, use a CAML query instead:

from office365.sharepoint.listitems.caml.query import CamlQuery

qry = CamlQuery()
qry.ViewXml = (
    "<View><Query><Where><Eq><FieldRef Name='Status'/><Value Type='Text'>Active</Value></Eq></Where></Query></View>"
)
items = ctx.web.lists.get_by_title("Tasks").get_items(qry).execute_query()

Projection¶

select() takes the names the service uses, and those names are declared on the model, so you do not have to type them:

from office365.outlook.calendar.events.event import Event
from office365.runtime.schema import TypeSchema

events = client.me.calendar.events.select(TypeSchema.of(Event).columns).get_all().execute_query()

TypeSchema.of(Event) answers those questions for a type: .columns is every column $select accepts, .expansions the navigation properties it does not — attachments, instances, singleValueExtendedProperties — .readonly the columns the service assigns (so .serializable is what a request sends), and .name_of("ical_uid") the wire name one attribute is declared under. All four are derived from the same declarations the getters read, so a projection cannot drift from the code that consumes it: a name like iCalUId (which no snake-case rule derives from ical_uid) is written once, on the getter, and asking for an attribute that has no declaration raises AttributeError instead of guessing.

A projection narrows what the object knows, not only what the service sends back: after select(["Title"]) a web has no Url at all. web.absolute_url answers with whichever identity spelling the read asked for -- Url first, ServerRelativeUrl second, None when neither was read -- so code that needs the address does not re-derive which columns somebody else projected.

Paging¶

get_all() follows server-driven paging (@odata.nextLink) so you never write paging code:

for user in client.users.get_all().execute_query():
    print(user.user_principal_name)

SharePoint collections expose paged(page_size, page_loaded) and get_all(page_size, page_loaded):

all_items = ctx.web.lists.get_by_title("Contacts_Large").items.get_all(500).execute_query()

Resuming a delta query¶

A Graph delta collection tracks its own cursor: after a page, delta_token returns the resumable token from the @odata.deltaLink. Pass it back into token(...) to pick up where the last run stopped:

changes = client.me.drive.root.delta.get().execute_query()
token = changes.delta_token  # persist this between runs
# ... later ...
more = client.me.drive.root.delta.token(token).get().execute_query()

A resume token already encodes the query it came from, so Graph rejects a request that carries both a token and a $filter with 400 DeltaFilterNotAllowed. The library therefore withholds $filter while resuming (and warns if a filter is still attached), which also covers the common "same collection, next run" case.

Batching¶

Queue operations and submit them as a single OData $batch request — and run several batches concurrently to cut wall time on large imports:

target_list = ctx.web.lists.get_by_title("Documents")
for i in range(100):
    target_list.add_item({"Title": f"Item {i}"})
ctx.execute_batch(items_per_batch=100)  # one $batch request
ctx.execute_batch(concurrency=5)  # 5 batches in flight, retries on 429

Graph batches run their sub-requests in parallel by default. When order matters — read-after-write on the same resource, delete-then-create, parent/child ordering — pass sequential=True to chain them with Graph's dependsOn:

graph.execute_batch(sequential=True)  # ordered; a failed dependency yields 424

Use it only for ordering-dependent side effects (parallel is faster for independent requests). A batch can't reference another sub-request's response, so sequential doesn't enable create-then-use-id flows — those need two round-trips. sequential=True implies concurrency=1 (Graph sequences within a single $batch).

A rejected sub-request normally fails the whole batch. When the writes are independent and you would rather report them per item than abort the run, open a tolerant scope — the rejected sub-request is recorded against its own query and the rest still execute:

target_list = ctx.web.lists.get_by_title("Documents")
queued = {}
with ctx.collect_batch_failures() as failures:
    for row in rows:
        queued[id(target_list.add_item(row))] = row["Title"]
    ctx.execute_batch()

for query, error in failures:
    print(f"{queued.get(id(query.return_type), '?')}: {error}")

Each recorded failure is (query, exception), and query.return_type is the object your queued call returned — so pair items with queries by identity, as above. Throttled sub-requests (429/503) are still retried inside the scope; only a failure that is still standing (denied, invalid, retries exhausted) is recorded. Outside the scope the default is unchanged: the first failure raises ClientRequestException.

Throttling & retries¶

Microsoft 365 throttles with HTTP 429 + a Retry-After header; Graph also reports x-ms-throttle-limit-percentage, x-ms-resource-unit, x-ms-throttle-scope and x-ms-throttle-information. The library honors Retry-After and otherwise falls back to exponential backoff with jitter:

result = client.users.top(10).get().execute_query_retry()  # per query
client.execute_batch(concurrency=5)  # retries throttled sub-requests

Make it the default for a whole client — every execute_query on the context then retries transient failures, including deferred requests built internally by ClientObject / ClientResult helpers:

client.with_auto_retry()  # opt-in; off by default
client.me.get().execute_query()  # retries 429/5xx with backoff + Retry-After

Retry any callable with the same policy — call it, or decorate one:

from office365.runtime.retry import retry


@retry(max_retry=5, timeout_secs=2)
def do_request(): ...

Pace a fleet proactively — the shared rate limiter reads the throttle signals (Retry-After, Graph's x-ms-throttle-limit-percentage, SharePoint's health score) on every response and gates the group before it hits 429:

client.with_rate_limit()  # or .with_rate_limiter(shared_limiter)

Mark background work as low priority so it's throttled before user-visible calls (Graph doesn't change the limits — low is throttled first, high last):

client.with_throttle_priority("low")  # low | normal | high

See Throttling for the signals, the quota reference and the best practices.

Limits¶

Service limits are first-class: declare them on the model with @limit, enforce them at a call site, and discover them from any resource or tool.

from office365.limits import catalog, catalog_products, limit, limits_of, verify_limits
from office365.sharepoint.thresholds import Limits


@limit(Limits.LIST_VIEW, arg="page_size")  # declare + enforce (warn by default)
def get_items(page_size=None): ...


lst.get_items(page_size=6000)  # warns; the limit is discoverable
lst.declared_limits()  # the limits declared on the class
verify_limits(lst.get_items, page_size=6000).ok  # False
catalog_products()  # ['graph', 'directory', 'outlook', 'teams', ...]
catalog("sharepoint")  # every registered SharePoint limit, as data
catalog("outlook")  # ... and every Outlook throttling quota

Mechanics — office365.limits re-exports Limit (a static threshold or a rate quota), the @limit decorator, and the guardrails (exceeds / warn_if_exceeds / ensure_within / hint).
Catalogs — sharepoint.thresholds.Limits for boundaries and thresholds, and one quotas module per Graph product (outlook, teams, reports, security, onedrive, onedrive.workbooks, directory, communications) for throttling quotas. A model references a catalog row rather than re-spelling it, and it does so where the request is made: @limit(*CALL_QUOTAS) on Call, @limit(*CALL_RECORD_BUDGETS) on CallRecord, @limit(CALL_RECORD_RESOURCE_QUOTA) on CallRecord.sessions and @limit(*PSTN_RECORD_QUOTAS) on CallRecordCollection.get_direct_routing_calls. Graph quotas are reference data: the library reads the server signals (Retry-After, x-ms-throttle-*, RateLimit-*) and never simulates the token bucket.
Registry — products register their catalogs (register_catalog); @limit on a class registers it; catalog(product) aggregates them and catalog_products() lists the keys. Importing office365.limits imports every catalog, so discovery doesn't depend on which product the caller touched first.
Migration — assessment findings carry the authoritative Limit (AssessmentIssue.limit) alongside the product-specific risk_code.

See Service limits for the SharePoint catalog and the Graph throttling quota tables.

Capabilities¶

Four facts about the same type are declared in four places — the shape on the model (@odata, @facet), the requirement beside the request it describes (@require_permission, and on the same declaration its licences), the ceilings (@limit) and the query rules (@query_capabilities). capabilities_of reads them together; it declares nothing and sends no request.

from office365.capabilities import capabilities_of
from office365.communications.callrecords.collection import CallRecordCollection
from office365.outlook.mail.messages.message import Message

caps = capabilities_of(CallRecordCollection)
caps.schema.columns  # the shape (empty: a collection has no columns of its own)
caps.declarations  # entry point -> PermissionRequirement
caps.licenses  # every service plan those declarations name, once each
caps.limits  # the ceilings, still as declarations, so `arg` survives
caps.queries.supports("top")  # False — the callRecords feed rejects $top

capabilities_of(Message.send).declarations["send"].delegated  # ('Mail.Send',)

A type answers all four; a single call answers its requirement and its limits, with no schema (a method has no columns) and no query rules (those belong to the collection that issues the query).
declarations is what the type promises, inherited declarations included and a subclass's own declaration replacing the one it overrides. An entry point that names nothing is absent rather than empty.
It decides nothing. A missing declaration is reported, not repaired and not refused, and the warn-or-raise policy belongs to the caller that knows whether it matters.

Licensing¶

capabilities_of says what a declaration asks for; the licence report says what an identity or a tenant actually has. Two licence lists look alike and answer different questions: me/licenseDetails is what the signed-in user was assigned, subscribedSkus is what the tenant bought — a bought SKU assigned to nobody appears only in the second.

Nothing is read until it is asked for: client.licences is a manager, get_all() / get_user(user) queue the read, and execute_query() on the result sends it.

held = client.licences.get_all().execute_query().value  # get_user(client.me) for the signed-in user
held.holds("M365_E5", "SPE_E5")  # one of these seats is enough
held.names  # SKU part numbers and provisioned plan names
held.plans  # the provisioned plan names alone - capability without the packaging
[sku.sku_part_number for sku in held.carrying("RECORDS_MANAGEMENT")]
[sku.sku_part_number for sku in held.unassigned]  # bought, nobody using it
[sku.sku_part_number for sku in held.suspended]  # bought, cannot be assigned from
held.plan_id("RECORDS_MANAGEMENT")  # the id both sides of an assignment spell the same
held.by_id  # skuId -> SKU, the key an assignment names the SKU by
held.to_rows()  # one row per SKU: prepaidUnits, consumedUnits, spareUnits, capabilityStatus

A SKU part number is packaging, not capability. DEVELOPERPACK_E5 carries RECORDS_MANAGEMENT, so a check written against the retail SKUs alone reports "not licensed" for a seat that can make the call. names returns both spellings, so a requirement names whichever one answers the question it asks.
Only a plan whose provisioningStatus is Success counts. A plan the tenant bought but has not provisioned is not held yet.
The read is deferred and unkept. client.licences sends nothing, and what a read answered is not remembered: a licence is granted while a program runs, so an answer kept for the run is one nobody knows is stale. Read once, ask the report as many questions as you like, and read again after the assignment that changed the tenant.
It reads only what this tenant owns. The public SKU × service-plan catalogue is Microsoft's published data, not something Graph returns: a product nobody has bought cannot be discovered this way, so "does the tenant own it" and "is this seat entitled to it" stay two questions - and the second one is a licensing page, not a library.

An assignment is a child of the user, not a column of it, and Graph will not project it: $expand=assignedLicenses answers "not a navigation property or complex property" and $select=assignedLicenses/skuId answers "Invalid property 'skuId' specified in $select". So the export takes the row unit itself — rows_from writes one row per child of the named property, and the child's own fields become the columns:

client.users.get_licensed().select(["userPrincipalName"]).get_all().to_csv(
    sys.stdout, rows_from="assignedLicenses"
).execute_query()
# userPrincipalName, skuId, disabledPlans

The file carries SKU ids, because the projection is the type's and not a join: a part number lives on subscribedSkus, and SubscribedSku.plan_name(plan_id) is how a reader turns a plan id back into the name a person recognises. For the same reason the file is worth reading back only in the tenant that wrote it. Read back, a row is one assignLicense call, so the verb takes many at once and the transport decides how many fit in a $batch. A row addresses its user (userPrincipalName or id) and carries the two names the API itself takes:

rows = [
    {
        "userPrincipalName": "ada@contoso.com",
        "addLicenses": [AssignedLicense(skuId=sku_id, disabledPlans=["EXCHANGE_S_FOUNDATION"])],
        "removeLicenses": [],
    },
]
client.users.assign_licences(rows)  # deferred — one sub-request per row
client.execute_batch()  # 20 per $batch; a refused row returns its own error

Nothing is read first: /users/{user-id} takes a UPN or an id, so the row addresses the user instead of loading them. Graph owns the preconditions — no usageLocation, a SKU this tenant does not subscribe to, no seat left, a licence held through a group — and a row it refuses comes back as that sub-request's response, which is where a report of "what did not go through" belongs.

Error handling¶

Every failed request raises a ClientRequestException, dispatched to the most specific registered type. Classification is locale-independent: SharePoint encodes the error as "<HRESULT>, <dotnet-type>" in error.code, so the library matches the numeric HRESULT and/or the embedded .NET type name — never the translated message. Graph's symbolic codes (resourceLocked, itemNotFound, …) are matched too.

from office365.sharepoint.exceptions import (
    SharePointException,  # catch-all for any Microsoft.SharePoint.* error
    SPFileCheckOutException,  # HTTP 423 — a releasable check-out
    SPDuplicateValuesFoundException,
    SPListDataValidationException,
    SPQueryThrottledException,
)
from office365.runtime.exceptions import AccessDeniedException, FileLockedException

try:
    ctx.web.lists.get_by_title("Tasks").add_item({"Title": "x"}).execute_query()
except SPListDataValidationException as exc:
    print(exc.message)
except SharePointException as exc:  # any other SharePoint error
    print(exc.code, exc.hresult, exc.error_type, exc.request_id)

DuplicatedObjectException (HTTP 400 / ConflictingObjects), ObjectNotFoundException (404 / ResourceNotFound), FileLockedException (shared coauthoring lock, HTTP 423) and AccessDeniedException (401/403, or Graph accessDenied/Authorization_RequestDenied / SharePoint E_ACCESSDENIED) are runtime-level and shared with Graph.
SharePointException is the catch-all base for product-specific errors; every SP* type subclasses it, so except SharePointException catches them all.
HTTP 423 is disambiguated: SPFileCheckOutException (an explicit check-out, releasable with File.checkin()/File.undocheckout()) is typed separately from the shared FileLockedException (open in Office, not breakable via the API).
exc.hresult / exc.error_type expose the parsed SharePoint code, while exc.request_id, exc.server_guid and exc.duration_ms carry correlation and server diagnostics.

Errors with a recovery path expose it in GUIDANCE, appended to str(exc): SPFileCheckOutException, FileLockedException, and SPQueryThrottledException (page or index a column — see Large lists).

Permission hints¶

A denied request appends the declared permission to the exception, so you see what to grant without opening the docs. Library methods carry their requirement as @require_permission(...) metadata (also folded into help(method)); when a decorated method's request is rejected, the requirement is attached to the query it built and surfaced on AccessDeniedException:

from office365.runtime.exceptions import AccessDeniedException

try:
    client.drives.get().execute_query()
except AccessDeniedException as exc:
    print(exc)
    # ('accessDenied', 'Access denied.')
    # The caller is not permitted to perform this request. ...
    # This API declares: application Files.Read.All, ... (List all drives in the tenant)

The generic part names the likely fix — an app-only role plus admin consent, or a delegated OAuth scope — and points at require_application_permission(...) / require_role(...) to fail fast with the exact requirement. Only permission denials are enriched; every other error is left untouched so the hint cannot mislead. Application code can append its own context with exc.add_hint("...") and read the collected text back via exc.hints. The most recently queued query is available as client.last_query.

SharePoint declares a level rather than a list, because its permission names form a ladder — AllSites.Read, .Write, .Manage, .FullControl, and the app-only Sites.Read.All … Sites.FullControl.All beside them — where each rung already does what the ones below it do:

from office365.sharepoint.permissions import SiteLevel, site_access


@site_access(SiteLevel.WRITE)
def add_item(self, creation_information): ...

A handful of operations reach past one site collection — the migration API, hub site registration, bring-your-own-key — where no rung is enough on its own and the caller must also be a SharePoint administrator. site_access takes that role as the alternative it is, so it is checked but never added to the list of things to grant:

from office365.permissions import DirectoryRole


@site_access(SiteLevel.FULL_CONTROL, role=DirectoryRole.SharePointAdministrator)
def register_hub_site(self): ...

The role is a DirectoryRole member rather than a string because it is compared against the displayName the tenant's directory answers. Misspelled it does not fail — it names a role nobody holds, so the requirement is simply never met, and on a client that cannot read roles at all it comes back unprovable, which passes. Pass a plain string only for a role the enum does not name; it holds the ones this library declares.

A second role qualifies the other way round. The compliance policy store — retention labels, holds, adaptive scopes — is administered in Microsoft Purview even though the request goes to the site's own _api/complianceStore, so SPPolicyStoreProxy names DirectoryRole.ComplianceAdministrator and not the site administrator. The grant still satisfies the call on its own; the role is stated because it is the one an operator is likely to hold when the label work is theirs.

Use @require_permission(..., resource=ResourceName.SharePoint) directly when a call needs a name outside that ladder, such as TermStore.Read.All or Sites.Selected.

Some declarations list several names and mean any one of them. A $ref membership call is the clear case: the same directoryObjects/add adds to whichever collection it was called on, so Group.ReadWrite.All is enough for a group's members and RoleManagement.ReadWrite.Directory for a role's. There is no single name the method could claim, and claiming only the first would tell a correctly-granted tenant to go and buy another grant.

The same any-of shape covers a drive item, for a different reason. Graph publishes the file permissions twice — Files.* and Sites.* — because a drive can be someone's OneDrive or a document library, and the same item is reachable through either set. DriveItem.preview and Drive.recent therefore declare the whole rung (Files.Read, Files.Read.All, Files.ReadWrite, Files.ReadWrite.All, Sites.Read.All, Sites.ReadWrite.All) instead of picking the two names that fit the endpoint: the tenant in front of you decides which half is granted.

The verb does not decide the rung either. Every workbook call declares Files.ReadWrite (Files.ReadWrite.All for an app-only token), reads included, because a workbook session is opened by a write grant and the reads happen inside it — the operation the endpoint checks is the session, not the GET.

Some endpoints exist for one flow only. Graph documents its tenant-wide message exports (chats/getAllMessages, teams/getAllMessages, channels/getAllMessages) for application tokens, and rejects a delegated one whatever scopes it holds — so those declare application= alone. Nothing is invented for the other column: declared(method, flow="delegated") raises and names the method instead of answering a grant that would never satisfy it.

The same holds for a whole subtree: every Calls.* permission is an application role, so the call-control methods on Call, Participant and CallCollection declare no delegated column either. A delegated client can neither be granted the name nor told it lacks one — the calls API answers a bot, not a signed-in user.

The reverse is just as real. filterByCurrentUser on app-consent and user-consent requests filters by who is asking, and ThreatAssessment.ReadWrite.All and UserActivity.ReadWrite.CreatedByApp are published as delegated scopes with no application role behind them — so seven methods under office365.directory declare delegated= alone, and declared(method, flow="application") raises for them the same way it does in the other direction.

A name is worth checking before it reaches a tenant. office365.permissions.catalog() answers what an API publishes — publishes(name, kind), requires_admin_consent(name, kind), and the names themselves — from a committed snapshot of that API's own service principal, so the question needs no sign-in:

from office365.permissions import APPLICATION, catalog

graph = catalog("Microsoft Graph")
graph.publishes("Files.ReadWrite.All", APPLICATION)  # True
graph.publishes("Files.ReadWrite", APPLICATION)  # False - the scope exists, the role does not

The same catalog is what keeps the library's own declarations honest, and a live suite re-reads the tenant, so a name that stops being published fails a test rather than a customer's deployment.

Power features¶

Proxy, custom SSL, timeouts¶

Configure the transport once — every request (including internal ones like the form digest) inherits it:

ctx = (
    ClientContext("https://contoso.sharepoint.com/sites/team")
    .with_client_certificate("tenant", "client_id", "thumbprint")
    .with_transport(
        proxies={"https": "http://proxy:8080"},
        verify="/path/to/ca-bundle.pem",  # or False for testing only
        timeout=30,
    )
)

For MSAL authentication traffic to login.microsoftonline.com, set the HTTPS_PROXY environment variable instead — MSAL reads it directly.

Transports ship bounded defaults, so a dead connection fails instead of hanging a script: connect 10 s and read 300 s (RequestsTransport), plus write 300 s and pool acquisition 10 s (HttpxTransport). The read bound is the gap between bytes, not a cap on the whole transfer — a slow but healthy 2 GB download is unaffected. Override it per transport with a number or a (connect, read) tuple, or per request where one is accepted; pass NO_TIMEOUT to remove the bound entirely:

from office365.runtime.transport.base import NO_TIMEOUT

# one long-running endpoint needs more patience than the default
ctx.with_transport(timeout=(10, 900))

# ...or no limit at all (e.g. a multi-hour streamed export)
ctx.with_transport(timeout=NO_TIMEOUT)

CSV export¶

Collections export straight to CSV (or a pandas DataFrame with the pandas extra):

with open("users.csv", "w", newline="") as f:
    client.users.get_all().select(["displayName", "userPrincipalName"]).to_csv(f).execute_query()

DataFrame import / bulk load¶

Every collection exposes the same adapters over one shared projection: export via to_records()/to_dataframe()/to_csv()/to_ndjson()/to_excel() (or the generic export_to(..., format=...), e.g. format="json"), and import via the streaming from_* family (from_records, from_dataframe, from_csv, from_json, from_ndjson, from_excel, …).

The naming is consistent: from_* returns a streaming ImportResult (bounded, resumable, idempotent); queue_* (queue_records/queue_dataframe) is the deferred queue-all path (run with execute_query()).

List.from_dataframe() provisions the typed columns once (inferred from the dtypes, or from an explicit schema), then queues, executes, and discards each chunk — so memory stays bounded no matter the row count. Pick the execution terminal — the driver itself carries no execution knobs:

import pandas as pd

lst = ctx.web.lists.ensure_list("Housing").execute_query()

lst.from_dataframe(df).execute_query()  # sequential
lst.from_dataframe(pd.read_csv("housing.csv", chunksize=2000)).execute_batch(
    items_per_batch=100, concurrency=5
)  # batched

For full control, iterate the driver and drive execution yourself:

for _ in lst.from_dataframe(pd.read_csv("housing.csv", chunksize=2000)):
    ctx.execute_batch(items_per_batch=100, concurrency=5)

concurrency>1 runs batches in parallel with per-sub-request throttling retries (honoring Retry-After).

For long-running jobs, pass a checkpoint: the committed cursor is persisted atomically after each chunk, so an interrupted run resumes by skipping the already-committed chunks (progress continues from that offset). on_error="collect" records a failing chunk (in ImportStats.errors and the checkpoint's failures) and continues instead of aborting:

lst.from_dataframe(
    pd.read_csv("housing.csv", chunksize=2000), checkpoint="housing.run.json", on_error="collect"
).execute_batch(items_per_batch=100, concurrency=5)

checkpoint accepts a path (FileCheckpointStore), an ImportCheckpoint or None (MemoryCheckpointStore), or any CheckpointStore — pluggable persistence, MSAL-cache style. The driver exposes ImportResult.resumed_from (rows already committed) and ImportResult.checkpoint (live cursor), and ImportStats.resumed_from / stats.summary() report the resumed offset.

The generic entry point is collection.from_records(batches) for any ClientObjectCollection. See examples/sharepoint/lists/from_dataframe.py and from_dataframe_large.py, and the examples/entraid DataFrame export for the Graph side.

Idempotent imports (skip / update)¶

Pass a natural key (one or more columns) to make the import duplicate-proof: a SHA-256 hash of those columns is stored in key_field (created if missing), the existing keys are loaded once, and a re-run either skips already-present rows (on_conflict="skip") or updates them (on_conflict="update"):

lst.from_dataframe(
    pd.read_csv("housing.csv", chunksize=2000),
    key=["region", "date"],  # natural key -> MigrationKey hash
    on_conflict="update",
).execute_batch(items_per_batch=100, concurrency=5)

This works alongside checkpoint (resume) — the checkpoint skips committed chunks for speed, and the key makes the import idempotent on a fresh run, on a resumed run, and when a chunk is replayed. The existing keys are loaded once per run (fresh or resumed; key values + Id only, paged). The checkpoint stores a source signature (format, chunk size, key columns): if it changes between runs (e.g. a different chunksize), the chunk skip is discarded and the source is re-scanned — the key keeps it duplicate-free.

Pass enforce_unique=True to mark the key column unique on the list (guards against a create race), and dry_run=True to preview the create/update/skip plan without writing anything.

The import/export surface lives on RecordCollection (the base of every typed EntityCollection), so the same API works on any collection. List mirrors it as a facade: List.from_dataframe/from_records/from_file (streaming), List.queue_dataframe/queue_records (deferred), List.export_to/to_dataframe (record export), and List.export (a .zip package export — per-item JSON + optional content — distinct from the record export):

collection.export_to(f, format="csv").execute_query()  # unified record export
collection.export_to("out.csv", page_size=2000).execute_query()  # streamed (bounded)
await collection.export_to_async("out.csv", page_size=2000)  # async twin (see docs/async.md)
collection.from_dataframe(df, key=["id"], on_conflict="update")  # unified streaming import
collection.from_records(batches, checkpoint="run.json")  # stream record batches

lst.queue_dataframe(df).execute_query()  # deferred (queue-all)
lst.export_to(f, format="csv").execute_query()  # list -> records
lst.export(zip_file, include_content=True).execute_query()  # list -> .zip package

result = lst.from_dataframe(df, key=["id"])
result.run(concurrency=5)  # == execute_batch
report = result.verify(df, key=["id"])  # counts + missing keys

See the data pipeline guide for the full model, formats, idempotency, typed columns, the file bridge and verification.

The format registry (office365.runtime.converters.registry) maps a format name to its reader/writer, so adding a format is a registration — the named to_csv/from_dataframe conveniences are thin wrappers over it. Built-in formats: csv, tsv, json, ndjson, excel, parquet, orc, feather, dataframe; databases stream through from_sql/from_duckdb (and to_sql/ to_duckdb). Optional dependencies are extras: [pandas], [excel], [parquet], [sql], [duckdb]. Readers and writers accept a path, a PathLike or an open file object.

Idempotency¶

Three idempotent layers, one model — a re-run never duplicates:

upsert (generic verb) — create-or-reconcile a single entity keyed by a natural property. RecordCollection.upsert(key, value, **properties) looks the entity up with $filter and then creates it (missing) or updates only the properties that actually differ (present), so an unchanged re-run queues no write at all. It is the portable client-side path shared by Graph and SharePoint:

client.groups.upsert("uniqueName", "contoso", displayName="Contoso").execute_query()

Endpoints that implement the Microsoft Graph upsert pattern can instead be written in a single round-trip against an alternate-key-addressed entity — a PATCH with Prefer: create-if-missing:

client.groups.get_by_alternate_key("uniqueName", "contoso").set_property(
    "description", "Team site"
).upsert().execute_query()

Entity.upsert accepts if_match= / if_none_match= for optimistic concurrency, and raises UpsertNotSupported when the service does not honour create-if-missing (fall back to RecordCollection.upsert).

Metadata (ensure_*) — get-or-create a resource (field, list, content type, term, folder, user …), deferred; run with execute_query(). Pass on_conflict="update" to reconcile an existing definition:

lst.ensure_field("Status", FieldType.Text).execute_query()  # create if missing
lst.ensure_fields({"Region": FieldType.Text, "Amount": FieldType.Number})  # -> list[Field]
lst.ensure_field("Status", FieldType.Choice, on_conflict="update")  # reconcile

All client-side ensure_* share the office365.runtime.queries.get_or_create primitives (get_or_create/create_or_get), so error classification and the deferred queueing behave the same everywhere. (Some ensure_* — e.g. ensure_site_pages_library, ensure_user — are server-side operations and are already idempotent.)

Data (from_records(key=…, on_conflict=…)) — get-or-create/update records by a natural key (see Idempotent imports above).

The verb defaults differ by intent: add always creates, upsert reconciles (on_conflict="update"), and ensure/ensure_<noun> create only if missing (on_conflict="skip"). They are all built on the same client-side office365.runtime.queries.get_or_create primitive.

Distinct from all of these: ClientObject.ensure_property/ensure_properties is an in-memory lazy load (fetch a property if not already loaded), not a server-side get-or-create.

Learn more¶

Products — start with the SharePoint area, then browse the rest from the left navigation.
Auth examples — all authentication flows.
README — installation, the full auth matrix, and per-service guides.