Sage X3 and Databricks integration

What moves between them

Data flows from Sage X3 into Databricks. Supplier invoices, purchase orders, GL accounts, and GL entries are polled from X3 on a configurable schedule, typically daily or weekly for analytics loads. Each record is transformed into the warehouse schema expected by Databricks and written into bronze or silver tables. Reference data such as supplier and GL account masters are synced first so fact tables can join on valid dimensions. GL entries are posted to a GL fact table keyed by account and transaction date. No data flows back from Databricks to Sage X3.

How ml-connector handles it

ml-connector stores X3 OAuth2 credentials (client_id, client_secret, and the customer-specific server URL and folder name) and Databricks Service Principal credentials (client_id, client_secret, and workspace URL) encrypted in the vault. On each sync, it refreshes the X3 token (which expires every 5 minutes) and the Databricks token (3600-second expiry). It queries X3 by polling the updatedDate field to detect changes since the last sync, since X3 has no webhooks. For each changed supplier invoice, purchase order, or GL entry, ml-connector transforms it into the Databricks table schema, mapping X3 GL account codes to Databricks account dimensions, and writes the result via Databricks SQL. If Databricks returns a rate-limit error (HTTP 429), ml-connector backs off and retries. The X3 OAuth2 refresh token is valid for 30 days, so token renewal must be handled before expiry. Every record carries an audit timestamp and sync metadata so late-arriving records can be replayed if a prior write failed.

A real-world example

A mid-sized manufacturer runs Sage X3 on-premise for finance, procurement, and inventory. The finance and supply-chain teams use Databricks for spend analysis and procurement performance dashboards. Before the integration, the finance team exported X3 GL and invoice registers every week as CSV, staged them in Databricks manually, and discovered data quality issues during analysis. With Sage X3 and Databricks connected, each week's GL entries and invoices are loaded automatically, transformation is centralized, and the analytics team starts from clean data. Month-end close includes spend analytics and supplier variance reports that were previously unavailable.

What you can do

• •Load Sage X3 supplier invoices, purchase orders, and GL entries into Databricks tables on a schedule you control.
• •Map X3 GL account codes and cost centers to Databricks account and cost-center dimensions for accurate financial reporting.
• •Handle OAuth2 token refresh on both sides, with X3 tokens expiring every 5 minutes and Databricks tokens requiring 3600-second refresh.
• •Poll Sage X3 via the updatedDate field to detect changes since the last load, with retry logic for transient failures.
• •Maintain a full audit trail of every record, including sync timestamp and source transaction ID, for reconciliation and replay.

Questions

How does the integration detect changes in Sage X3 without webhooks?

Sage X3 supports the updatedDate and modifiedDateTime fields on all financial and procurement records. ml-connector queries X3 with a filter on these fields to fetch only records changed since the last poll, bypassing the need for webhooks. The poll schedule is configurable, typically daily for analytics workloads.

What happens when the X3 or Databricks OAuth2 token expires?

X3 tokens expire every 5 minutes; Databricks tokens expire after 3600 seconds. ml-connector refreshes both tokens before each sync using the 30-day-valid refresh token for X3 and the Service Principal credentials for Databricks. If a token refresh fails, the sync is retried and an alert is raised.

Can data flow back from Databricks to Sage X3?

No. The integration is unidirectional, from Sage X3 into Databricks. Databricks is a read-mostly analytics platform with no native ERP objects like invoices or GL accounts, so ml-connector does not write data back to X3. If you need to post GL entries created in Databricks back into X3, that requires a separate reverse flow.