Oracle JD Edwards and Databricks integration

What moves between them

Data flows one direction: from Oracle JD Edwards into Databricks. GL transaction headers and detail rows, account masters, supplier masters, purchase order headers and lines, and accounts payable ledger records are polled from JD Edwards on a schedule you set, then inserted or upserted into Databricks SQL tables within your workspace. Reference data such as account masters and supplier information is synced first so downstream GL detail rows reference valid chart of accounts and vendors. Because JD Edwards is pull-only and Databricks webhooks do not cover data events, ml-connector polls both systems on a fixed cadence, typically daily or weekly depending on your reporting cycle.

How ml-connector handles it

ml-connector stores the JD Edwards AIS Server URL, username, and password encrypted, then requests a session token on each poll cycle by posting to the tokenrequest endpoint. JD Edwards tokens expire after 30 to 60 minutes (configurable), so ml-connector refreshes the token before each poll if needed and receives a new one if the previous call returns HTTP 444 (invalid token). On the Databricks side, ml-connector uses OAuth 2.0 client credentials to obtain a bearer token at the workspace or account level, then refreshes it every hour since the token expires at 3600 seconds. When querying JD Edwards, ml-connector uses date filters on UPMJ to track which records have changed since the last poll, handles pagination with the maxPageSize parameter and moreRecords flag, and respects any IP allowlist configured on your AIS Server by whitelisting the connector egress IPs. Databricks table writes are issued as SQL MERGE or INSERT statements through the SQL API, so ml-connector does not need to pre-create schemas or catalogs. Because JD Edwards has no explicit rate limits beyond what the AIS Server JVM can handle, ml-connector includes exponential backoff on 429 or 503 responses. Every record carries an audit trail and can be replayed if a downstream table write fails.

A real-world example

A mid-sized manufacturing company runs Oracle JD Edwards EnterpriseOne on-premises to manage production, procurement, and general ledger across three distribution centers. Their finance team has been manually exporting GL account balances and transaction detail to spreadsheets every month to reconcile inventory movements and cost allocations across plants. The data arrives weeks late due to infrastructure constraints. With Oracle JD Edwards and Databricks connected, GL and supplier master records load into Databricks daily, and the finance team now runs variance analysis and drill-down reports directly on current data, eliminating the spreadsheet bottleneck and enabling real-time visibility into plant-level operating costs.

What you can do

• •Load Oracle JD Edwards general ledger transactions and account masters into Databricks SQL tables on a polling schedule you control.
• •Sync purchase order headers, lines, and supplier master records to build procurement analytics and vendor spend reports in Databricks.
• •Handle JD Edwards session token refresh and AIS Server connectivity, including IP allowlisting and on-premises infrastructure constraints.
• •Map Oracle JD Edwards table names and field names to Databricks schema and column names without schema migration.
• •Retry failed loads with exponential backoff and track every record in an audit log for replay and troubleshooting.

Questions

Why must ml-connector poll Oracle JD Edwards instead of receiving webhooks?

Oracle JD Edwards EnterpriseOne has no native outbound webhooks for data changes. ml-connector polls the data service tables on a schedule using date filters on the UPMJ (date updated) field to track changes since the last poll. You can also configure JD Edwards orchestration to call ml-connector's ingest endpoint on a cron schedule if preferred.

How does ml-connector handle the on-premises AIS Server URL and session token expiry?

ml-connector stores the full AIS Server URL, username, and password encrypted and requests a session token by posting to the tokenrequest endpoint before each poll. The token expires after 30 to 60 minutes (configurable), so ml-connector refreshes it when a call returns HTTP 444 or before the timeout, ensuring uninterrupted polling without manual token management.

How are the data loads actually written to Databricks tables?

ml-connector uses SQL MERGE or INSERT statements through the Databricks SQL API to write or upsert data into tables within your workspace. You define which Oracle JD Edwards tables map to which Databricks schemas and tables, and ml-connector handles the field mapping and refresh cadence automatically.