Deltek and Databricks integration

What moves between them

The flow runs from Deltek into Databricks. ml-connector reads Deltek Vantagepoint AP invoices, AR invoices, journal entries, GL accounts, projects, firms, and employees and writes each record set into Delta Lake tables in the Databricks catalog and schema you choose. New and changed records are picked up on a schedule tied to your close calendar, since Deltek cloud is polled rather than pushed. Databricks holds no finance source records of its own, so it is treated as a read-only analytics destination and ml-connector never writes invoices or journals back into Deltek.

How ml-connector handles it

ml-connector stores both credential sets encrypted and refreshes each one-hour token before it expires rather than waiting for a 401. On the Deltek side it accepts the full tenant URL per customer, since Vantagepoint publishes no shared hostname, and it pages through the REST API using page and pagesize parameters because there is no cursor. Because Vantagepoint webhooks are workflow-driven and unsigned, it polls Deltek on a schedule rather than relying on a push. On the Databricks side it writes through the SQL Statement Execution API against a running SQL warehouse, or submits a Job, and it sends a unique Jobs idempotency token so a network retry returns the existing run instead of double-loading rows. Deltek invoices, journals, and projects are mapped to the target catalog, schema, and table on the Databricks side, and the service principal must be added to that workspace before any write. Databricks rate limits return HTTP 429, so it backs off and retries with jitter. Every record carries a full audit trail and can be replayed if a load fails.

A real-world example

A 600-person architecture and engineering firm runs Deltek Vantagepoint for project accounting across dozens of active contracts. Before the integration, the finance team exported AP invoices, journal entries, and project budgets to spreadsheets every week so the analytics group could build project profitability and utilization dashboards, and the numbers were always a few days stale and prone to copy errors. With Deltek and Databricks connected, those records land in Delta Lake tables on the firm's close schedule, the dashboards read live governed data, and the weekly export step is gone.

What you can do

• •Load Deltek Vantagepoint AP and AR invoices, journal entries, and GL accounts into Delta Lake tables for analytics.
• •Sync Deltek projects, firms, and employees into Databricks so finance data joins the rest of your warehouse.
• •Bridge the Deltek OAuth password grant and the Databricks OAuth client-credentials service principal.
• •Write rows through the SQL Statement Execution API or a Job with a Jobs idempotency token so retries never double-load.
• •Poll Deltek on your close schedule, back off on Databricks 429 limits, and keep a full audit trail on every record.

Questions

Which direction does data move between Deltek and Databricks?

Data moves from Deltek into Databricks. Deltek Vantagepoint is the finance source for AP invoices, AR invoices, journal entries, GL accounts, projects, firms, and employees, and ml-connector writes those records into Delta Lake tables. Databricks has no native finance objects, so it is treated as a read-only analytics destination and nothing financial is written back.

How are records actually written into Databricks tables?

The Databricks Tables API manages metadata only, so rows are not written by direct field updates. ml-connector writes table data through the SQL Statement Execution API against a running SQL warehouse, or by submitting a Databricks Job. It passes a unique Jobs idempotency token so a network retry returns the existing run rather than loading the data twice.

Does the integration use webhooks or polling?

It uses polling. Deltek Vantagepoint webhooks are workflow-driven callbacks with no HMAC signature, and Databricks only offers webhooks for the MLflow model registry, not for tables or jobs. ml-connector reads Deltek on a schedule tied to your close calendar, paging through the REST API, and refreshes the one-hour OAuth tokens on both sides before they expire.