Oracle E-Business Suite and Google BigQuery integration

What moves between them

The flow runs from Oracle E-Business Suite into Google BigQuery. On a schedule, ml-connector reads EBS open interface views for AP invoices and lines, purchase order headers and lines, GL journal entries, suppliers, customers, payments, and items, filtered by their last-update timestamp, then writes each record set into the matching customer-defined BigQuery table. BigQuery is a warehouse and the accounting source of truth stays in EBS, so the connector treats BigQuery as write-in and read-back only and never posts financial records back into EBS. Cadence is set per flow, commonly hourly or nightly, and aligned with the customer's off-peak window so bulk reads do not slow the shared EBS server.

How ml-connector handles it

ml-connector stores both credential sets encrypted. On the EBS side it accepts the full customer hostname, port, and service aliases, obtains a session token at startup, sends the operating unit org ID and responsibility context on every call, and re-authenticates when EBS returns 401 because token sessions expire in thirty to sixty minutes. On the BigQuery side it signs a JWT with the service-account private key, exchanges it for a one-hour Bearer token, and refreshes before expiry. EBS reads page through limit and offset until a short page signals the end, and a stored last-update cursor advances so each run only pulls changed rows. Writes use streaming inserts with a stable insertId per row, or load jobs with a caller-supplied jobId, so a retried batch deduplicates rather than doubling the data. Field names are mapped per entity, since EBS column names such as INVOICE_NUM and VENDOR_ID are loaded into the column names of your BigQuery schema. EBS has no rate limiter, so reads run at low concurrency to protect the shared instance, while BigQuery requests stay inside its per-project quotas. Adding columns to a BigQuery table is safe, but removing or renaming one is destructive, and the service-account private key keeps its literal newline characters or JWT signing fails. Every record carries a full audit trail and can be replayed if a load fails.

A real-world example

A mid-sized manufacturer with around 700 employees runs Oracle E-Business Suite R12.2 for payables, purchasing, and the general ledger across several operating units. Before the integration, analysts pulled CSV extracts from EBS by hand and uploaded them into a reporting tool each week, so spend dashboards were always a few days stale and the export step broke whenever a column changed. With Oracle E-Business Suite and Google BigQuery connected, AP invoices, purchase orders, and GL journals load into BigQuery on a nightly schedule, partitioned by date, and the finance team queries fresh spend and accrual data directly with SQL. The manual extract step is gone and dashboards refresh against current numbers.

What you can do

• •Load Oracle E-Business Suite AP invoices, purchase orders, GL journals, suppliers, and payments into Google BigQuery tables on a schedule.
• •Read EBS open interface views incrementally by last-update timestamp and page through results with limit and offset.
• •Write rows with streaming inserts or load jobs using a stable insertId or jobId so retried batches deduplicate instead of doubling.
• •Bridge EBS Basic or session-token login on a customer hostname to the BigQuery service-account JWT Bearer flow, refreshing each token before it expires.
• •Run reads at low concurrency to protect the shared EBS server, with a full audit trail and replay on every record.

Questions

Which direction does data move between Oracle E-Business Suite and Google BigQuery?

Data moves from Oracle E-Business Suite into Google BigQuery. AP invoices, purchase orders, GL journals, suppliers, payments, and items are read from EBS and written into BigQuery tables. BigQuery is a warehouse and the accounting source of truth stays in EBS, so ml-connector treats BigQuery as write-in and read-back only and does not post records back into EBS.

Does either system send webhooks, or is this polling?

Neither system pushes events. Oracle E-Business Suite has no modern webhook for external connectors, and Google BigQuery sends no outbound notifications. ml-connector reads EBS open interface views filtered by their last-update timestamp on a schedule you set, storing the last-seen timestamp as a cursor so each run only pulls changed rows.

How are duplicate records avoided when a sync retries?

On the write side BigQuery supports a caller-supplied insertId for streaming inserts and a jobId for load jobs, both of which deduplicate a retried batch. On the read side ml-connector sets a unique source value on EBS records and advances a stored last-update cursor, so a re-run does not reprocess rows it already loaded. Together these keep retries from doubling data in the warehouse.