MYOB and Google BigQuery integration

What moves between them

The main flow is MYOB into BigQuery. ml-connector polls MYOB on a schedule you set, querying contacts (suppliers and customers), invoices (sales and purchases), general ledger accounts and journals, purchase orders, and supplier or customer payments since the last sync. Each batch is inserted into BigQuery tables in your dataset, partitioned or clustered by date for fast analytical queries. The sync respects MYOB's 1 second minimum polling interval and backs off on rate limit responses.

How ml-connector handles it

ml-connector stores both MYOB OAuth credentials and Google service account private key encrypted in its database. On each poll it refreshes the MYOB bearer token and includes the company file username and password in the required header to authenticate. It queries MYOB using $filter with the LastModified timestamp from the previous sync to find new and changed records, paginating through results 1,000 at a time, and respects the 8 requests/second rate limit by spacing requests. Each record is validated against the BigQuery table schema before insertion, and duplicate records are deduped by the streaming insert insertId if the job fails and replays. If a BigQuery job fails, ml-connector retries with exponential backoff. Every insert carries a sync job ID and timestamp so you can trace which MYOB export became which BigQuery records.

A real-world example

A mid-sized Australian accounting firm runs MYOB for client bookkeeping and wants to offer its principals a unified dashboard showing cash flow and aging across all clients. Rather than exporting MYOB reports as CSV and building a data warehouse by hand, the firm connects MYOB to BigQuery and loads all invoices, payments, and GL entries every hour. The team then uses BigQuery SQL and Looker to build a single view of client aging and monthly cash receipts across the entire book of business, updated without manual intervention.

What you can do

• •Poll MYOB invoices (sales and purchases), supplier payments, and customer receipts, then load them into BigQuery tables.
• •Query MYOB general ledger accounts and journal entries and materialize them in BigQuery for cost and revenue analysis.
• •Authenticate MYOB using OAuth2 and company file credentials, with automatic token refresh every 20 minutes.
• •Handle MYOB rate limits (8 requests/second, 1,000,000 per day) and page sizes (1,000 records max) transparently.
• •Detect new and changed records in MYOB using LastModified timestamp polling and sync them to BigQuery on your schedule.

Questions

What MYOB entities can be synced to BigQuery?

ml-connector syncs contacts (suppliers, customers, employees), invoices and purchase orders, supplier and customer payments, general ledger accounts, and journal entries. You define the BigQuery table schema to match the fields you need from MYOB, and ml-connector inserts each row with a job ID for deduplication and an audit timestamp.

How does ml-connector handle MYOB's dual authentication and rate limits?

MYOB requires OAuth2 bearer tokens plus company file credentials in a header; ml-connector refreshes the bearer token every 20 minutes and includes the company file username and password on every request. For rate limits, ml-connector spaces requests to stay below 8 per second and respects HTTP 429 responses by backing off exponentially before retrying.

Is data pushed from MYOB to BigQuery, or does BigQuery pull it?

MYOB has no webhook system, so ml-connector runs on a schedule you set and polls MYOB for records changed since the last sync using the LastModified timestamp and OData $filter. BigQuery then receives and stores the records; there is no outbound push from BigQuery.