Plex and Google BigQuery integration

What moves between them

Records flow from Plex into BigQuery on a schedule tied to your reporting cycle. Invoices, purchase orders, GL accounts, parts, and inventory are polled from Plex every 15 minutes by default, transformed into rows, and inserted into BigQuery tables matching your schema. Each table is keyed by the Plex entity primary key and a load timestamp, so you can track when each record arrived. Plex is a source only; BigQuery receives the data but does not write back to Plex.

How ml-connector handles it

ml-connector stores your Plex OAuth credentials and BigQuery Service Account key encrypted. On each poll, it refreshes the Plex bearer token, queries Plex for records modified since the last run, transforms each row into your BigQuery table schema, and streams inserts with an insertId for deduplication. Because Plex enforces role-based permissions, the integration user must have the correct Plex role assigned or API calls return 403 or empty results; ml-connector logs these permission errors clearly. Plex rate limits are not publicly documented, so ml-connector backs off with exponential jitter on HTTP 429 and retries. Google BigQuery tokens expire after one hour, so ml-connector refreshes the Service Account JWT before every batch. The poll interval is configurable (default 15 minutes for most tables, longer for bulk historical GL accounts to avoid BigQuery quota stress), and every record carries the exact source timestamp and a full audit trail so you can replay failed loads if needed.

A real-world example

A mid-sized discrete manufacturer runs Plex ERP for production, inventory, and accounts payable. The finance team needs visibility into invoice aging, PO spend by supplier, and inventory valuation for month-end close. Before the integration, they exported data from Plex manually each week, loaded it into a spreadsheet, and built pivot tables. With Plex and BigQuery connected, invoices and POs flow automatically into BigQuery tables, and analysts query the tables directly with SQL to build real-time dashboards and close reports. The weekly export and manual load step is gone, and the data is fresh within 15 minutes.

What you can do

• •Stream Plex invoices, purchase orders, GL accounts, parts, and inventory into BigQuery tables on a configurable schedule.
• •Detect changes in Plex using modified_date and created_date filtering so only new or updated records are transferred.
• •Authenticate Plex with OAuth 2.0 client credentials and Google BigQuery with a Service Account JWT.
• •Use best-effort deduplication with BigQuery insertId to prevent duplicate rows if a poll runs twice.
• •Maintain a full audit trail of every load, including source timestamp, arrival time, and record count, so failed loads can be replayed.

Questions

Which direction does data move between Plex and Google BigQuery?

Data flows from Plex into BigQuery only. Invoices, purchase orders, GL accounts, parts, and inventory are polled from Plex and inserted into BigQuery tables. BigQuery does not write back to Plex, so ml-connector treats BigQuery as a data sink for analytics and reporting.

How does the integration handle Plex's role-based permissions and lack of webhooks?

ml-connector respects Plex role-based access control and fails clearly if the integration user lacks the required role. Because Plex has no native webhooks, ml-connector polls the REST API on a configurable interval (default 15 minutes) and filters by modified_date to catch only records that have changed since the last run, reducing load on both systems.

What happens if a BigQuery insert fails or a Plex poll times out?

ml-connector uses BigQuery's insertId for best-effort deduplication, so a retry of the same batch will not create duplicate rows. Every record includes a source timestamp and load timestamp, so you can query the audit trail to see which records arrived and when, and failed loads can be replayed without re-polling Plex if needed.