.svg){kind=logo}
.svg)
About
Integrating Salesforce data with BigQuery for better insights - with our Google Cloud skills we are able to integrate practically any data source with BigQuery. It's not a question of whether it's possible or not. It's just a question of how.
01.
Launch
2023
02.
Scope
integration
03.
SOLUTION
04.
RESULTS
About the client
To protect the privacy of our clients we can't always share the details of projects and services provided. We would sometimes share anonymous stories of our projects to show you what's possible in the data and AI world today.
Services
Our client needed to integrate the Salesforce data directly with a BigQuery warehouse to be able to leverage the power of advanced analytics. They provided Laurens Coster with a Bulk API and LC needed to find a way to integrate clients data (with no GDPR protected data, just analytics) into BigQuery.
We integrated an external data source - Salesforce this time - into Google BigQuery. The integration was made possible through a number of custom made solutions and scripts that combine a number of Google Cloud products.
Our solution
We wrote a custom Python code to communicate with Bulk API, that was used in an Apache Beam script later run on Dataflow. Once a user was verified by Bulk API and data was ready for transfer, with a custom made script they were saved in a Google Cloud bucket. Once the data was processed by Dataflow, it was imported into BigQuery’s External Table and merged through daily incremental procedure.
Results
Thanks to the integration of an additional data source into the BigQuery warehouse, our client was able to enhance their analytics and have a more complex overview of their data, and be able to base the analytics and data-driven decisions on a comprehensive landscape.
