Skip to content

Commit 40d65b0

Browse files
committed
update retl bigquery setup with storage-api
1 parent dd6eebc commit 40d65b0

File tree

1 file changed

+18
-11
lines changed

1 file changed

+18
-11
lines changed

src/connections/reverse-etl/reverse-etl-source-setup-guides/bigquery-setup.md

Lines changed: 18 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -70,17 +70,24 @@ You can find the location of your BigQuery resources using the following method:
7070
3. The Location of the dataset (like US or EU) is displayed in the Dataset Info.
7171
7272
## Set up BigQuery as your Reverse ETL source
73-
1. In the BigQuery console, search for the service account you created.
74-
2. When your service account pulls up, click the 3 dots under **Actions** and select **Manage keys**.
75-
3. Click **Add Key > Create new key**.
76-
4. In the pop-up window, select **JSON** for the key type and click **Create**. The file will be downloaded.
77-
5. Copy all the content in the JSON file you created in the previous step.
78-
6. Open the Segment app and navigate to **Connections > Sources**.
79-
7. On the My sources page, click **+ Add source**.
80-
8. Search for "BigQuery" and select the BigQuery source from the sources catalog. On the BigQuery overview page, click **Add Source**.
81-
9. On the Set up BigQuery page, enter a name for your source and paste all the credentials you copied from previous step into the **Enter your credentials** section.
82-
10. Enter the location of your BigQuery warehouse in the **Data Location** field.
83-
11. Click **Test Connection** to test to see if the connection works. If the connection fails, make sure you have the right permissions and credentials and try again.
73+
1. In the BigQuery console, search for the service account you created.
74+
2. When your service account pulls up, click the 3 dots under **Actions** and select **Manage keys**.
75+
3. Click **Add Key > Create new key**.
76+
4. In the pop-up window, select **JSON** for the key type and click **Create**. The file will be downloaded.
77+
5. Copy all the content in the JSON file you created in the previous step.
78+
6. Open the Segment app and navigate to **Connections > Sources**.
79+
7. On the My sources page, click **+ Add source**.
80+
8. Search for "BigQuery" and select the BigQuery source from the sources catalog. On the BigQuery overview page, click **Add Source**.
81+
9. On the Set up BigQuery page, enter a name for your source and paste all the credentials you copied from previous step into the **Enter your credentials** section.
82+
10. Enter the location of your BigQuery warehouse in the **Data Location** field.
83+
84+
When setting up a BigQuery Reverse ETL source, you can choose which API Segment uses to read from BigQuery. You can make this selection during the initial setup, or later from Sources > Reverse ETL > BigQuery Source > Settings > Connection Settings.
85+
- **REST API**
86+
Recommended for most tables. REST is generally more cost-efficient, but syncs may be slower for very large datasets.
87+
- **Storage API**
88+
Recommended for large tables. The Storage API can significantly improve throughput and reduce sync times, but may incur higher costs. See [BigQuery Storage API client libraries](https://cloud.google.com/bigquery/docs/reference/storage/libraries) and [BigQuery pricing for data extraction](https://cloud.google.com/bigquery/pricing?hl=en#data_extraction_pricing) for more information
89+
90+
11. Click **Test Connection** to test to see if the connection works. If the connection fails, make sure you have the right permissions and credentials and try again.
8491
12. If the test connection completes successfully, click **Add source** to complete the setup process.
8592
8693
After you've added BigQuery as a source, you can [add a model](/docs/connections/reverse-etl/setup/#step-2-add-a-model) and follow the rest of the steps in the Reverse ETL setup guide.

0 commit comments

Comments
 (0)