Skip to content

Commit 5fd8fb9

Browse files
stayseesongstayseesong
authored
Merge pull request #7173 from segmentio/DOC-1001
BigQuery Data Graph Public Beta
2 parents feb9d3d + 0787d87 commit 5fd8fb9

File tree

4 files changed

+76
-43
lines changed

4 files changed

+76
-43
lines changed

‎src/_data/sidenav/main.yml‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -358,10 +358,12 @@ sections:
358358
- section_title: Setup Guides
359359
slug: /unify/data-graph/setup-guides/
360360
section:
361-
- path: /unify/data-graph/setup-guides/snowflake-setup/
362-
title: Snowflake Setup
361+
- path: /unify/data-graph/setup-guides/bigquery-setup/
362+
title: BigQuery Data Graph Setup
363363
- path: /unify/data-graph/setup-guides/databricks-setup/
364-
title: Databricks Setup
364+
title: Databricks Data Graph Setup
365+
- path: /unify/data-graph/setup-guides/snowflake-setup/
366+
title: Snowflake Data Graph Setup
365367
- section_title: Linked Events
366368
slug: /unify/data-graph/linked-events/
367369
section:

‎src/unify/data-graph/setup-guides/BigQuery-setup.md‎

Lines changed: 69 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,85 @@
11
---
2-
title: BigQuery Setup
2+
title: BigQuery Data Graph Setup
33
beta: true
44
plan: unify
5-
hidden: true
65
redirect_from:
76
- '/unify/linked-profiles/setup-guides/BigQuery-setup'
87
---
98

109
> info ""
11-
> At this time, you can only use BigQuery with Linked Events.
10+
> BigQuery for Data Graph is in beta and Segment is actively working on this feature. Some functionality may change before it becomes generally available. This feature is governed by Segment’s [First Access and Beta Preview Terms](https://www.twilio.com/en-us/legal/tos){:target="_blank"}.
1211
13-
On this page, you'll learn how to connect your BigQuery data warehouse to Segment.
12+
Set up your BigQuery data warehouse to Segment for the [Data Graph](/docs/unify/data-graph/data-graph/).
1413

1514

16-
## Set up BigQuery
17-
15+
## Step 1: Roles and permissions
1816
> warning ""
19-
> You need to be an account admin to set up the Segment BigQuery connector as well as write permissions for the `__segment_reverse_etl` dataset.
20-
21-
To set up the Segment BigQuery connector:
17+
> You need to be an account admin to set up the Segment BigQuery connector as well as write permissions for the `__segment_reverse_etl` dataset.
2218
23-
1. Navigate to **IAM & Admin > Service Accounts** in BigQuery.
19+
To set the roles and permissions:
20+
1. Navigate to **IAM & Admin > Service Accounts** in BigQuery.
2421
2. Click **+ Create Service Account** to create a new service account.
25-
3. Enter your **Service account name** and a description of what the account will do.
22+
3. Enter your Service account name and a description of what the account will do.
2623
4. Click **Create and Continue**.
27-
5. In the **Grant this service account access to project** section, select the [*BigQuery User*](https://cloud.google.com/bigquery/docs/access-control#bigquery.user){:target="_blank"} role to add.
28-
6. Click **+ Add another role** and add the *BigQuery Job User* role.
29-
7. Click **+ Add another role** and add the [*BigQuery Metadata Viewer*](https://cloud.google.com/bigquery/docs/access-control#bigquery.metadataViewer){:target="_blank"} role.
30-
8. Click **Continue**, then click **Done**.
31-
9. Search for the service account you've just created.
32-
11. From your service account, click the three dots under **Actions** and select **Manage keys**.
33-
12. Click **Add Key > Create new key**.
34-
13. In the pop-up window, select **JSON** for the key type, and click **Create**.
35-
14. Copy all the content within the file you've created and downloaded.
36-
15. Navigate to Segment and paste all the credentials you've just copied into the **Enter your credentials** section as you connect your warehouse destination.
37-
38-
## Grant access to datasets and tables for enrichment
39-
40-
Grant access to datasets and tables so that Segment can list datasets, tables, and columns, and create Linked Events.
41-
42-
Grant
43-
- [`BigQuery Data Viewer`](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataViewer){:target="_blank"} role <br>
44-
OR
45-
- Permissions:
46-
- `bigquery.datasets.get`
47-
- `bigquery.tables.list`
48-
- `bigquery.tables.get`
49-
- `bigquery.tables.getData`
50-
51-
These can be scoped to projects or [datasets](https://cloud.google.com/bigquery/docs/control-access-to-resources-iam#grant_access_to_a_dataset){:target="_blank"}.
24+
5. Click **+ Add another role** and add the *[BigQuery User](https://cloud.google.com/bigquery/docs/access-control#bigquery.user){:target="_blank"}* role.
25+
6. Click **Continue**, then click **Done**.
26+
7. Search for the service account you just created.
27+
8. From your service account, click the three dots under **Actions** and select **Manage keys**.
28+
9. Navigate to **Add Key > Create new key**.
29+
10. In the pop-up window, select **JSON** for the key type, and click **Create**. The file will download.
30+
11. Copy all the content in the JSON file you created in the previous step, and save it for Step 5.
5231

53-
> info ""
54-
> To create Linked Events on your listed tables, Segment needs `bigquery.tables.get` and `bigquery.tables.getData` at dataset level. However, you can still scope `bigquery.tables.get` and `bigquery.tables.getData` to specific tables. See BigQuery's [docs](https://cloud.google.com/bigquery/docs/control-access-to-resources-iam#grant_access_to_a_table_or_view){:target="_blank"} for more info.
32+
33+
## Step 2: Grant read-only access for the Data Graph
34+
Grant the [BigQuery Data Viewer](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataViewer){:target="_blank"} role to the service account at the project level. Make sure to grant read-only access to the Profiles Sync project in case you have a separate project.
35+
36+
To grant read-only access for the Data Graph:
37+
1. Navigate to **IAM & Admin > IAM** in BigQuery.
38+
2. Search for the service account you just created.
39+
3. From your service account, click the **Edit principals pencil**.
40+
4. Click **ADD ANOTHER ROLE**.
41+
5. Select the **BigQuery Data Viewer role**.
42+
6. Click **Save**.
43+
44+
## *(Optional)* Step 3: Restrict read-only access
45+
If you want to restrict access to specific datasets, grant the BigQuery Data Viewer role on datasets to the service account. Make sure to grant read-only access to the Profiles Sync dataset.
46+
47+
To restrict read-only access:
48+
1. In the Explorer pane in BigQuery, expand your project and select a dataset.
49+
2. Navigate to **Sharing > Permissions**.
50+
3. Click **Add Principal**.
51+
4. Enter your service account in the New principals section.
52+
5. Select the **BigQuery Data Viewer** role in the **Select a role** section.
53+
6. Click **Save**.
54+
55+
You can also run the following command:
56+
57+
```
58+
GRANT `roles/bigquery.dataViewer` ON SCHEMA `YOUR_DATASET_NAME` TO "serviceAccount:<YOUR SERVICE ACCOUNT EMAIL>";
59+
```
60+
61+
## Step 4: Validate permissions
62+
1. Navigate to **IAM & Admin > Service Accounts** in BigQuery.
63+
2. Search for the service account you’ve just created.
64+
3. From your service account, click the three dots under **Actions** and select **Manage permissions**.
65+
4. Click **View Access** and click **Continue**.
66+
5. Select a box with List resources within resource(s) matching your query.
67+
6. Click **Analyze**, then click **Run query**.
68+
69+
## Step 5: Connect your warehouse to Segment
70+
1. Navigate to **Unify > Data Graph** in Segment. This should be a Unify space with Profiles Sync already set up.
71+
2. Click **Connect warehouse**.
72+
3. Select *BigQuery* as your warehouse type.
73+
4. Enter your warehouse credentials. Segment requires the following settings to connect to your BigQuery warehouse:
74+
* **Service Account Credentials:** JSON credentials for a GCP Service Account that has BigQuery read/write access. This is the credential created in Step 1.
75+
* **Data Location:** This specifies the primary data location. This can be either region or multi-region.
76+
5. Test your connection, then click **Save**.
77+
78+
## Update user access for Segment Reverse ETL dataset
79+
If you ran Segment Reverse ETL in the project you are configuring as the Segment connection project, a Segment-managed dataset is already created and you need to provide the new Segment user access to the existing dataset.
80+
81+
If you run into an error on the Segment app indicating that the user doesn’t have sufficient privileges on an existing `__segment_reverse_etl` dataset, grant the [BigQuery Data Editor](https://cloud.google.com/bigquery/docs/access-control#bigquery.dataEditor){:target="_blank"} role on the `__segment_reverse_etl` dataset to the service account . Note that the `__segment_reverse_etl` dataset is hidden in the console. Run the following SQL command:
82+
83+
```
84+
GRANT `roles/bigquery.dataEditor` ON SCHEMA `__segment_reverse_etl` TO "serviceAccount:<YOUR SERVICE ACCOUNT EMAIL>";
85+
```

‎src/unify/data-graph/setup-guides/redshift-setup.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Redshift Setup
2+
title: Redshift Data Graph Setup
33
beta: true
44
plan: unify
55
hidden: true

‎src/unify/data-graph/setup-guides/snowflake-setup.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Snowflake Setup
2+
title: Snowflake Data Graph Setup
33
plan: unify
44
redirect_from:
55
- '/unify/linked-profiles/setup-guides/snowflake-setup'

0 commit comments

Comments
 (0)