You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+23-21Lines changed: 23 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,26 +1,26 @@
1
-
# Cloud SQL IAM Database Authentication for Groups
1
+
# GroupSync: Cloud SQL IAM Database Authentication for Groups
2
2
**Note:** This project is experimental and is not an officially supported Google product.
3
3
4
-
This project is a self-deployed service that provides support for managing [Cloud SQL IAM Database Authentication](https://cloud.google.com/sql/docs/mysql/authentication) for groups. This service leverages [Cloud Run](https://cloud.google.com/run), [Cloud Scheduler](https://cloud.google.com/scheduler), and the [Cloud SQL Python Connector](https://github.com/googlecloudplatform/cloud-sql-python-connector) to consistently update and sync Cloud SQL instances based on IAM groups. It will create missing database IAM users, GRANT roles to database IAM users based on their IAM groups, and REVOKE roles from database IAM users no longer in IAM groups.
4
+
GroupSync is a self-deployed service that provides support for managing [Cloud SQL IAM Database Authentication](https://cloud.google.com/sql/docs/mysql/authentication) for groups. GroupSync leverages [Cloud Run](https://cloud.google.com/run), [Cloud Scheduler](https://cloud.google.com/scheduler), and the [Cloud SQL Python Connector](https://github.com/googlecloudplatform/cloud-sql-python-connector) to consistently update and sync Cloud SQL instances based on IAM groups. It will create missing database IAM users, GRANT roles to database IAM users based on their IAM groups, and REVOKE roles from database IAM users no longer in IAM groups.
5
5
6
6
## Supported Databases
7
7
Currently only the following databases are supported:
8
8
-**MySQL 8.0**
9
9
-**PostgreSQL 13, 12, 11, 10, 9.6**
10
10
11
11
## Overview
12
-
The Cloud SQL IAM Database Authentication for Groups service at an overview is made of Cloud Scheduler Job(s) and Cloud Run instance(s).
12
+
The GroupSync service at an overview is made of Cloud Scheduler Job(s) and Cloud Run instance(s).
13
13
14
-
The Cloud Scheduler Job(s) are configured to run on the interval of your choosing (every 10 mins, 1 hour, daily etc.) When ran, the Cloud Scheduler calls the IAM Database Authentication for Groups Cloud Run service, passing in the configured request body from the scheduler, which contains parameters that tell the service which IAM groups and which Cloud SQL instances to sync and manage.
14
+
The Cloud Scheduler Job(s) are configured to run on the interval of your choosing (every 10 mins, 1 hour, daily etc.) When ran, the Cloud Scheduler calls a GroupSync Cloud Run service, passing in the configured request body from the scheduler, which contains parameters that tell the service which IAM groups and which Cloud SQL instances to sync and manage.
The Cloud Run service calls the required Google APIs to get a snapshot of the current IAM group(s) members and the current Cloud SQL instance(s) database users, it then adds any new IAM members who have been added to the IAM group since the last sync as an IAM database user on the corresponding Cloud SQL instances. The Cloud Run service then also verifies or creates a database role within each configured database for each configured IAM group. Mapping each IAM group to a database role, the service can then GRANT/REVOKE this group role with the appropriate database permissions for the IAM group to all the proper IAM database users who are missing it or should not have it based on the members of the IAM group.
There are a few initial setups steps to get the service ready and grant it the permissions needed in order to successfully operate. However, after this setup is complete, minimal configuration is needed in the future.
23
+
There are a few initial setups steps to get the GroupSync service ready and grant it the permissions needed in order to successfully operate. However, after this setup is complete, minimal configuration is needed in the future.
24
24
25
25
### Installation
26
26
To run this service successfully, please clone this repository to an environment that thas the Google Cloud SDK installed and initialized. [(Install and initialize the Cloud SDK)](https://cloud.google.com/sdk/docs/install)
@@ -44,15 +44,15 @@ gcloud config set project <PROJECT_ID>
44
44
```
45
45
46
46
### Enabling APIs
47
-
This service requires enabling the following Cloud APIs for a successful deployment and lifecycle.
47
+
GroupSync requires enabling the following Cloud APIs for a successful deployment and lifecycle.
48
48
-[Cloud Run API](https://console.cloud.google.com/apis/api/run.googleapis.com/overview)
@@ -71,9 +71,9 @@ This service requires enabling the following Cloud APIs for a successful deploym
71
71
```
72
72
73
73
### Creating the Service Account
74
-
A service account must be created and granted the proper IAM permissions in order for the service to have appropriate credentials and permissions to access APIs, IAM groups and database users.
74
+
A service account must be created and granted the proper IAM permissions in order for the GroupSync service to have appropriate credentials and permissions to access APIs, IAM groups and database users.
75
75
76
-
The following commands will create a service account and grant it the proper IAM roles and permissions for the service to run successfully.
76
+
The following commands will create a service account and grant it the proper IAM roles and permissions for the GroupSync service to run successfully.
77
77
78
78
Replace the following values:
79
79
-`SERVICE_ACCOUNT_ID`: The ID (name) for the service account.
#### Assigning Group Administrator Role to Service Account
116
-
To properly allow read-access of an organization's IAM group members (i.e. which IAM users belong within a specific IAM group) within the service, we need to assign the Google Workspace Group Administrator Role to the service account created above. This will allow the service account to properly call the [List Members Discovery API](https://developers.google.com/admin-sdk/directory/reference/rest/v1/members/list) to keep track of the IAM members being managed through this service.
116
+
To properly allow read-access of an organization's IAM group members (i.e. which IAM users belong within a specific IAM group) within the GroupSync service, we need to assign the Google Workspace Group Administrator Role to the service account created above. This will allow the service account to properly call the [List Members Discovery API](https://developers.google.com/admin-sdk/directory/reference/rest/v1/members/list) to keep track of the IAM members being managed through this service.
117
117
118
118
To assign the Group Administator Role to the service account follow these four quick steps. ([How to Assign Group Administrator Role](https://cloud.google.com/identity/docs/how-to/setup#auth-no-dwd))
**NOTE**: The above command is required for each IAM group and the binding is added at the project level. Therefore, if an IAM group is being synced with Cloud SQL instances across several different projects, the command will need to be run for each project.
136
136
137
137
### Configuring Cloud SQL Instances
138
-
This service requires Cloud SQL instances to be already created and to have the `cloudsql_iam_authentication` flag turned **On**. [(See how to enable flag here.)](https://cloud.google.com/sql/docs/mysql/create-edit-iam-instances)
138
+
GroupSync requires Cloud SQL instances to be already created and to have the `cloudsql_iam_authentication` flag turned **On**. [(See how to enable flag here.)](https://cloud.google.com/sql/docs/mysql/create-edit-iam-instances)
139
139
140
140
#### Create a database user for service
141
-
To properly manage the database users on each Cloud SQL instance that is configured with the service, the service needs to GRANT/REVOKE database users the proper role(s) corresponding to their IAM group(s). This is achieved by creating an IAM database authenticated service account user on each instance using the service account previously created. This will allow the service account to authenticate to the instance(s) while running the service through the [Cloud SQL Python Connector](https://github.com/GoogleCloudPlatform/cloud-sql-python-connector).
141
+
To properly manage the database users on each Cloud SQL instance that is configured with GroupSync, the service needs to GRANT/REVOKE database users the proper role(s) corresponding to their IAM group(s). This is achieved by creating an IAM database authenticated service account user on each instance using the service account previously created. This will allow the service account to authenticate to the instance(s) while running the GroupSync service through the [Cloud SQL Python Connector](https://github.com/GoogleCloudPlatform/cloud-sql-python-connector).
142
142
143
143
Add the service account as an IAM authenticated database user on each Cloud SQL instance that needs managing through IAM groups. Can be done both manually through the Google Cloud Console or through the following `gcloud` command.
#### Granting Database Permissions to the Service Account's Database User
155
-
For the service to run smoothly it needs the IAM service account database user to be granted several permissions on all Cloud SQL instances that the user was added to above. This allows for the service to read usernames of other database users and GRANT/REVOKE the group role(s) appropriately.
155
+
For GroupSync to run smoothly it needs the IAM service account database user to be granted several permissions on all Cloud SQL instances that the user was added to above. This allows for the GroupSync service to read usernames of other database users and GRANT/REVOKE the group role(s) appropriately.
156
156
157
157
Connect to all Cloud SQL instances in question with an admin user or another database user with appropriate permissions for the following commands. ([Connecting to an Instance](https://cloud.google.com/sql/docs/mysql/connect-overview))
158
158
@@ -239,7 +239,7 @@ There is an additional optional parameter `"log_level"` for the JSON payload whi
239
239
It is recommended to save your JSON payload as a `.json` file (ex. "config.json").
240
240
241
241
### Creating a Cloud Scheduler Job
242
-
An example command creating a Cloud Scheduler job to run the IAM database authentication service for IAM groups and Cloud SQL instances can be seen below.
242
+
An example command creating a Cloud Scheduler job to call the Cloud Run service to sync IAM groups and Cloud SQL instances can be seen below.
243
243
244
244
**NOTE:** If error occurs mentioning "...does not contain App Engine application", simply run `gcloud app create` and try running command again.
245
245
@@ -274,7 +274,7 @@ The Cloud Run service maps each IAM group configured in the JSON payload into a
274
274
The name of the mapped IAM group database role is the email of the IAM group without everything after and including the **"@"** sign of the IAM group email.
275
275
(Ex. IAM group with email "example-group@test.com", would map to a database role **"example-group"** on each Cloud SQL instance it is configured with.)
276
276
277
-
The service verifies that a group role exists or creates one on the database if it does not exist. It is recommended to configure the Cloud Scheduler job(s) and after having it triggered **at least** once, have a Database Administrator or project admin verify the creation of the group roles and **GRANT** the group roles the appropriate privileges on each Cloud SQL instance that should be inherited by database users of those IAM groups on all consecutive Cloud Scheduler runs.
277
+
The Cloud Run service verifies that a group role exists or creates one on the database if it does not exist. It is recommended to configure the Cloud Scheduler job(s) and after having it triggered **at least** once, have a Database Administrator or project admin verify the creation of the group roles and **GRANT** the group roles the appropriate privileges on each Cloud SQL instance that should be inherited by database users of those IAM groups on all consecutive Cloud Scheduler runs.
278
278
279
279
To verify the creation of group roles after Cloud Scheduler has triggered at least once, the following command can be run for **MySQL** instances (**PostgreSQL** instances require connecting to the database to verify):
280
280
@@ -285,17 +285,19 @@ gcloud sql users list --instance=<INSTANCE_NAME>
285
285
```
286
286
The above command should return a list of all database users on the configured instance, with the **new** group role(s) showing up within the list.
287
287
288
-
**NOTE:** It is up to a Database Administrator or project admin to configure the proper privileges on each group role. The service will then be able to grant or revoke each group role with privileges to the proper database users.
288
+
**NOTE:** It is up to a Database Administrator or project admin to configure the proper privileges on each group role. GroupSync will then be able to grant or revoke each group role with privileges to the proper database users.
289
289
290
290
## Running Service with Private IP Cloud SQL Connections
291
-
This service does work for Private IP database connections however, there are some additional configurations needed and some limitations to mention.
291
+
GroupSync does work for Private IP database connections however, there are some additional configurations needed and some limitations to mention.
292
292
293
-
To run this service with Private IP, first make sure all Cloud SQL instances that are going to be connected to have a Private IP address configured. ([Configure Private IP for Cloud SQL](https://cloud.google.com/sql/docs/mysql/configure-private-ip))
293
+
To run the GroupSync service with Private IP, first make sure all Cloud SQL instances that are going to be connected to have a Private IP address configured. ([Configure Private IP for Cloud SQL](https://cloud.google.com/sql/docs/mysql/configure-private-ip))
294
294
295
295
Private IP Cloud SQL instance(s) should be connected to a [VPC Network](https://cloud.google.com/vpc/docs/using-vpc) which can be accessed securely via Cloud Run using [Serverless VPC Access](https://console.cloud.google.com/networking/connectors) which creates a VPC Connector.
296
296
297
297
Thie VPC Connector can be attached to the Cloud Run service previously created to allow Private IP connections to the Cloud SQL instances on the **same VPC Network**.
Update the Cloud Run service with a VPC Connector.
300
302
301
303
Replace the following values:
@@ -311,11 +313,11 @@ Multiple different Cloud Scheduler and Cloud Run configurations can be configure
311
313
312
314
A Cloud Scheduler job maps which IAM group(s) and hence which IAM users to manage permissions for any given Cloud SQL instance(s). When configuring a Cloud Scheduler job, all IAM groups listed in the JSON body will be mapped to all Cloud SQL instances in the JSON body. Therefore, for custom configurations where certain IAM groups need to be mapped to one instance, and other IAM groups to a different Cloud SQL instance, the solution is to deploy multiple Cloud Scheduler jobs.
A single Cloud Run service can be used for multiple Cloud Scheduler jobs across an organization's different Google Cloud projects as long as they allow Public IP connections.
317
319
318
320
### Private IP Configurations
319
321
**NOTE:** For custom configurations with Private IP connections, multiple Cloud Run services may be required. Since Private IP configurations require the Cloud Run service and Cloud SQL instances to be connected to the same VPC network, thus for different projects with different VPC networks, a different Cloud Run service will be needed for each.
0 commit comments